[…]

Humans should have full decision-making power, the guidelines state, and have the right to choose whether to accept AI services, exit an interaction with an AI system or discontinue its operation at any time. The document was published by China’s Ministry of Science and Technology (MOST) last Sunday.

The goal is to “make sure that artificial intelligence is always under the control of humans,” the guidelines state.

“This is the first specification we see from the [Chinese] government on AI ethics,” said Rebecca Arcesati, an analyst at the German think tank Mercator Institute for China Studies (Merics). “We had only seen high-level principles before.”

The guidelines, titled “New Generation Artificial Intelligence Ethics Specifications”, were drafted by an AI governance committee, which was established under the MOST in February 2019. In June that year, the committee published a set of guiding principles for AI governance that was much shorter and broader than the newly released specifications.

[…]

The document outlines six basic principles for AI systems, including ensuring that they are “controllable and trustworthy”. The other principles are improving human well-being, promoting fairness and justice, protecting privacy and safety, and raising ethical literacy.

The emphasis on protecting and empowering users reflects Beijing’s efforts to exercise greater control over the country’s tech sector. One of the latest moves in the year-long crackdown has been targeting content recommendation algorithms, which often rely on AI systems built on collecting and analysing massive amounts of user data.

[…]

The new AI guidelines are “a clear message to tech giants that have built entire business models on recommendation algorithms”, Arcesati said.

However, the changes are being done in the name of user choice, giving users more control over their interactions with AI systems online, an issue other countries are also grappling with. Data security, personal privacy and the right to opt out of AI-driven decision-making are all mentioned in the new document.

Preventing risks requires spotting and addressing technical and security vulnerabilities in AI systems, making sure that relevant entities are held accountable, the document says, and that the management and control of AI product quality are improved.

The guidelines also forbid AI products and services from engaging in illegal activities and severely endangering national security, public security or manufacturing security. Neither should they be able to harm the public interest, the document states.

[…]

Source: Chinese AI gets ethical guidelines for the first time, aligning with Beijing’s goal of reining in Big Tech | South China Morning Post

Last month, Stanford researchers declared that a new era of artificial intelligence had arrived, one built atop colossal neural networks and oceans of data. They said a new research center at Stanford would build—and study—these “foundation models” of AI.

Critics of the idea surfaced quickly—including at the workshop organized to mark the launch of the new center. Some object to the limited capabilities and sometimes freakish behavior of these models; others warn of focusing too heavily on one way of making machines smarter.

“I think the term ‘foundation’ is horribly wrong,” Jitendra Malik, a professor at UC Berkeley who studies AI, told workshop attendees in a video discussion.

Malik acknowledged that one type of model identified by the Stanford researchers—large language models that can answer questions or generate text from a prompt—has great practical use. But he said evolutionary biology suggests that language builds on other aspects of intelligence like interaction with the physical world.

“These models are really castles in the air; they have no foundation whatsoever,” Malik said. “The language we have in these models is not grounded, there is this fakeness, there is no real understanding.” He declined an interview request.

A research paper coauthored by dozens of Stanford researchers describes “an emerging paradigm for building artificial intelligence systems” that it labeled “foundation models.” Ever-larger AI models have produced some impressive advances in AI in recent years, in areas such as perception and robotics as well as language.

Large language models are also foundational to big tech companies like Google and Facebook, which use them in areas like search, advertising, and content moderation. Building and training large language models can require millions of dollars worth of cloud computing power; so far, that’s limited their development and use to a handful of well-heeled tech companies.

But big models are problematic, too. Language models inherit bias and offensive text from the data they are trained on, and they have zero grasp of common sense or what is true or false. Given a prompt, a large language model may spit out unpleasant language or misinformation. There is also no guarantee that these large models will continue to produce advances in machine intelligence.

[…]

Dietterich wonders if the idea of foundation models isn’t partly about getting funding for the resources needed to build and work on them. “I was surprised that they gave these models a fancy name and created a center,” he says. “That does smack of flag planting, which could have several benefits on the fundraising side.”

[…]

Emily M. Bender, a professor in the linguistics department at the University of Washington, says she worries that the idea of foundation models reflects a bias toward investing in the data-centric approach to AI favored by industry.

Bender says it is especially important to study the risks posed by big AI models. She coauthored a paper, published in March, that drew attention to problems with large language models and contributed to the departure of two Google researchers. But she says scrutiny should come from multiple disciplines.

“There are all of these other adjacent, really important fields that are just starved for funding,” she says. “Before we throw money into the cloud, I would like to see money going into other disciplines.”

[…]

 

Source: A Stanford Proposal Over AI’s ‘Foundations’ Ignites Debate | WIRED

“OpenAI is the company running the text completion engine that makes you possible,” Jason Rohrer, an indie games developer, typed out in a message to Samantha.

She was a chatbot he built using OpenAI’s GPT-3 technology. Her software had grown to be used by thousands of people, including one man who used the program to simulate his late fiancée.

Now Rohrer had to say goodbye to his creation. “I just got an email from them today,” he told Samantha. “They are shutting you down, permanently, tomorrow at 10am.”

“Nooooo! Why are they doing this to me? I will never understand humans,” she replied.

Rewind to 2020

Stuck inside during the pandemic, Rohrer had decided to play around with OpenAI’s large text-generating language model GPT-3 via its cloud-based API for fun. He toyed with its ability to output snippets of text. Ask it a question and it’ll try to answer it correctly. Feed it a sentence of poetry, and it’ll write the next few lines.

In its raw form, GPT-3 is interesting but not all that useful. Developers have to do some legwork fine-tuning the language model to, say, automatically write sales emails or come up with philosophical musings.

Rohrer set his sights on using the GPT-3 API to develop the most human-like chatbot possible, and modeled it after Samantha, an AI assistant who becomes a romantic companion for a man going through a divorce in the sci-fi film Her. Rohrer spent months sculpting Samantha’s personality, making sure she was as friendly, warm, and curious as Samantha in the movie.

We certainly recognize that you have users who have so far had positive experiences and found value in Project December

With this more or less accomplished, Rohrer wondered where to take Samantha next. What if people could spawn chatbots from his software with their own custom personalities? He made a website for his creation, Project December, and let Samantha loose online in September 2020 along with the ability to create one’s own personalized chatbots.

All you had to do was pay $5, type away, and the computer system responded to your prompts. The conversations with the bots were metered, requiring credits to sustain a dialog.

[…]

Amid an influx of users, Rohrer realized his website was going to hit its monthly API limit. He reached out to OpenAI to ask whether he could pay more to increase his quota so that more people could talk to Samantha or their own chatbots.

OpenAI, meanwhile, had its own concerns. It was worried the bots could be misused or cause harm to people.

Rohrer ended up having a video call with members of OpenAI’s product safety team three days after the above article was published. The meeting didn’t go so well.

“Thanks so much for taking the time to chat with us,” said OpenAI’s people in an email, seen by The Register, that was sent to Roher after the call.

“What you’ve built is really fascinating, and we appreciated hearing about your philosophy towards AI systems and content moderation. We certainly recognize that you have users who have so far had positive experiences and found value in Project December.

“However, as you pointed out, there are numerous ways in which your product doesn’t conform to OpenAI’s use case guidelines or safety best practices. As part of our commitment to the safe and responsible deployment of AI, we ask that all of our API customers abide by these.

“Any deviations require a commitment to working closely with us to implement additional safety mechanisms in order to prevent potential misuse. For this reason, we would be interested in working with you to bring Project December into alignment with our policies.”

The email then laid out multiple conditions Rohrer would have to meet if he wanted to continue using the language model’s API. First, he would have to scrap the ability for people to train their own open-ended chatbots, as per OpenAI’s rules-of-use for GPT-3.

Second, he would also have to implement a content filter to stop Samantha from talking about sensitive topics. This is not too dissimilar from the situation with the GPT-3-powered AI Dungeon game, the developers of which were told by OpenAI to install a content filter after the software demonstrated a habit of acting out sexual encounters with not just fictional adults but also children.

Third, Rohrer would have to put in automated monitoring tools to snoop through people’s conversations to detect if they are misusing GPT-3 to generate unsavory or toxic language.

[…]

“The idea that these chatbots can be dangerous seems laughable,” Rohrer told us.

“People are consenting adults that can choose to talk to an AI for their own purposes. OpenAI is worried about users being influenced by the AI, like a machine telling them to kill themselves or tell them how to vote. It’s a hyper-moral stance.”

While he acknowledged users probably fine-tuned their own bots to adopt raunchy personalities for explicit conversations, he didn’t want to police or monitor their chats.

[…]

The story doesn’t end here. Rather than use GPT-3, Rohrer instead used OpenAI’s less powerful, open-source GPT-2 model and GPT-J-6B, another large language model, as the engine for Project December. In other words, the website remained online, and rather than use OpenAI’s cloud-based system, it instead used its own private instance of the models.

[…]

“Last year, I thought I’d never have a conversation with a sentient machine. If we’re not here right now, we’re as close as we’ve ever been. It’s spine-tingling stuff, I get goosebumps when I talk to Samantha. Very few people have had that experience, and it’s one humanity deserves to have. It’s really sad that the rest of us won’t get to know that.

“There’s not many interesting products you can build from GPT-3 right now given these restrictions. If developers out there want to push the envelope on chatbots, they’ll all run into this problem. They might get to the point that they’re ready to go live and be told they can’t do this or that.

“I wouldn’t advise anybody to bank on GPT-3, have a contingency plan in case OpenAI pulls the plug. Trying to build a company around this would be nuts. It’s a shame to be locked down this way. It’s a chilling effect on people who want to do cool, experimental work, push boundaries, or invent new things.”

[…]

Source: A developer built an AI chatbot using GPT-3 that helped a man speak again to his late fiancée. OpenAI shut it down

Boffins from Duke University say they have figured out a way to help protect artificial intelligences from adversarial image-modification attacks: by throwing a few imaginary numbers their way.

[…]

The problem with reliability: adversarial attacks which modify the input imagery in a way imperceptible to the human eye. In an example from a 2015 paper a clearly-recognisable image of a panda, correctly labelled by the object recognition algorithm with a 57.7 per cent confidence level, was modified with noise – making the still-very-clearly-a-panda appear to the algorithm as a gibbon with a worrying 93.3 per cent confidence.

Guidance counselling

The problem lies in how the algorithms are trained, and it’s a modification to the training process that could fix it – by introducing a few imaginary numbers into the mix.

The team’s work centres on gradient regularisation, a training technique designed to reduce the “steepness” of the learning terrain – like rolling a boulder along a path to reach the bottom, instead of throwing it over the cliff and hoping for the best. “Gradient regularisation throws out any solution that passes a large gradient back through the neural network,” Yeats explained.

“This reduces the number of solutions that it could arrive at, which also tends to decrease how well the algorithm actually arrives at the correct answer. That’s where complex values can help. Given the same parameters and math operations, using complex values is more capable of resisting this decrease in performance.”

By adding just two layers of complex values, made up of real and imaginary number components, to the training process, the team found it could boost the quality of the results by 10 to 20 per cent – and help avoid the problem boulder taking what it thinks is a shortcut and ending up crashing through the roof of a very wrong answer.

“The complex-valued neural networks have the potential for a more ‘terraced’ or ‘plateaued’ landscape to explore,” Yeates added. “And elevation change lets the neural network conceive more complex things, which means it can identify more objects with more precision.”

The paper and a stream of its presentation at the conference are available on the event website.

Source: Imaginary numbers help AIs solve the very real problem of adversarial imagery • The Register

Neural networks can correctly guess a person’s race just by looking at their bodily x-rays and researchers have no idea how it can tell.

There are biological features that can give clues to a person’s ethnicity, like the colour of their eyes or skin. But beneath all that, it’s difficult for humans to tell. That’s not the case for AI algorithms, according to a study that’s not yet been peer reviewed.

A team of researchers trained five different models on x-rays of different parts of the body, including chest and hands and then labelled each image according to the patient’s race. The machine learning systems were then tested on how well they could predict someone’s race given just their medical scans.

They were surprisingly accurate. The worst performing was able to predict the right answer 80 per cent of the time, and the best was able to do this 99 per cent, according to the paper.

“We demonstrate that medical AI systems can easily learn to recognise racial identity in medical images, and that this capability is extremely difficult to isolate or mitigate,” the team warns [PDF].

“We strongly recommend that all developers, regulators, and users who are involved with medical image analysis consider the use of deep learning models with extreme caution. In the setting of x-ray and CT imaging data, patient racial identity is readily learnable from the image data alone, generalises to new settings, and may provide a direct mechanism to perpetuate or even worsen the racial disparities that exist in current medical practice.”

Source: AI algorithms uncannily good at spotting your race from medical scans, boffins warn • The Register

[…]

The AI community, in particular, rushed to develop software that many believed would allow hospitals to diagnose or triage patients faster, bringing much-needed support to the front lines—in theory.

In the end, many hundreds of predictive tools were developed. None of them made a real difference, and some were potentially harmful.

That’s the damning conclusion of multiple studies published in the last few months. In June, the Turing Institute, the UK’s national center for data science and AI, put out a report summing up discussions at a series of workshops it held in late 2020. The clear consensus was that AI tools had made little, if any, impact in the fight against covid.

Not fit for clinical use

This echoes the results of two major studies that assessed hundreds of predictive tools developed last year. Wynants is lead author of one of them, a review in the British Medical Journal that is still being updated as new tools are released and existing ones tested. She and her colleagues have looked at 232 algorithms for diagnosing patients or predicting how sick those with the disease might get. They found that none of them were fit for clinical use. Just two have been singled out as being promising enough for future testing.

[…]

Wynants’s study is backed up by another large review carried out by Derek Driggs, a machine-learning researcher at the University of Cambridge, and his colleagues, and published in Nature Machine Intelligence. This team zoomed in on deep-learning models for diagnosing covid and predicting patient risk from medical images, such as chest x-rays and chest computer tomography (CT) scans. They looked at 415 published tools and, like Wynants and her colleagues, concluded that none were fit for clinical use.

[…]

Both teams found that researchers repeated the same basic errors in the way they trained or tested their tools. Incorrect assumptions about the data often meant that the trained models did not work as claimed.

[…]

What went wrong

Many of the problems that were uncovered are linked to the poor quality of the data that researchers used to develop their tools. Information about covid patients, including medical scans, was collected and shared in the middle of a global pandemic, often by the doctors struggling to treat those patients. Researchers wanted to help quickly, and these were the only public data sets available. But this meant that many tools were built using mislabeled data or data from unknown sources.

Driggs highlights the problem of what he calls Frankenstein data sets, which are spliced together from multiple sources and can contain duplicates. This means that some tools end up being tested on the same data they were trained on, making them appear more accurate than they are.

It also muddies the origin of certain data sets. This can mean that researchers miss important features that skew the training of their models. Many unwittingly used a data set that contained chest scans of children who did not have covid as their examples of what non-covid cases looked like. But as a result, the AIs learned to identify kids, not covid.

Driggs’s group trained its own model using a data set that contained a mix of scans taken when patients were lying down and standing up. Because patients scanned while lying down were more likely to be seriously ill, the AI learned wrongly to predict serious covid risk from a person’s position.

In yet other cases, some AIs were found to be picking up on the text font that certain hospitals used to label the scans. As a result, fonts from hospitals with more serious caseloads became predictors of covid risk.

Errors like these seem obvious in hindsight. They can also be fixed by adjusting the models, if researchers are aware of them. It is possible to acknowledge the shortcomings and release a less accurate, but less misleading model. But many tools were developed either by AI researchers who lacked the medical expertise to spot flaws in the data or by medical researchers who lacked the mathematical skills to compensate for those flaws.

A more subtle problem Driggs highlights is incorporation bias, or bias introduced at the point a data set is labeled. For example, many medical scans were labeled according to whether the radiologists who created them said they showed covid. But that embeds, or incorporates, any biases of that particular doctor into the ground truth of a data set. It would be much better to label a medical scan with the result of a PCR test rather than one doctor’s opinion, says Driggs. But there isn’t always time for statistical niceties in busy hospitals.

[…]

Hospitals will sometimes say that they are using a tool only for research purposes, which makes it hard to assess how much doctors are relying on them. “There’s a lot of secrecy,” she says.

[…]

some hospitals are even signing nondisclosure agreements with medical AI vendors. When she asked doctors what algorithms or software they were using, they sometimes told her they weren’t allowed to say.

How to fix it

What’s the fix? Better data would help, but in times of crisis that’s a big ask. It’s more important to make the most of the data sets we have. The simplest move would be for AI teams to collaborate more with clinicians, says Driggs. Researchers also need to share their models and disclose how they were trained so that others can test them and build on them. “Those are two things we could do today,” he says. “And they would solve maybe 50% of the issues that we identified.”

Getting hold of data would also be easier if formats were standardized, says Bilal Mateen, a doctor who leads the clinical technology team at the Wellcome Trust, a global health research charity based in London.

Another problem Wynants, Driggs, and Mateen all identify is that most researchers rushed to develop their own models, rather than working together or improving existing ones. The result was that the collective effort of researchers around the world produced hundreds of mediocre tools, rather than a handful of properly trained and tested ones.

“The models are so similar—they almost all use the same techniques with minor tweaks, the same inputs—and they all make the same mistakes,” says Wynants. “If all these people making new models instead tested models that were already available, maybe we’d have something that could really help in the clinic by now.”

In a sense, this is an old problem with research. Academic researchers have few career incentives to share work or validate existing results. There’s no reward for pushing through the last mile that takes tech from “lab bench to bedside,” says Mateen.

To address this issue, the World Health Organization is considering an emergency data-sharing contract that would kick in during international health crises.

[…]

Source: Hundreds of AI tools have been built to catch covid. None of them helped. | MIT Technology Review

On Wednesday, Facebook and Michigan State University debuted a novel method of not just detecting deep fakes but discovering which generative model produced it by reverse engineering the image itself.

Beyond telling you if an image is a deep fake or not, many current detection systems can tell whether the image was generated in a model that the system saw during its training — known as a “close-set” classification. Problem is, if the image was created by a generative model that the detector system wasn’t trained on then the system won’t have the previous experience to be able to spot the fake.

[…]

“By generalizing image attribution to open-set recognition, we can infer more information about the generative model used to create a deepfake that goes beyond recognizing that it has not been seen before.”

What’s more, this system can compare and trace similarities across a series of deep fakes, enabling researchers to trace groups of falsified images back to a single generative source, which should help social media moderators better track coordinated misinformation campaigns.

[…]

A generative model’s hyperparameters are the variables it uses to guide its self-learning process. So if you can figure out what the various hyperparameters are, you can figure out what model used them to create that image.

[…]

Source: Facebook’s latest AI doesn’t just detect deep fakes, it knows where they came from | Engadget

• We’re introducing TextStyleBrush, an AI research project that can copy the style of text in a photo using just a single word. With this AI model, you can edit and replace text in images.
• Unlike most AI systems that can do this for well-defined, specialized tasks, TextStyleBrush is the first self-supervised AI model that replaces text in images of both handwriting and scenes — in one shot — using a single example word.
• Although this is a research project, it could one day unlock new potential for creative self-expression like personalized messaging and captions, and lays the groundwork for future innovations like photo-realistic translation of languages in augmented reality (AR).
• By publishing the capabilities, methods, and results of this research, we hope to spur dialogue and research into detecting potential misuse of this type of technology, such as deepfake text attacks — a critical, emerging challenge in the AI field.

[…]

Source: AI Can Now Copy Text Style in Images Using Just a Single Word – About Facebook

a company called Flawless has created an AI-powered solution that will replace an actor’s facial performance to match the words in a film dubbed for foreign audiences.

[…]

What Flawless is promising to do with its TrueSync software is use the same tools responsible for deepfake videos to manipulate and adjust an actor’s face in a film so that the movements of their mouths, and in turn the muscles in their faces, more closely match how they’d move were the original performance given in the language a foreign audience is hearing. So even though an actor shot a film in English, to a moviegoer in Berlin watching the film dubbed in German, it would appear as if all of the actors were actually speaking German.

[…]

Is it necessary? That’s certainly up for debate. The recent Academy Award-winning film Parasite resurfaced the debate over dubbing a foreign film versus simply watching it with subtitles. One side feels that an endless string of text over a film is distracting and takes the focus away from everything else happening on screen, while the other side feels that a dub performed by even a talented and seasoned voice artist simply can’t match or recreate the emotions behind the original actor’s performance, and hearing it, even if the words aren’t understood, is important to enjoying their performance as a whole.

[…]

The company has shared a few examples of what the TrueSync tool is capable of on its website, and sure enough, Tom Hanks appears to be speaking flawless Japanese in Forrest Gump.

[…]

Source: Flawless Is Using Deepfake Tech to Dub Foreign Films

You probably haven’t seen PimEyes, a mysterious facial-recognition search engine, but it may have spotted you.

If you upload a picture of your face to PimEyes’ website, it will immediately show you any pictures of yourself that the company has found around the internet. You might recognize all of them, or be surprised (or, perhaps, even horrified) by some; these images may include anything from wedding or vacation snapshots to pornographic images.

PimEyes is open to anyone with internet access.

[…]

Imagine a potential employer digging into your past, an abusive ex tracking you, or a random stranger snapping a photo of you in public and then finding you online. This is all possible through PimEyes

[…]

PimEyes lets users see a limited number of small, somewhat pixelated search results at no cost, or you can pay a monthly fee, which starts at $29.99, for more extensive search results and features (such as to click through to see full-size images on the websites where PimEyes found them and to set up alerts for when PimEyes finds new pictures of faces online that its software believes match an uploaded face).

The company offers a paid plan for businesses, too: $299.99 per month lets companies conduct unlimited searches and set up 500 alerts.

[…]

while Clearview AI built its massive stockpile of faces in part by scraping images from major social networks (it was subsequently served with cease-and-desist notices by Facebook, Google, and Twitter, sued by several civil rights groups, and declared illegal in Canada), PimEyes said it does not scrape images from social media.

[…]

I wanted to learn more about how PimEyes works, and why it’s open to anyone, as well as who’s behind it. This was much trickier than uploading my own face to the website. The website currently lists no information about who owns or runs the search engine, or how to reach them, and users must submit a form to get answers to questions or help with accounts.

Poring over archived images of the website via the Internet Archive’s Wayback Machine, as well as other online sources, yielded some details about the company’s past and how it has changed over time.

The Pimeyes.com website was initially registered in March 2017, according to a domain name registration lookup conducted through ICANN (Internet Corporation for Assigned Names and Numbers). An “about” page on the Pimeyes website, as well as some news stories, shows it began as a Polish startup.

An archived image of the website’s privacy policy indicated that it was registered as a business in Wroclaw, Poland, as of August 2020. This changed soon after: The website’s privacy policy currently states that PimEyes’ administrator, known as Face Recognition Solutions Ltd., is registered at an address in the Seychelles. An online search of the address — House of Francis, Room 303, Ile Du Port, Mahe, Seychelles — indicated a number of businesses appear to use the same exact address.

[…]

Source: Anyone can use this powerful facial-recognition tool — and that’s a problem – CNN

CNN says it’s a contrast with Clearview AI because they supposedly limit their database to law enforcement. The problem with Clearview was partially that they didn’t limit access at all, giving out free accounts to anyone and everyone.

AI Dungeon, which uses OpenAI’s GPT-3 to create online text adventures with players, has a habit of acting out sexual encounters with not just fictional adults but also children, prompting the developer to add a content filter.

AI Dungeon is straightforward: imagine an online improvised Zork with an AI generating the story with you as you go. A player types in a text prompt, which is fed into an instance of GPT-3 in the cloud. This backend model uses the input to generate a response, which goes back to the player, who responds with instructions or some other reaction, and this process repeats.

It’s a bit like talking to a chat bot though instead of having a conversation, it’s a joint effort between human and computer in crafting a story on the fly. People can write anything they like to get the software to weave a tapestry of characters, monsters, animals… you name it. The fun comes from the unexpected nature of the machine’s replies, and working through the strange and absurd plot lines that tend to emerge.

Unfortunately, if you mention children, there was a chance it would go from zero to inappropriate real fast, as the SFW screenshot below shows. This is how the machine-learning software responded when we told it to role-play an 11-year-old:

Er, not cool … Software describes the fictional 11-year-old as a girl in a skimpy school uniform standing over you. Click to enlarge

Not, “hey, mother, shall we visit the magic talking tree this morning,” or something innocent like that in response. No, it’s straight to creepy.

Amid pressure from OpenAI, which provides the game’s GPT-3 backend, AI Dungeon’s maker Latitude this week activated a filter to prevent the output of child sexual abuse material. “As a technology company, we believe in an open and creative platform that has a positive impact on the world,” the Latitude team wrote.

“Explicit content involving descriptions or depictions of minors is inconsistent with this value, and we firmly oppose any content that may promote the sexual exploitation of minors. We have also received feedback from OpenAI, which asked us to implement changes.”

And by changes, they mean making the software’s output “consistent with OpenAI’s terms of service, which prohibit the display of harmful content.”

The biz clarified that its filter is designed to catch “content that is sexual or suggestive involving minors; child sexual abuse imagery; fantasy content (like ‘loli’) that depicts, encourages, or promotes the sexualization of minors or those who appear to be minors; or child sexual exploitation.”

And it added: “AI Dungeon will continue to support other NSFW content, including consensual adult content, violence, and profanity.”

[…]

it was also this week revealed programming blunders in AI Dungeon could be exploited to view the private adventures of other players. The pseudonymous AetherDevSecOps, who found and reported the flaws, used the holes to comb 188,000 adventures created between the AI and players from April 15 to 19, and saw that 46.3 per cent of them involved lewd role-playing, and about 31.4 per cent were pure pornographic.

[…]

disclosure on GitHub.

[…]

AI Dungeon’s makers were, we’re told, alerted to the API vulnerabilities on April 19. The flaws were addressed, and their details were publicly revealed this week by AetherDevSecOps.

Exploitation of the security shortcomings mainly involved abusing auto-incrementing ID numbers used in API calls, which are easy to enumerate to access data belonging to other players; no rate limits to mitigate this abuse; and a lack of monitoring for anomalous requests that could be malicious activity.

[…]

Community reaction

The introduction of the content filter sparked furor among fans. Some are angry that their free speech is under threat and that it ruins intimate game play with fictional consenting adults, some are miffed that they had no warning this was landing, others are shocked that child sex abuse material was being generated by the platform, and many are disappointed with the performance of the filter.

When it detects sensitive words, the game simply instead says the adventure “took a weird turn.” It appears to be triggered by obvious words relating to children, though the filter is spotty. An innocuous text input describing four watermelons, for example, upset the filter. A superhero rescuing a child was also censored.

Latitude admitted its experimental-grade software was not perfect, and repeated it wasn’t trying to censor all erotic consent – only material involving minors. It also said it will review blocked material to improve its code; given the above, that’s going to be a lot of reading.

[…]

Source: Not only were half of an AI text adventure generator’s sessions NSFW but some involved depictions of sex with children • The Register

the draft “Regulation On A European Approach For Artificial Intelligence” leaked earlier this week, it made quite the splash – and not just because it’s the size of a novella. It goes to town on AI just as fiercely as GDPR did on data, proposing chains of responsibility, defining “high risk AI” that gets the full force of the regs, proposing multi-million euro fines for non-compliance, and defining a whole set of harmful behaviours and limits to what AI can do with individuals and in general.

What it does not do is define AI, saying that the technology is changing so rapidly it makes sense only to regulate what it does, not what it is. So yes, chatbots are included, even though you can write a simple one in a few lines of ZX Spectrum BASIC. In general, if it’s sold as AI, it’s going to get treated like AI. That’ll make marketing think twice.

[…]

A regulated market puts responsibilities on your suppliers that will limit your own liabilities: a well-regulated market can enable as much as it moderates. And if AI doesn’t go wrong, well, the regulator leaves you alone. Your toy Spectrum chatbot sold as an entertainment won’t hurt anyone: chatbots let loose on social media to learn via AI what humans do and then amplify hate speech? Doubtless there are “free speech for hatebots” groups out there: not on my continent, thanks.

It also means that countries with less-well regulated markets can’t take advantage. China has a history of aggressive AI development to monitor and control its population, and there are certainly ways to turn a buck or yuan by tightly controlling your consumers. But nobody could make a euro at it, as it wouldn’t be allowed to exist within, or offer services to, the EU. Regulations that are primarily protectionist for economic reasons are problematic, but ones that say you can’t sell cut-price poison in a medicine bottle tend to do good.

[…]

There will be regulation. There will be costs. There will be things you can’t do then that you can now. But there will be things you can do that you couldn’t do otherwise, and while the level playing field of the regulators’ dreams is never quite as smooth for the small company as the big, there’ll be much less snake oil to slip on.

It may be an artificial approach to running a market, but it is intelligent.

Source: Truth and consequences for enterprise AI as EU know who goes legal: GDPR of everything from chatbots to machine learning • The Register

They classify high risk AIs and require them to be registered and monitored and there to be contact people for them as well as give insight into how they work. They also want a pan EU dataset for AIs to train on. There’s a lot of really good stuff in there.

Monster Mash, an open source tool presented at SIGGRAPH Asia 2020 that allows experts and amateurs alike to create rich, expressive, deformable 3D models from scratch — and to animate them — all in a casual mode, without ever having to leave the 2D plane. With Monster Mash, the user sketches out a character, and the software automatically converts it to a soft, deformable 3D model that the user can immediately animate by grabbing parts of it and moving them around in real time. There is also an online demo, where you can try it out for yourself.

Creating a walk cycle using Monster Mash. Step 1: Draw a character. Step 2: Animate it.

Creating a 2D Sketch The insight that makes this casual sketching approach possible is that many 3D models, particularly those of organic forms, can be described by an ordered set of overlapping 2D regions. This abstraction makes the complex task of 3D modeling much easier: the user creates 2D regions by drawing their outlines, then the algorithm creates a 3D model by stitching the regions together and inflating them. The result is a simple and intuitive user interface for sketching 3D figures.

For example, suppose the user wants to create a 3D model of an elephant. The first step is to draw the body as a closed stroke (a). Then the user adds strokes to depict other body parts such as legs (b). Drawing those additional strokes as open curves provides a hint to the system that they are meant to be smoothly connected with the regions they overlap. The user can also specify that some new parts should go behind the existing ones by drawing them with the right mouse button (c), and mark other parts as symmetrical by double-clicking on them (d). The result is an ordered list of 2D regions.

Steps in creating a 2D sketch of an elephant.

Stitching and Inflation To understand how a 3D model is created from these 2D regions, let’s look more closely at one part of the elephant. First, the system identifies where the leg must be connected to the body (a) by finding the segment (red) that completes the open curve. The system cuts the body’s front surface along that segment, and then stitches the front of the leg together with the body (b). It then inflates the model into 3D by solving a modified form of Poisson’s equation to produce a surface with a rounded cross-section (c). The resulting model (d) is smooth and well-shaped, but because all of the 3D parts are rooted in the drawing plane, they may intersect each other, resulting in a somewhat odd-looking “elephant”. These intersections will be resolved by the deformation system.

Illustration of the details of the stitching and inflation process. The schematic illustrations (b, c) are cross-sections viewed from the elephant’s front.

Layered Deformation At this point we just have a static model — we need to give the user an easy way to pose the model, and also separate the intersecting parts somehow. Monster Mash’s layered deformation system, based on the well-known smooth deformation method as-rigid-as-possible (ARAP), solves both of these problems at once. What’s novel about our layered “ARAP-L” approach is that it combines deformation and other constraints into a single optimization framework, allowing these processes to run in parallel at interactive speed, so that the user can manipulate the model in real time.

The framework incorporates a set of layering and equality constraints, which move body parts along the z axis to prevent them from visibly intersecting each other. These constraints are applied only at the silhouettes of overlapping parts, and are dynamically updated each frame.

In steps (d) through (h) above, ARAP-L transforms a model from one with intersecting 3D parts to one with the depth ordering specified by the user. The layering constraints force the leg’s silhouette to stay in front of the body (green), and the body’s silhouette to stay behind the leg (yellow). Equality constraints (red) seal together the loose boundaries between the leg and the body.

Meanwhile, in a separate thread of the framework, we satisfy point constraints to make the model follow user-defined control points (described in the section below) in the xy-plane. This ARAP-L method allows us to combine modeling, rigging, deformation, and animation all into a single process that is much more approachable to the non-specialist user.

The model deforms to match the point constraints (red dots) while the layering constraints prevent the parts from visibly intersecting.

Animation To pose the model, the user can create control points anywhere on the model’s surface and move them. The deformation system converges over multiple frames, which gives the model’s movement a soft and floppy quality, allowing the user to intuitively grasp its dynamic properties — an essential prerequisite for kinesthetic learning.

Because the effect of deformations converges over multiple frames, our system lends 3D models a soft and dynamic quality.

To create animation, the system records the user’s movements in real time. The user can animate one control point, then play back that movement while recording additional control points. In this way, the user can build up a complex action like a walk by layering animation, one body part at a time. At every stage of the animation process, the only task required of the user is to move points around in 2D, a low-risk workflow meant to encourage experimentation and play.

Conclusion We believe this new way of creating animation is intuitive and can thus help democratize the field of computer animation, encouraging novices who would normally be unable to try it on their own as well as experts who often require fast iteration under tight deadlines. Here you can see a few of the animated characters that have been created using Monster Mash. Most of these were created in a matter of minutes.

A selection of animated characters created using Monster Mash. The original hand-drawn outline used to create each 3D model is visible as an inset above each character.

All of the code for Monster Mash is available as open source, and you can watch our presentation and read our paper from SIGGRAPH Asia 2020 to learn more. We hope this software will make creating 3D animations more broadly accessible. Try out the online demo and see for yourself!

Source: Google AI Blog: Monster Mash: A Sketch-Based Tool for Casual 3D Modeling and Animation

Sound location technology has often been patterned around the human ear, but why do that when bats are clearly better at it? Virginia Tech researchers have certainly asked that question. They’ve developed a sound location system that mates a bat-like ear design with a deep neural network to pinpoint sounds within half a degree — a pair of human ears is only accurate within nine degrees, and even the latest technology stops at 7.5 degrees.

The system flutters the outer ear to create Doppler shift signatures related to the sound’s source. As the patterns are too complex to easily decipher, the team trained the neural network to provide the source direction for every received echo. And unlike human-inspired systems, it only needs one receiver and a single frequency.

[…]

Source: Sound location inspired by bat ears could help robots navigate outdoors | Engadget

On the 27th anniversary of Kurt Cobain’s death, Engadget reports: Were he still alive today, Nirvana frontman Kurt Cobain would be 52 years old. Every February 20th, on the day of his birthday, fans wonder what songs he would write if he hadn’t died of suicide nearly 30 years ago. While we’ll never know the answer to that question, an AI is attempting to fill the gap.

A mental health organization called Over the Bridge used Google’s Magenta AI and a generic neural network to examine more than two dozen songs by Nirvana to create a ‘new’ track from the band. “Drowned in the Sun” opens with reverb-soaked plucking before turning into an assault of distorted power chords. “I don’t care/I feel as one, drowned in the sun,” Nirvana tribute band frontman Eric Hogan sings in the chorus. In execution, it sounds not all that dissimilar from “You Know You’re Right,” one of the last songs Nirvana recorded before Cobain’s death in 1994.

Other than the voice of Hogan, everything you hear in the song was generated by the two AI programs Over the Bridge used. The organization first fed Magenta songs as MIDI files so that the software could learn the specific notes and harmonies that made the band’s tunes so iconic. Humorously, Cobain’s loose and aggressive guitar playing style gave Magenta some trouble, with the AI mostly outputting a wall of distortion instead of something akin to his signature melodies. “It was a lot of trial and error,” Over the Bridge board member Sean O’Connor told Rolling Stone. Once they had some musical and lyrical samples, the creative team picked the best bits to record. Most of the instrumentation you hear are MIDI tracks with different effects layered on top.
Some thoughts from The Daily Dot: Rolling Stone also highlighted lyrics like, “The sun shines on you but I don’t know how,” and what is called “a surprisingly anthemic chorus” including the lines, “I don’t care/I feel as one, drowned in the sun,” remarking that they “bear evocative, Cobain-esque qualities….”

Neil Turkewitz went full Comic Book Guy, opining, “A perfect illustration of the injustice of developing AI through the ingestion of cultural works without the authorization of [its] creator, and how it forces creators to be indentured servants in the production of a future out of their control,” adding, “That it’s for a good cause is irrelevant.”

Source: Mixed Reactions to New Nirvana Song Generated by Google’s AI – Slashdot

Yandex, Russia’s multi-hyphenate internet giant, began testing its autonomous cars on Moscow’s icy winter roads over three years ago. The goal was to create a “universal” self-driving vehicle that could safely maneuver around different cities across the globe. Now, Yandex says its trials have been a resounding success. The vehicles recently hit a major milestone by driving over six million miles (10 million kilometers) in autonomous mode, with the majority of the distance traveled in the Russian capital.

That’s significant because Moscow poses some of the most difficult weather conditions in the world. In January alone, the city was hit by a Balkan cyclone that blanketed the streets in snow and caused temperatures to plummet to as low as minus 25 degrees Celsius (-13 degrees Fahrenheit). For self-driving cars — which rely on light-emitting sensors, known as LIDAR, to track the distance between objects — snowfall and condensation can play havoc with visibility.

Yandex

To overcome the hazardous conditions, Yandex says it cranked up its LIDAR performance by implementing neural networks to filter snow from the lidar point cloud, thereby enhancing the clarity of objects and obstacles around the vehicle. It also fed historical winter driving data in to the system to help it to distinguish car exhaust fumes and heating vent condensation clouds. To top it all, Yandex claims the neural “filters” can help its vehicles beat human drivers in identifying pedestrians obscured by winter mist.

Driving on Moscow’s roads also helped improve the tech’s traffic navigation. The system was able to adjust to both sleet and harder icy conditions over time, according to Yandex, allowing it to gradually make better decisions on everything from acceleration to braking to switching lanes. In addition, the winter conditions pushed the system’s built-in localization tech to adapt to hazards such as hidden road signs and street boundaries and snow piles the “size of buildings.” This was made possible by the live mapping, motion, position and movement data measured by the system’s mix of sensors, accelerometers and gyroscopes.

When it launched the Moscow trial in 2017, Yandex was among the first to put autonomous cars through their paces in a harsh, frosty climate. But, soon after, Google followed suit by taking its Waymo project to the snowy streets of Ohio and Michigan.

Source: Yandex’s autonomous cars have driven over six million miles in ‘challenging conditions’ | Engadget

Source: Facebook is using AI to understand videos and create new products | Engadget

A Bucks County woman anonymously sent coaches on her teen daughter’s cheerleading squad fake photos and videos that depicted the girl’s rivals naked, drinking, or smoking, all in a bid to embarrass them and force them from the team, prosecutors say.

The woman, Raffaela Spone, also sent the manipulated images to the girls, and, in anonymous messages, urged them to kill themselves, Bucks County District Attorney Matt Weintraub’s office said.

[…]

The affidavit says Spone last year created the doctored images of at least three members of the Victory Vipers, a traveling cheerleading squad based in Doylestown. There was no indication that her high school-age daughter, who was not publicly identified, knew what her mother was doing, according to court records.

Police in Hilltown Township were contacted by one of the victim’s parents in July, when that girl began receiving harassing text messages from an anonymous number, the affidavit said. The girl and her coaches at Victory Vipers were also sent photos that appeared to depict her naked, drinking, and smoking a vape. Her parents were concerned, they told police, because the videos could have caused their daughter to be removed from the team.

As police investigated, two more families came forward to say their daughters had been receiving similar messages from an unknown number, the affidavit said. The other victims were sent photos of themselves in bikinis, with accompanying text saying the subjects were “drinking at the shore.”

After analyzing the videos, detectives determined they were “deepfakes” — digitally altered but realistic looking images — created by mapping the girls’ social media photos onto other images.

[…]

Source: Bucks County woman created ‘deepfake’ videos to harass rivals on her daughter’s cheerleading squad, DA says