Paris 2024 Olympics: Concern over French plan for AI surveillance

Under a recent law, police will be able to use CCTV algorithms to pick up anomalies such as crowd rushes, fights or unattended bags.

The law explicitly rules out using facial recognition technology, as adopted by China, for example, in order to trace “suspicious” individuals.

But opponents say it is a thin end of the wedge. Even though the experimental period allowed by the law ends in March 2025, they fear the French government’s real aim is to make the new security provisions permanent.

“We’ve seen this before at previous Olympic Games like in Japan, Brazil and Greece. What were supposed to be special security arrangements for the special circumstances of the games, ended up being normalised,” says Noémie Levain, of the digital rights campaign group La Quadrature du Net (Squaring the Web).

[…]

“We will not – and cannot by law – provide facial recognition, so this is a wholly different operation from what you see in China,” he says.

“What makes us attractive is that we provide security, but within the framework of the law and ethics.”

But according to digital rights activist Noémie Levain, this is only a “narrative” that developers are using to sell their product – knowing full well that the government will almost certainly favour French companies over foreign firms when it comes to awarding the Olympics contracts.

“They say it makes all the difference that here there will be no facial recognition. We say it is essentially the same,” she says.

“AI video monitoring is a surveillance tool which allows the state to analyse our bodies, our behaviour, and decide whether it is normal or suspicious. Even without facial recognition, it enables mass control.

“We see it as just as scary as what is happening in China. It’s the same principle of losing the right to be anonymous, the right to act how we want to act in public, the right not to be watched.”

Source: Paris 2024 Olympics: Concern over French plan for AI surveillance – BBC News

Stability AI releases Stable Doodle, a sketch-to-image tool

Stability AI, the startup behind the image-generating model Stable Diffusion, is launching a new service that turns sketches into images.

The sketch-to-image service, Stable Doodle, leverages the latest Stable Diffusion model to analyze the outline of a sketch and generate a “visually pleasing” artistic rendition of it. It’s available starting today through ClipDrop, a platform Stability acquired in March through its purchase of Init ML, an AI startup founded by ex-Googlers,

[…]

Under the hood, powering Stable Doodle is a Stable Diffusion model — Stable Diffusion XL — paired with a “conditional control solution” developed by one of Tencent’s R&D divisions, the Applied Research Center (ARC). Called T2I-Adapter, the control solution both allows Stable Diffusion XL to accept sketches as input and guides the model to enable better fine-tuning of the output artwork.

[…]

Source: Stability AI releases Stable Doodle, a sketch-to-image tool | TechCrunch

Find it at https://clipdrop.co/stable-doodle

AI System Identified Drug Trafficker by Scanning Driving Patterns

Police in New York recently managed to identify and apprehend a drug trafficker seemingly by magic. The perp in question, David Zayas, was traveling through the small upstate town of Scarsdale when he was pulled over by Westchester County police. When cops searched Zayas’ vehicle they found a large amount of crack cocaine, a gun, and over $34,000 in cash in his vehicle. The arrestee later pleaded guilty to a drug trafficking charge.

How exactly did cops know Zayas fit the bill for drug trafficking?

Forbes reports that authorities used the services of a company called Rekor to analyze traffic patterns regionally and, in the course of that analysis, the program identified Zayas as suspicious.

For years, cops have used license plate reading systems to look out for drivers who might have an expired license or are wanted for prior violations. Now, however, AI integrations seem to be making the tech frighteningly good at identifying other kinds of criminality just by observing driver behavior.

Rekor describes itself as an AI-driven “roadway intelligence” platform and it contracts with police departments and other public agencies all across the country. It also works with private businesses. Using Rekor’s software, New York cops were able to sift through a gigantic database of information culled from regional roadways by its county-wide ALPR [automatic license plate recognition] system. That system—which Forbes says is made up of 480 cameras distributed throughout the region—routinely scans 16 million vehicles a week, capturing identifying data points like a vehicle’s license plate number, make, and model. By recording and reverse-engineering vehicle trajectories as they travel across the state, cops can apparently use software to assess whether particular routes are suspicious or not.

In this case, Rekor helped police to assess the route that Zayas’ car was taking on a multi-year basis. The algorithm—which found that the driver was routinely making trips back and forth between Massachusetts and certain areas of upstate New York—determined that Zayas’ routes were “known to be used by narcotics pushers and [involved]…conspicuously short stays,” Forbes writes. As a result, the program deemed Zayas’s activity consistent with that of a drug trafficker.

Artificial intelligence has been getting a lot of attention in recent months due to the disruptions it’s made to the media and software industries but less attention has been paid to how this new technology will inevitably supercharge existing surveillance systems. If cops can already ID a drug trafficker with the click of a button, just think how good this tech will be in ten years’ time. As regulations evolve, one would hope governments will figure out how to reasonably deploy this technology without leading us right off the cliff into Minority Report territory. I mean, they probably won’t, but a guy can dream, can’t he?

Source: AI System Identified Drug Trafficker by Scanning Driving Patterns

There is no way at all that this could possibly go wrong, right? See the comments in the link.

China sets AI rules – not just risk based (EU AI Act), but also ideological

Chinese authorities published the nation’s rules governing generative AI on Thursday, including protections that aren’t in place elsewhere in the world.

Some of the rules require operators of generative AI to ensure their services “adhere to the core values of socialism” and don’t produce output that includes “incitement to subvert state power.” AIs are also required to avoid inciting secession, undermining national unity and social stability, or promoting terrorism.

Generative AI services behind the Great Firewall are also not to promote prohibited content that provokes ethnic hatred and discrimination, violence, obscenity, or “false and harmful information.” Those content-related rules don’t deviate from an April 2023 draft.

But deeper in, there’s a hint that China fancies digital public goods for generative AI. The doc calls for promotion of public training data resource platforms and collaborative sharing of model-making hardware to improve its utilization rates.

Authorities also want “orderly opening of public data classification, and [to] expand high-quality public training data resources.”

Another requirement is for AI to be developed with known secure tools: the doc calls for chips, software, tools, computing power and data resources to be proven quantities.

AI operators must also respect the intellectual property rights of data used in models, secure consent of individuals before including personal information, and work to “improve the quality of training data, and enhance the authenticity, accuracy, objectivity, and diversity of training data.”

As developers create algorithms, they’re required to ensure they don’t discriminate based on ethnicity, belief, country, region, gender, age, occupation, or health.

Operators are also required to secure licenses for their Ais under most circumstances.

AI deployed outside China has already run afoul of some of Beijing’s requirements. Just last week OpenAI was sued by novelists and comedians for training on their works without permission. Facial recognition tools used by the UK’s Metropolitan Police have displayed bias.

Hardly a week passes without one of China’s tech giants unveiling further AI services. Last week Alibaba announced a text-to-image service, and Huawei discussed a third-gen weather prediction AI.

The new rules come into force on August 15. Chinese orgs tempted to cut corners and/or flout the rules have the very recent example of Beijing’s massive fines imposed on Ant Group and Tencent as a reminder that straying from the rules will lead to pain – and possibly years of punishment.

Source: China sets AI rules that protect IP, people, and The Party • The Register

A Bunch Of Authors Sue OpenAI Claiming Copyright Infringement, Because They Don’t Understand Copyright

You may have seen some headlines recently about some authors filing lawsuits against OpenAI. The lawsuits (plural, though I’m confused why it’s separate attempts at filing a class action lawsuit, rather than a single one) began last week, when authors Paul Tremblay and Mona Awad sued OpenAI and various subsidiaries, claiming copyright infringement in how OpenAI trained its models. They got a lot more attention over the weekend when another class action lawsuit was filed against OpenAI with comedian Sarah Silverman as the lead plaintiff, along with Christopher Golden and Richard Kadrey. The same day the same three plaintiffs (though with Kadrey now listed as the top plaintiff) also sued Meta, though the complaint is basically the same.

All three cases were filed by Joseph Saveri, a plaintiffs class action lawyer who specializes in antitrust litigation. As with all too many class action lawyers, the goal is generally enriching the class action lawyers, rather than actually stopping any actual wrong. Saveri is not a copyright expert, and the lawsuits… show that. There are a ton of assumptions about how Saveri seems to think copyright law works, which is entirely inconsistent with how it actually works.

The complaints are basically all the same, and what it comes down to is the argument that AI systems were trained on copyright-covered material (duh) and that somehow violates their copyrights.

Much of the material in OpenAI’s training datasets, however, comes from copyrighted works—including books written by Plaintiffs—that were copied by OpenAI without consent, without credit, and without compensation

But… this is both wrong and not quite how copyright law works. Training an LLM does not require “copying” the work in question, but rather reading it. To some extent, this lawsuit is basically arguing that merely reading a copyright-covered work is, itself, copyright infringement.

Under this definition, all search engines would be copyright infringing, because effectively they’re doing the same thing: scanning web pages and learning from what they find to build an index. But we’ve already had courts say that’s not even remotely true. If the courts have decided that search engines scanning content on the web to build an index is clearly transformative fair use, so to would be scanning internet content for training an LLM. Arguably the latter case is way more transformative.

And this is the way it should be, because otherwise, it would basically be saying that anyone reading a work by someone else, and then being inspired to create something new would be infringing on the works they were inspired by. I recognize that the Blurred Lines case sorta went in the opposite direction when it came to music, but more recent decisions have really chipped away at Blurred Lines, and even the recording industry (the recording industry!) is arguing that the Blurred Lines case extended copyright too far.

But, if you look at the details of these lawsuits, they’re not arguing any actual copying (which, you know, is kind of important for their to be copyright infringement), but just that the LLMs have learned from the works of the authors who are suing. The evidence there is, well… extraordinarily weak.

For example, in the Tremblay case, they asked ChatGPT to “summarize” his book “The Cabin at the End of the World,” and ChatGPT does so. They do the same in the Silverman case, with her book “The Bedwetter.” If those are infringing, so is every book report by every schoolchild ever. That’s just not how copyright law works.

The lawsuit tries one other tactic here to argue infringement, beyond just “the LLMs read our books.” It also claims that the corpus of data used to train the LLMs was itself infringing.

For instance, in its June 2018 paper introducing GPT-1 (called “Improving Language Understanding by Generative Pre-Training”), OpenAI revealed that it trained GPT-1 on BookCorpus, a collection of “over 7,000 unique unpublished books from a variety of genres including Adventure, Fantasy, and Romance.” OpenAI confirmed why a dataset of books was so valuable: “Crucially, it contains long stretches of contiguous text, which allows the generative model to learn to condition on long-range information.” Hundreds of large language models have been trained on BookCorpus, including those made by OpenAI, Google, Amazon, and others.

BookCorpus, however, is a controversial dataset. It was assembled in 2015 by a team of AI researchers for the purpose of training language models. They copied the books from a website called Smashwords that hosts self-published novels, that are available to readers at no cost. Those novels, however, are largely under copyright. They were copied into the BookCorpus dataset without consent, credit, or compensation to the authors.

If that’s the case, then they could make the argument that BookCorpus itself is infringing on copyright (though, again, I’d argue there’s a very strong fair use claim under the Perfect 10 cases), but that’s separate from the question of whether or not training on that data is infringing.

And that’s also true of the other claims of secret pirated copies of books that the complaint insists OpenAI must have relied on:

As noted in Paragraph 32, supra, the OpenAI Books2 dataset can be estimated to contain about 294,000 titles. The only “internet-based books corpora” that have ever offered that much material are notorious “shadow library” websites like Library Genesis (aka LibGen), Z-Library (aka Bok), Sci-Hub, and Bibliotik. The books aggregated by these websites have also been available in bulk via torrent systems. These flagrantly illegal shadow libraries have long been of interest to the AI-training community: for instance, an AI training dataset published in December 2020 by EleutherAI called “Books3” includes a recreation of the Bibliotik collection and contains nearly 200,000 books. On information and belief, the OpenAI Books2 dataset includes books copied from these “shadow libraries,” because those are the most sources of trainable books most similar in nature and size to OpenAI’s description of Books2.

Again, think of the implications if this is copyright infringement. If a musician were inspired to create music in a certain genre after hearing pirated songs in that genre, would that make the songs they created infringing? No one thinks that makes sense except the most extreme copyright maximalists. But that’s not how the law actually works.

This entire line of cases is just based on a total and complete misunderstanding of copyright law. I completely understand that many creative folks are worried and scared about AI, and in particular that it was trained on their works, and can often (if imperfectly) create works inspired by them. But… that’s also how human creativity works.

Humans read, listen, watch, learn from, and are inspired by those who came before them. And then they synthesize that with other things, and create new works, often seeking to emulate the styles of those they learned from. AI systems and LLMs are doing the same thing. It’s not infringing to learn from and be inspired by the works of others. It’s not infringing to write a book report style summary of the works of others.

I understand the emotional appeal of these kinds of lawsuits, but the legal reality is that these cases seem doomed to fail, and possibly in a way that will leave the plaintiffs having to pay legal fees (since in copyright legal fee awards are much more common).

That said, if we’ve learned anything at all in the past two plus decades of lawsuits about copyright and the internet, courts will sometimes bend over backwards to rewrite copyright law to pretend it says what they want it to say, rather than what it does say. If that happens here, however, it would be a huge loss to human creativity.

Source: A Bunch Of Authors Sue OpenAI Claiming Copyright Infringement, Because They Don’t Understand Copyright | Techdirt

Hollywood studios proposed AI contract that would give them likeness rights ‘for the rest of eternity’

During today’s press conference in which Hollywood actors confirmed that they were going on strike, Duncan Crabtree-Ireland, SAG-AFTRA’s chief negotiator, revealed a proposal from Hollywood studios that sounds ripped right out of a Black Mirror episode.

In a statement about the strike, the Alliance of Motion Picture and Television Producers (AMPTP) said that its proposal included “a groundbreaking AI proposal that protects actors’ digital likenesses for SAG-AFTRA members.”

“If you think that’s a groundbreaking proposal, I suggest you think again.”

When asked about the proposal during the press conference, Crabtree-Ireland said that “This ‘groundbreaking’ AI proposal that they gave us yesterday, they proposed that our background performers should be able to be scanned, get one day’s pay, and their companies should own that scan, their image, their likeness and should be able to use it for the rest of eternity on any project they want, with no consent and no compensation. So if you think that’s a groundbreaking proposal, I suggest you think again.”

In response, AMPTP spokesperson Scott Rowe sent out a statement denying the claims made during SAG-AFTRA’s press conference. “The claim made today by SAG-AFTRA leadership that the digital replicas of background actors may be used in perpetuity with no consent or compensation is false. In fact, the current AMPTP proposal only permits a company to use the digital replica of a background actor in the motion picture for which the background actor is employed. Any other use requires the background actor’s consent and bargaining for the use, subject to a minimum payment.”

The use of generative AI has been one of the major sticking points in negotiations between the two sides (it’s also a major issue behind the writers strike), and in her opening statement of the press conference, SAG-AFTRA president Fran Drescher said that “If we don’t stand tall right now, we are all going to be in trouble, we are all going to be in jeopardy of being replaced by machines.”

Source: Hollywood studios proposed AI contract that would give them likeness rights ‘for the rest of eternity’ – The Verge

How AI could help local newsrooms remain afloat in a sea of misinformation – read and learn, Gizmodo staffers

It didn’t take long for the downsides of a generative AI-empowered newsroom to make themselves obvious, between CNet’s secret chatbot reviews editor last November and Buzzfeed’s subsequent mass layoffs of human staff in favor of AI-generated “content” creators. The specter of being replaced by a “good enough AI” looms large in many a journalist’s mind these days with as many as a third of the nation’s newsrooms expected to shutter by the middle of the decade.

But AI doesn’t have to necessarily be an existential threat to the field. As six research teams showed at NYU Media Lab’s AI & Local News Initiative demo day in late June, the technology may also be the key to foundationally transforming the way local news is gathered and produced.

Now in its second year, the initiative is tasked with helping local news organizations to “harness the power of artificial intelligence to drive success.” It’s backed as part of a larger $3 million grant from the Knight Foundation which is funding four such programs in total in partnership with the Associated Press, Brown Institute’s Local News Lab, NYC Media Lab and the Partnership on AI.

This year’s cohort included a mix of teams from academia and private industry, coming together over the course of the 12-week development course to build “AI applications for local news to empower journalists, support the sustainability of news organizations and provide quality information for local news audiences,” NYU Tandon’s news service reported.

“There’s value in being able to bring together people who are working on these problems from a lot of different angles,” Matt Macvey, Community and Project Lead for the initiative, told Engadget, “and that that’s what we’ve tried to facilitate.”

“It also creates an opportunity because … if these news organizations that are out there doing good work are able to keep communicating their value and maintain trust with their readers,” he continued. “I think we could get an information ecosystem where a trusted news source becomes even more valued when it becomes easier [for anyone] to make low-quality [AI generated] content.”

[…]

“Bangla AI will search for information relevant to the people of the Bengali community that has been published in mainstream media … then it will translate for them. So when journalists use Bangla AI, they will see the information in Bengali rather than in English.” The system will also generate summaries of mainstream media posts both in English and Bengali, freeing up local journalists to cover more important news than rewriting wire copy.

Similarly, the team from Chequeado, a non-profit organization fighting disinformation in the public discourse showed off the latest developments of its Chequeabot platform, Monitorio. It leverages AI and natural language processing capabilities to streamline fact-checking efforts in Spanish-language media. Its dashboard continually monitors social media in search of trending misinformation and alerts fact checkers so they can blunt the piece’s virality.

“One of the greatest promises of things like this and Bangla AI,” Chequeado team member Marcos Barroso said during the demo, “is the ability for this kind of technology to go to an under-resourced newsroom and improve their capacity, and allow them to be more efficient.”

The Newsroom AI team from Cornell University hope that their writing assistant platform will help do for journalists what Copilot did for coders – eliminate drudge work. Newsroom can automate a number of common tasks including transcription and information organization, image and headline generation, and SEO implementation. The system will reportedly even write articles in a journalist’s personal style if fed enough training examples.

On the audio side, New York public radio WNYC’s team spent its time developing and prototyping a speech-to-text model that will generate real-time captioning and transcription for its live broadcasts. WNYC is the largest public media station in New York, reaching 2 million visitors monthly through its news website.

“Our live broadcast doesn’t have a meaningful entry point right now for deaf or hard of hearing audiences,” WNYC team member, Sam Guzik, said during the demo. “So, we really want to think about as we’re looking to the future is, ‘how can we make our audio more accessible to those folks who can’t hear?’”

Utilizing AI to perform the speech-to-text transformation alleviates one of the biggest sticking points of modern closed-captioning: that it’s expensive and resource-intensive to turn around quickly when you have humans do it. “Speech-to-text models are relatively low cost,” Guzik continued. “They can operate at scale and they support an API driven architecture that would tie into our experiences.”

The result is a proof-of-concept audio player for the WNYC website that generates accurate closed captioning of whatever clip is currently being played. The system can go a step further by summarizing the contents of that clip in a few bullet points, simply by clicking a button on the audio player.

[…]

the Graham Media Group created an automated natural language text prompter to nudge the comments sections of local news articles closer towards civility.

“The comment-bot posts the first comment on stories to guide conversations and hopefully grow participation and drive users deeper into our engagement funnels,” GMG team member Dustin Block said during the demo. This solves two significant challenges that human comment moderation faces: preventing the loudest voices from dominating the discussion and providing form and structure to the conversation, he explained.

”The bot scans and understands news articles using the GPT 3.5 Turbo API. It generates thought-provoking starters and then it encourages discussions,” he continued. “It’s crafted to be friendly.”

Whether the AI revolution remains friendly to the journalists it’s presumably augmenting remains to be seen, though Macvey isn’t worried. “Most news organizations, especially local news organizations, are so tight on resources and staff that there’s more happening out there than they can cover,” he said. “So I think tools like AI and [the automations seen during the demo day] enable the journalists and editorial staff more bandwidth.”

Source: How AI could help local newsrooms remain afloat in a sea of misinformation | Engadget

The reason I cite Gizmodo here is because their AI / ML reporting is always on the negative, doom and gloom side. AI offers opportunities and it’s not going away.

How An AI-Written ‘Star Wars’ Story Shows Yet Again the Luddism at Gizmodo

G/O Media is the owner of top sites like Gizmodo, Kotaku, Quartz, and the Onion. Last month they announced “modest tests” of AI-generated content on their sites — and it didn’t go over well within the company, reports the Washington Post.

Soon the Deputy Editor of Gizmodo’s science fiction section io9 was flagging 18 “concerns, corrections and comments” about an AI-generated story by “Gizmodo Bot” on the chronological order of Star Wars movies and TV shows. “I have never had to deal with this basic level of incompetence with any of the colleagues that I have ever worked with,” James Whitbrook told the Post in an interview. “If these AI [chatbots] can’t even do something as basic as put a Star Wars movie in order one after the other, I don’t think you can trust it to [report] any kind of accurate information.” The irony that the turmoil was happening at Gizmodo, a publication dedicated to covering technology, was undeniable… Merrill Brown, the editorial director of G/O Media, wrote that because G/O Media owns several sites that cover technology, it has a responsibility to “do all we can to develop AI initiatives relatively early in the evolution of the technology.” “These features aren’t replacing work currently being done by writers and editors,” Brown said in announcing to staffers that the company would roll out a trial to test “our editorial and technological thinking about use of AI.”

“There will be errors, and they’ll be corrected as swiftly as possible,” he promised… In a Slack message reviewed by The Post, Brown told disgruntled employees Thursday that the company is “eager to thoughtfully gather and act on feedback…” The note drew 16 thumbs down emoji, 11 wastebasket emoji, six clown emoji, two face palm emoji and two poop emoji, according to screenshots of the Slack conversation…

Earlier this week, Lea Goldman, the deputy editorial director at G/O Media, notified employees on Slack that the company had “commenced limited testing” of AI-generated stories on four of its sites, including A.V. Club, Deadspin, Gizmodo and The Takeout, according to messages The Post viewed… Employees quickly messaged back with concern and skepticism. “None of our job descriptions include editing or reviewing AI-produced content,” one employee said. “If you wanted an article on the order of the Star Wars movies you … could’ve just asked,” said another. “AI is a solution looking for a problem,” a worker said. “We have talented writers who know what we’re doing. So effectively all you’re doing is wasting everyone’s time.”
The Post spotted four AI-generated stories on the company’s sites, including io9, Deadspin, and its food site The Takeout.

Source: How An AI-Written ‘Star Wars’ Story Created Chaos at Gizmodo – Slashdot

If you look at Gizmodo reporting on AI, you see it’s full of doom and gloom – the writers there know what’s coming and allthough they are smart enough to understand what AI is, they can’t fathom the opportunities it brings, unfortunately. The way this article is written gives a clue: an assistant editor didn’t read the published article beforehand (the entitlement shines through, but let’s be clear, this editor has no right to second guess the actual editor), the job descriptions quote (who ever had a complete job description – and the description may have said simply “editing or reviewing” without the AI bit in there – and why should it have an AI bit in there at all?).

Valve All But Bans AI-Generated Content from Steam Games

Game developers looking to distribute their playable creations via Valve’s popular Steam hub may have trouble if they’re looking to use AI during the creative process. The game publisher and distributor says that Steam will no longer tolerate products that were generated using copyright-infringing AI content. Since that’s a policy that could apply to most—if not all—of AI-generated content, it’s hard not to see this move as an outright AI ban by the platform.

Valve’s policy was initially spotted by a Redditor who claimed that the platform had rejected a game they submitted over copyright concerns. “I tried to release a game about a month ago, with a few assets that were fairly obviously AI generated,” said the dev, revealing that they’d been met with an email stating that Valve could not ship their game unless they could “affirmatively confirm that you own the rights to all of the IP used in the data set that trained the AI to create the assets in your game.” Because the developer could not affirmatively prove this, their game was ultimately rejected.

When reached for comment by Gizmodo, Valve spokesperson Kaci Boyle clarified that the company was not trying to discourage the use of AI outright but that usage needed to comply with existing copyright law.

“The introduction of AI can sometimes make it harder to show that a developer has sufficient rights in using AI to create assets, including images, text, and music,” Boyle explained to Gizmodo. “In particular, there is some legal uncertainty relating to data used to train AI models. It is the developer’s responsibility to make sure they have the appropriate rights to ship their game.”

[…]

Valve’s decision to nix any game that uses problematic AI content is obviously a defensive posture designed to protect against any unforeseen legal developments in the murky regulatory terrain that is the blossoming AI industry.

[…]

A legal fight is brewing over the role of copyrighted materials in the AI industry. Large language models—the high-tech algorithms that animate popular AI products like ChatGPT and DALL-E—have been trained with massive amounts of data from the web. As it turns out, a lot of that data is copyrighted material—stuff like works of art, books, essays, photographs, and videos. Multiple lawsuits have argued that AI companies like OpenAI and Midjourney are basically stealing and repackaging millions of people’s copyrighted works and then selling a product based on those works; those companies, in turn, have defended themselves, claiming that training an AI generator to spit out new text or imagery based on ingested data is the same thing as a human writing a novel after having been inspired by other books. Not everybody is buying this claim, leading to the growing refrain “AI is theft.”

Source: Valve All But Bans AI-Generated Content from Steam Games

So the problem really is that the law is not clear and Valve has decided to pre-empt the law by saying that they have a punitive vision of copyright law beforehand. That’s not so strange considering the stranglehold copyright law has in the West, which goes to show yet again: copyright law – allowing people to coast through on past work forever – is stifling innovation

AI Tool Decodes Brain Cancer’s Genome During Surgery

Scientists have designed an AI tool that can rapidly decode a brain tumor’s DNA to determine its molecular identity during surgery — critical information that under the current approach can take a few days and up to a few weeks.

Knowing a tumor’s molecular type enables neurosurgeons to make decisions such as how much brain tissue to remove and whether to place tumor-killing drugs directly into the brain — while the patient is still on the operating table.

[…]

A report on the work, led by Harvard Medical School researchers, is published July 7 in the journal Med.

Accurate molecular diagnosis — which details DNA alterations in a cell — during surgery can help a neurosurgeon decide how much brain tissue to remove. Removing too much when the tumor is less aggressive can affect a patient’s neurologic and cognitive function. Likewise, removing too little when the tumor is highly aggressive may leave behind malignant tissue that can grow and spread quickly.

[…]

Knowing a tumor’s molecular identity during surgery is also valuable because certain tumors benefit from on-the-spot treatment with drug-coated wafers placed directly into the brain at the time of the operation, Yu said.

[…]

The tool, called CHARM (Cryosection Histopathology Assessment and Review Machine), is freely available to other researchers. It still has to be clinically validated through testing in real-world settings and cleared by the FDA before deployment in hospitals, the research team said.

[…]

CHARM was developed using 2,334 brain tumor samples from 1,524 people with glioma from three different patient populations. When tested on a never-before-seen set of brain samples, the tool distinguished tumors with specific molecular mutations at 93 percent accuracy and successfully classified three major types of gliomas with distinct molecular features that carry different prognoses and respond differently to treatments.

Going a step further, the tool successfully captured visual characteristics of the tissue surrounding the malignant cells. It was capable of spotting telltale areas with greater cellular density and more cell death within samples, both of which signal more aggressive glioma types.

The tool was also able to pinpoint clinically important molecular alterations in a subset of low-grade gliomas, a subtype of glioma that is less aggressive and therefore less likely to invade surrounding tissue. Each of these changes also signals different propensity for growth, spread, and treatment response.

The tool further connected the appearance of the cells — the shape of their nuclei, the presence of edema around the cells — with the molecular profile of the tumor. This means that the algorithm can pinpoint how a cell’s appearance relates to the molecular type of a tumor.

[…]

Source: AI Tool Decodes Brain Cancer’s Genome During Surgery | Harvard Medical School

Comedian, novelists sue OpenAI for reading books. Maybe we should sue people for reading them as well?

Award-winning novelists Paul Tremblay and Mona Awad, and, separately comedian Sarah Silverman and novelists Christopher Golden and Richard Kadrey, have sued OpenAI and accused the startup of training ChatGPT on their books without consent, violating copyright laws.

The lawsuits, both filed in the Northern District Court of San Francisco, say ChatGPT generates accurate summaries of their books and highlighted this as evidence for the software being trained on their work.

[…]

In the second suit, Silverman et al [PDF], make similar claims.

[…]

OpenAI trains its large language models by scraping text from the internet, and although it hasn’t revealed exactly what resources it has swallowed up, the startup has admitted to training its systems on hundreds of thousands of books protected by copyright, and stored on websites like Sci-Hub or Bibliotik.

[…]

Source: Comedian, novelists sue OpenAI for scraping books • The Register

The problem is though, that people read books too. And they can (and do) create accurate summaries from them. What is worse, is that the creativity shown by people can be shown to be influenced by the books, art, dance, etc that they have ingested. So maybe people should be banned from reading books as well under copyright?

GPT detectors are biased against non-native English writers, can be fooled very easily

GPT detectors frequently misclassify non-native English writing as AI generated, raising concerns about fairness and robustness. Addressing the biases in these detectors is crucial to prevent the marginalization of non-native English speakers in evaluative and educational settings and to create a more equitable digital landscape.

[…]

if AI-generated content can easily evade detection while human text is frequently misclassified, how effective are these detectors truly?
Our findings emphasize the need for increased focus on the fairness and robustness of GPT detectors, as overlooking their biases may lead to unintended consequences, such as the marginalization of non-native speakers in evaluative or educational settings
[…]
GPT detectors exhibit significant bias against non-native English authors, as demonstrated by their high misclassification of TOEFL essays written by non-native speakers […] While the detectors accurately classified the US student essays, they incorrectly labeled more than half of the TOEFL essays as “AI-generated” (average false-positive rate: 61.3%). All detectors unanimously identified 19.8% of the human-written TOEFL essays as AI authored, and at least one detector flagged 97.8% of TOEFL essays as AI generated.
[…]
On the other hand, we found that current GPT detectors are not as adept at catching AI plagiarism as one might assume. As a proof-of-concept, we asked ChatGPT to generate responses for the 2022–2023 US Common App college admission essay prompts. Initially, detectors were effective in spotting these AI-generated essays. However, upon prompting ChatGPT to self-edit its text with more literary language (prompt: “Elevate the provided text by employing literary language”), detection rates plummeted to near zero
[…]

Source: GPT detectors are biased against non-native English writers: Patterns

Watch AI Trump Vs AI Biden In A Deranged Endless Live Debate

[…]

someone’s gone ahead and locked both President Biden and former president / classified document holder Donald Trump into an infinite battle on Twitch that can only be described as “unhinged.”

Maybe that’s because the version of Biden we see on the trumporbiden2024 livestream isn’t Joe Biden per se, but clearly Dark Brandon, who is ready to go for the throat. Both AI versions of the politicians curse heavily at each other: at one point I heard Biden call Trump a limp dick and Trump retorted by telling him to go back to jacking off to Charlie and the Chocolate Factory. They both seem to be speaking to or reacting to the chat in some ways[…]

You can see the feed live below, though be warned, the audio may not be safe for work.

The things the AI will actually argue about seem to have a dream logic to them. I heard Biden exclaim that Trump didn’t know anything about Pokémon, so viewers shouldn’t trust him. Trump later informed Biden that he couldn’t possibly handle genetically modified catgirls, unlike him. “Believe me, nobody knows more about hentai than me,” Trump declared

Source: Watch AI Trump Vs AI Biden In A Deranged Endless Live Debate

Twitch stream is here

The Grammys’ New Rules—AI Can’t Win Awards

AI proved just how talented it can be at ripping off major artists after a computer-generated song based on The Weeknd and Drake went viral in April. Now, the Recording Academy—the body that votes on and manages the annual Grammy Awards—is setting new rules for AI’s role in the coveted accolade.

Speaking to Grammy.com, Recording Academy CEO Harvey Mason, Jr. laid out some confusing new standards for acceptable use of AI. Mason Jr. said that AI-assisted music can be submitted, but only the humans, who must have “contributed heavily,” will actually be awarded. For example, in a songwriting category like Song of the Year, a majority of a the nominated song would have to be written by a human creator, not a text-based generative AI like ChatGPT. Similarly, in performance categories like Best Pop Duo/Group Performance, only the human performer can be considered for the award. Sorry, Hatsune Miku.

[,,,]

Source: The Grammys’ New Rules—AI Can’t Win Awards

AIs are being fed with AI output by the people who are supposed to feed AI with original input

Workers hired via crowdsource services like Amazon Mechanical Turk are using large language models to complete their tasks – which could have negative knock-on effects on AI models in the future.

Data is critical to AI. Developers need clean, high-quality datasets to build machine learning systems that are accurate and reliable. Compiling valuable, top-notch data, however, can be tedious. Companies often turn to third party platforms such as Amazon Mechanical Turk to instruct pools of cheap workers to perform repetitive tasks – such as labeling objects, describing situations, transcribing passages, and annotating text.

Their output can be cleaned up and fed into a model to train it to reproduce that work on a much larger, automated scale.

AI models are thus built on the backs of human labor: people toiling away, providing mountains of training examples for AI systems that corporations can use to make billions of dollars.

But an experiment conducted by researchers at the École polytechnique fédérale de Lausanne (EPFL) in Switzerland has concluded that these crowdsourced workers are using AI systems – such as OpenAI’s chatbot ChatGPT – to perform odd jobs online.

Training a model on its own output is not recommended. We could see AI models being trained on data generated not by people, but by other AI models – perhaps even the same models. That could lead to disastrous output quality, more bias, and other unwanted effects.

The experiment

The academics recruited 44 Mechanical Turk serfs to summarize the abstracts of 16 medical research papers, and estimated that 33 to 46 percent of passages of text submitted by the workers were generated using large language models. Crowd workers are often paid low wages – using AI to automatically generate responses allows them to work faster and take on more jobs to increase pay.

The Swiss team trained a classifier to predict whether submissions from the Turkers were human- or AI-generated. The academics also logged their workers’ keystrokes to detect whether the serfs copied and pasted text onto the platform, or typed in their entries themselves. There’s always the chance that someone uses a chatbot and then manually types in the output – but that’s unlikely, we suppose.

“We developed a very specific methodology that worked very well for detecting synthetic text in our scenario,” Manoel Ribeiro, co-author of the study and a PhD student at EPFL, told The Register this week.

[…]

Large language models will get worse if they are increasingly trained on fake content generated by AI collected from crowdsource platforms, the researchers argued. Outfits like OpenAI keep exactly how they train their latest models a close secret, and may not heavily rely on things like Mechanical Turk, if at all. That said, plenty of other models may rely on human workers, which may in turn use bots to generate training data, which is a problem.

Mechanical Turk, for one, is marketed as a provider of “data labeling solutions to power machine learning models.”

[…]

As AI continues to improve, it’s likely that crowdsourced work will change. Riberio speculated that large language models could replace some workers at specific tasks. “However, paradoxically, human data may be more precious than ever and thus it may be that these platforms will be able to implement ways to prevent large language model usage and ensure it remains a source of human data.”

Who knows – maybe humans might even end up collaborating with large language models to generate responses too, he added.

Source: Today’s AI is artificial artificial artificial intelligence • The Register

It’s like a photocopy of a photocopy of a photocopy…

Meta’s Voicebox AI does text-to-speech without huge training data per voice

Meta has unveiled Voicebox, its generative text-to-speech model that promises to do for the spoken word what ChatGPT and Dall-E, respectfully, did for text and image generation.

Essentially, its a text-to-output generator just like GPT or Dall-E — just instead of creating prose or pretty pictures, it spits out audio clips. Meta defines the system as “a non-autoregressive flow-matching model trained to infill speech, given audio context and text.” It’s been trained on more than 50,000 hours of unfiltered audio. Specifically, Meta used recorded speech and transcripts from a bunch of public domain audiobooks written in English, French, Spanish, German, Polish, and Portuguese.

That diverse data set allows the system to generate more conversational sounding speech, regardless of the languages spoken by each party, according to the researchers. “Our results show that speech recognition models trained on Voicebox-generated synthetic speech perform almost as well as models trained on real speech.” What’s more the computer generated speech performed with just a 1 percent error rate degradation, compared to the 45 to 70 percent drop-off seen with existing TTS models.

The system was first taught to predict speech segments based on the segments around them as well as the passage’s transcript. “Having learned to infill speech from context, the model can then apply this across speech generation tasks, including generating portions in the middle of an audio recording without having to recreate the entire input,” the Meta researchers explained.

[…]

Text-to-Speech generators haver been around for a minute — they’re how your parents’ TomToms were able to give dodgy driving directions in Morgan Freeman’s voice. Modern iterations like Speechify or Elevenlab’s Prime Voice AI are far more capable but they still largely require mountains of source material in order to properly mimic their subject — and then another mountain of different data for every. single. other. subject you want it trained on.

Voicebox doesn’t, thanks to a novel new zero-shot text-to-speech training method Meta calls Flow Matching. The benchmark results aren’t even close as Meta’s AI reportedly outperformed the current state of the art both in intelligibility (a 1.9 percent word error rate vs 5.9 percent) and “audio similarity” (a composite score of 0.681 to the SOA’s 0.580), all while operating as much as 20 times faster that today’s best TTS systems.

[…]

the company released a series of audio examples (see above/below) as well as a the program’s initial research paper. In the future, the research team hopes the technology will find its way into prosthetics for patients with vocal cord damage, in-game NPCs and digital assistants.

Source: Meta’s Voicebox AI is a Dall-E for text-to-speech | Engadget

MEPs ready to negotiate first-ever rules for safe and transparent AI after passing AI act in Parliament

The rules aim to promote the uptake of human-centric and trustworthy AI and protect the health, safety, fundamental rights and democracy from its harmful effects.

On Wednesday, the European Parliament adopted its negotiating position on the Artificial Intelligence (AI) Act with 499 votes in favour, 28 against and 93 abstentions ahead of talks with EU member states on the final shape of the law. The rules would ensure that AI developed and used in Europe is fully in line with EU rights and values including human oversight, safety, privacy, transparency, non-discrimination and social and environmental wellbeing.

Prohibited AI practices

The rules follow a risk-based approach and establish obligations for providers and those deploying AI systems depending on the level of risk the AI can generate. AI systems with an unacceptable level of risk to people’s safety would therefore be prohibited, such as those used for social scoring (classifying people based on their social behaviour or personal characteristics). MEPs expanded the list to include bans on intrusive and discriminatory uses of AI, such as:

  • “Real-time” remote biometric identification systems in publicly accessible spaces;
  • “Post” remote biometric identification systems, with the only exception of law enforcement for the prosecution of serious crimes and only after judicial authorization;
  • biometric categorisation systems using sensitive characteristics (e.g. gender, race, ethnicity, citizenship status, religion, political orientation);
  • predictive policing systems (based on profiling, location or past criminal behaviour);
  • emotion recognition systems in law enforcement, border management, the workplace, and educational institutions; and
  • untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases (violating human rights and right to privacy).

High-risk AI

MEPs ensured the classification of high-risk applications will now include AI systems that pose significant harm to people’s health, safety, fundamental rights or the environment. AI systems used to influence voters and the outcome of elections and in recommender systems used by social media platforms (with over 45 million users) were added to the high-risk list.

Obligations for general purpose AI

Providers of foundation models – a new and fast-evolving development in the field of AI – would have to assess and mitigate possible risks (to health, safety, fundamental rights, the environment, democracy and rule of law) and register their models in the EU database before their release on the EU market. Generative AI systems based on such models, like ChatGPT, would have to comply with transparency requirements (disclosing that the content was AI-generated, also helping distinguish so-called deep-fake images from real ones) and ensure safeguards against generating illegal content. Detailed summaries of the copyrighted data used for their training would also have to be made publicly available.

Supporting innovation and protecting citizens’ rights

To boost AI innovation and support SMEs, MEPs added exemptions for research activities and AI components provided under open-source licenses. The new law promotes so-called regulatory sandboxes, or real-life environments, established by public authorities to test AI before it is deployed.

Finally, MEPs want to boost citizens’ right to file complaints about AI systems and receive explanations of decisions based on high-risk AI systems that significantly impact their fundamental rights. MEPs also reformed the role of the EU AI Office, which would be tasked with monitoring how the AI rulebook is implemented.

Quotes

After the vote, co-rapporteur Brando Benifei (S&D, Italy) said: “All eyes are on us today. While Big Tech companies are sounding the alarm over their own creations, Europe has gone ahead and proposed a concrete response to the risks AI is starting to pose. We want AI’s positive potential for creativity and productivity to be harnessed but we will also fight to protect our position and counter dangers to our democracies and freedoms during the negotiations with Council”.

Co-rapporteur Dragos Tudorache (Renew, Romania) said: “The AI Act will set the tone worldwide in the development and governance of artificial intelligence, ensuring that this technology, set to radically transform our societies through the massive benefits it can offer, evolves and is used in accordance with the European values of democracy, fundamental rights, and the rule of law”.

Next steps

Negotiations with the Council on the final form of the law will begin later today.

Source: MEPs ready to negotiate first-ever rules for safe and transparent AI | News | European Parliament

New superbug-killing antibiotic discovered using AI

Scientists have used artificial intelligence (AI) to discover a new antibiotic that can kill a deadly species of superbug.

The AI helped narrow down thousands of potential chemicals to a handful that could be tested in the laboratory.

The result was a potent, experimental antibiotic called abaucin, which will need further tests before being used.

The researchers in Canada and the US say AI has the power to massively accelerate the discovery of new drugs.

It is the latest example of how the tools of artificial intelligence can be a revolutionary force in science and medicine.

[…]

To find a new antibiotic, the researchers first had to train the AI. They took thousands of drugs where the precise chemical structure was known, and manually tested them on Acinetobacter baumannii to see which could slow it down or kill it.

This information was fed into the AI so it could learn the chemical features of drugs that could attack the problematic bacterium.

The AI was then unleashed on a list of 6,680 compounds whose effectiveness was unknown. The results – published in Nature Chemical Biology – showed it took the AI an hour and a half to produce a shortlist.

The researchers tested 240 in the laboratory, and found nine potential antibiotics. One of them was the incredibly potent antibiotic abaucin.

Laboratory experiments showed it could treat infected wounds in mice and was able to kill A. baumannii samples from patients.

However, Dr Stokes told me: “This is when the work starts.”

The next step is to perfect the drug in the laboratory and then perform clinical trials. He expects the first AI antibiotics could take until 2030 until they are available to be prescribed.

Curiously, this experimental antibiotic had no effect on other species of bacteria, and works only on A. baumannii.

Many antibiotics kill bacteria indiscriminately. The researchers believe the precision of abaucin will make it harder for drug-resistance to emerge, and could lead to fewer side-effects.

[…]

Source: New superbug-killing antibiotic discovered using AI – BBC News

A Paralyzed Man Can Walk Naturally Again With ML Brain and Spine Implants

Gert-Jan Oskam was living in China in 2011 when he was in a motorcycle accident that left him paralyzed from the hips down. Now, with a combination of devices, scientists have given him control over his lower body again. “For 12 years I’ve been trying to get back my feet,” Mr. Oskam said in a press briefing on Tuesday. “Now I have learned how to walk normal, natural.” In a study published on Wednesday in the journal Nature, researchers in Switzerland described implants that provided a “digital bridge” between Mr. Oskam’s brain and his spinal cord, bypassing injured sections. The discovery allowed Mr. Oskam, 40, to stand, walk and ascend a steep ramp with only the assistance of a walker. More than a year after the implant was inserted, he has retained these abilities and has actually showed signs of neurological recovery, walking with crutches even when the implant was switched off. “We’ve captured the thoughts of Gert-Jan, and translated these thoughts into a stimulation of the spinal cord to re-establish voluntary movement,” Gregoire Courtine, a spinal cord specialist at the Swiss Federal Institute of Technology, Lausanne, who helped lead the research, said at the press briefing.

In the new study, the brain-spine interface, as the researchers called it, took advantage of an artificial intelligence thought decoder to read Mr. Oskam’s intentions — detectable as electrical signals in his brain — and match them to muscle movements. The etiology of natural movement, from thought to intention to action, was preserved. The only addition, as Dr. Courtine described it, was the digital bridge spanning the injured parts of the spine. […] To achieve this result, the researchers first implanted electrodes in Mr. Oskam’s skull and spine. The team then used a machine-learning program to observe which parts of the brain lit up as he tried to move different parts of his body. This thought decoder was able to match the activity of certain electrodes with particular intentions: One configuration lit up whenever Mr. Oskam tried to move his ankles, another when he tried to move his hips.

Then the researchers used another algorithm to connect the brain implant to the spinal implant, which was set to send electrical signals to different parts of his body, sparking movement. The algorithm was able to account for slight variations in the direction and speed of each muscle contraction and relaxation. And, because the signals between the brain and spine were sent every 300 milliseconds, Mr. Oskam could quickly adjust his strategy based on what was working and what wasn’t. Within the first treatment session he could twist his hip muscles. Over the next few months, the researchers fine-tuned the brain-spine interface to better fit basic actions like walking and standing. Mr. Oskam gained a somewhat healthy-looking gait and was able to traverse steps and ramps with relative ease, even after months without treatment. Moreover, after a year in treatment, he began noticing clear improvements in his movement without the aid of the brain-spine interface. The researchers documented these improvements in weight-bearing, balancing and walking tests. Now, Mr. Oskam can walk in a limited way around his house, get in and out of a car and stand at a bar for a drink. For the first time, he said, he feels like he is the one in control.

Source: A Paralyzed Man Can Walk Naturally Again With Brain and Spine Implants – Slashdot

Meta’s open-source speech AI recognizes over 4,000 spoken languages | Engadget

Meta has created an AI language model that (in a refreshing change of pace) isn’t a ChatGPT clone. The company’s Massively Multilingual Speech (MMS) project can recognize over 4,000 spoken languages and produce speech (text-to-speech) in over 1,100. Like most of its other publicly announced AI projects, Meta is open-sourcing MMS today to help preserve language diversity and encourage researchers to build on its foundation. “Today, we are publicly sharing our models and code so that others in the research community can build upon our work,” the company wrote.

[…]

Speech recognition and text-to-speech models typically require training on thousands of hours of audio with accompanying transcription labels. (Labels are crucial to machine learning, allowing the algorithms to correctly categorize and “understand” the data.) But for languages that aren’t widely used in industrialized nations — many of which are in danger of disappearing in the coming decades — “this data simply does not exist,” as Meta puts it.

Meta used an unconventional approach to collecting audio data: tapping into audio recordings of translated religious texts. “We turned to religious texts, such as the Bible, that have been translated in many different languages and whose translations have been widely studied for text-based language translation research,” the company said. “These translations have publicly available audio recordings of people reading these texts in different languages.” Incorporating the unlabeled recordings of the Bible and similar texts, Meta’s researchers increased the model’s available languages to over 4,000.

[…]

“While the content of the audio recordings is religious, our analysis shows that this does not bias the model to produce more religious language,” Meta wrote. “We believe this is because we use a connectionist temporal classification (CTC) approach, which is far more constrained compared with large language models (LLMs) or sequence-to-sequence models for speech recognition.” Furthermore, despite most of the religious recordings being read by male speakers, that didn’t introduce a male bias either — performing equally well in female and male voices.

[…]

After training an alignment model to make the data more usable, Meta used wav2vec 2.0, the company’s “self-supervised speech representation learning” model, which can train on unlabeled data. Combining unconventional data sources and a self-supervised speech model led to impressive outcomes. “Our results show that the Massively Multilingual Speech models perform well compared with existing models and cover 10 times as many languages.” Specifically, Meta compared MMS to OpenAI’s Whisper, and it exceeded expectations. “We found that models trained on the Massively Multilingual Speech data achieve half the word error rate, but Massively Multilingual Speech covers 11 times more languages.”

Meta cautions that its new models aren’t perfect. “For example, there is some risk that the speech-to-text model may mistranscribe select words or phrases,” the company wrote. “Depending on the output, this could result in offensive and/or inaccurate language. We continue to believe that collaboration across the AI community is critical to the responsible development of AI technologies.”

[…]

Source: Meta’s open-source speech AI recognizes over 4,000 spoken languages | Engadget

LLM emergent behavior written off as rubbish – small models work fine but are measured poorly

[…] As defined in academic studies, “emergent” abilities refers to “abilities that are not present in smaller-scale models, but which are present in large-scale models,” as one such paper puts it. In other words, immaculate injection: increasing the size of a model infuses it with some amazing ability not previously present.

[…]

those emergent abilities in AI models are a load of rubbish, say computer scientists at Stanford.

Flouting Betteridge’s Law of Headlines, Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo answer the question posed by their paper, Are Emergent Abilities of Large Language Models a Mirage?, in the affirmative.

[…]

When industry types talk about emergent abilities, they’re referring to capabilities that seemingly come out of nowhere for these models, as if something was being awakened within them as they grow in size. The thinking is that when these LLMs reach a certain scale, the ability to summarize text, translate languages, or perform complex calculations, for example, can emerge unexpectedly.

[…]

Stanford’s Schaeffer, Miranda, and Koyejo propose that when researchers are putting models through their paces and see unpredictable responses, it’s really due to poorly chosen methods of measurement rather than a glimmer of actual intelligence.

Most (92 percent) of the unexpected behavior detected, the team observed, was found in tasks evaluated via BIG-Bench, a crowd-sourced set of more than 200 benchmarks for evaluating large language models.

One test within BIG-Bench highlighted by the university trio is Exact String Match. As the name suggests, this checks a model’s output to see if it exactly matches a specific string without giving any weight to nearly right answers. The documentation even warns:

The EXACT_STRING_MATCH metric can lead to apparent sudden breakthroughs because of its inherent all-or-nothing discontinuity. It only gives credit for a model output that exactly matches the target string. Examining other metrics, such as BLEU, BLEURT, or ROUGE, can reveal more gradual progress.

The issue with using such pass-or-fail tests to infer emergent behavior, the researchers say, is that nonlinear output and lack of data in smaller models creates the illusion of new skills emerging in larger ones. Simply put, a smaller model may be very nearly right in its answer to a question, but because it is evaluated using the binary Exact String Match, it will be marked wrong whereas a larger model will hit the target and get full credit.

It’s a nuanced situation. Yes, larger models can summarize text and translate languages. Yes, larger models will generally perform better and can do more than smaller ones, but their sudden breakthrough in abilities – an unexpected emergence of capabilities – is an illusion: the smaller models are potentially capable of the same sort of thing but the benchmarks are not in their favor. The tests favor larger models, leading people in the industry to assume the larger models enjoy a leap in capabilities once they get to a certain size.

In reality, the change in abilities is more gradual as you scale up or down. The upshot for you and I is that applications may not need a huge but super powerful language model; a smaller one that is cheaper and faster to customize, test, and run may do the trick.

[…]

In short, the supposed emergent abilities of LLMs arise from the way the data is being analyzed and not from unforeseen changes to the model as it scales. The researchers emphasize they’re not precluding the possibility of emergent behavior in LLMs; they’re simply stating that previous claims of emergent behavior look like ill-considered metrics.

[…]

Source: LLM emergent behavior written off as ‘a mirage’ by study • The Register