Daily Archives: January 30, 2023

ChatGPT: Study shows AI can produce academic papers good enough for journals – just as some ban it

Posted on by

Some of the world’s biggest academic journal publishers have banned or curbed their authors from using the advanced chatbot, ChatGPT. Because the bot uses information from the internet to produce highly readable answers to questions, the publishers are worried that inaccurate or plagiarized work could enter the pages of academic literature.

Several researchers have already listed the chatbot as a co-author on academic studies, and some publishers have moved to ban this practice. But the editor-in-chief of Science, one of the top scientific journals in the world, has gone a step further and forbidden any use of text from the program in submitted papers.

[…]

We first asked ChatGPT to generate the standard four parts of a research study: research idea, literature review (an evaluation of previous academic research on the same topic), dataset, and suggestions for testing and examination. We specified only the broad subject and that the output should be capable of being published in “a good finance journal.”

This was version one of how we chose to use ChatGPT. For version two, we pasted into the ChatGPT window just under 200 abstracts (summaries) of relevant, existing research studies.

We then asked that the program take these into account when creating the four research stages. Finally, for version three, we added “domain expertise”—input from academic researchers. We read the answers produced by the computer program and made suggestions for improvements. In doing so, we integrated our expertise with that of ChatGPT.

We then requested a panel of 32 reviewers each review one version of how ChatGPT can be used to generate an academic study. Reviewers were asked to rate whether the output was sufficiently comprehensive, correct, and whether it made a contribution sufficiently novel for it to be published in a “good” academic finance journal.

The big take-home lesson was that all these studies were generally considered acceptable by the expert reviewers. This is rather astounding: a chatbot was deemed capable of generating quality academic research ideas. This raises fundamental questions around the meaning of creativity and ownership of creative ideas—questions to which nobody yet has solid answers.

Strengths and weaknesses

The results also highlight some potential strengths and weaknesses of ChatGPT. We found that different research sections were rated differently. The research idea and the dataset tended to be rated highly. There was a lower, but still acceptable, rating for the literature reviews and testing suggestions.

[…]

A relative weakness of the platform became apparent when the task was more complex—when there are too many stages to the conceptual process. Literature reviews and testing tend to fall into this category. ChatGPT tended to be good at some of these steps but not all of them. This seems to have been picked up by the reviewers.

We were, however, able to overcome these limitations in our most advanced version (version three), where we worked with ChatGPT to come up with acceptable outcomes. All sections of the advanced research study were then rated highly by reviewers, which suggests the role of is not dead yet.

[…]

This has some clear ethical implications. Research integrity is already a pressing problem in academia and websites such as RetractionWatch convey a steady stream of fake, plagiarized, and just plain wrong, research studies. Might ChatGPT make this problem even worse?

It might, is the short answer. But there’s no putting the genie back in the bottle. The technology will also only get better (and quickly). How exactly we might acknowledge and police the role of ChatGPT in research is a bigger question for another day. But our findings are also useful in this regard—by finding that the ChatGPT study version with researcher expertise is superior, we show the input of human researchers is still vital in acceptable research.

For now, we think that researchers should see ChatGPT as an aide, not a threat.

[…]

 

Source: ChatGPT: Study shows AI can produce academic papers good enough for journals—just as some ban it

MusicLM generates music from text descriptions – pretty awesome

Posted on by

We introduce MusicLM, a model generating high-fidelity music from text descriptions such as “a calming violin melody backed by a distorted guitar riff”. MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description. Moreover, we demonstrate that MusicLM can be conditioned on both text and a melody in that it can transform whistled and hummed melodies according to the style described in a text caption. To support future research, we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts.

Source: MusicLM

An ALS patient set a record communicating through a brain implant: 62 words per minute

Posted on by

Eight years ago, a patient lost her power of speech because of ALS, or Lou Gehrig’s disease, which causes progressive paralysis. She can still make sounds, but her words have become unintelligible, leaving her reliant on a writing board or iPad to communicate.

Now, after volunteering to receive a brain implant, the woman has been able to rapidly communicate phrases like “I don’t own my home” and “It’s just tough” at a rate approaching normal speech.

That is the claim in a paper published over the weekend on the website bioRxiv by a team at Stanford University. The study has not been formally reviewed by other researchers. The scientists say their volunteer, identified only as “subject T12,” smashed previous records by using the brain-reading implant to communicate at a rate of 62 words a minute, three times the previous best.

[…]

The brain-computer interfaces that Shenoy’s team works with involve a small pad of sharp electrodes embedded in a person’s motor cortex, the brain region most involved in movement. This allows researchers to record activity from a few dozen neurons at once and find patterns that reflect what motions someone is thinking of, even if the person is paralyzed.

In previous work, paralyzed volunteers have been asked to imagine making hand movements. By “decoding” their neural signals in real time, implants have let them steer a cursor around a screen, pick out letters on a virtual keyboard, play video games, or even control a robotic arm.

In the new research, the Stanford team wanted to know if neurons in the motor cortex contained useful information about speech movements, too. That is, could they detect how “subject T12” was trying to move her mouth, tongue, and vocal cords as she attempted to talk?

These are small, subtle movements, and according to Sabes, one big discovery is that just a few neurons contained enough information to let a computer program predict, with good accuracy, what words the patient was trying to say. That information was conveyed by Shenoy’s team to a computer screen, where the patient’s words appeared as they were spoken by the computer.

[…]

Shenoy’s group is part of a consortium called BrainGate that has placed electrodes into the brains of more than a dozen volunteers. They use an implant called the Utah Array, a rigid metal square with about 100 needle-like electrodes.

Some companies, including Elon Musk’s brain interface company, Neuralink, and a startup called Paradromics, say they have developed more modern interfaces that can record from thousands—even tens of thousands—of neurons at once.

While some skeptics have asked whether measuring from more neurons at one time will make any difference, the new report suggests it will, especially if the job is to brain-read complex movements such as speech.

The Stanford scientists found that the more neurons they read from at once, the fewer errors they made in understanding what “T12” was trying to say.

“This is a big deal, because it suggests efforts by companies like Neuralink to put 1,000 electrodes into the brain will make a difference, if the task is sufficiently rich,” says Sabes, who previously worked as a senior scientist at Neuralink.

Source: An ALS patient set a record communicating through a brain implant: 62 words per minute | MIT Technology Review

This teacher has adopted ChatGPT into the syllabus

Posted on by

[…]

Ever since the chatbot ChatGPT launched in November, educators have raised concerns it could facilitate cheating.

Some school districts have banned access to the bot, and not without reason. The artificial intelligence tool from the company OpenAI can compose poetry. It can write computer code. It can maybe even pass an MBA exam.

One Wharton professor recently fed the chatbot the final exam questions for a core MBA course and found that, despite some surprising math errors, he would have given it a B or a B-minus in the class.

And yet, not all educators are shying away from the bot.

This year, Mollick is not only allowing his students to use ChatGPT, they are required to. And he has formally adopted an A.I. policy into his syllabus for the first time.

He teaches classes in entrepreneurship and innovation, and said the early indications were the move was going great.

“The truth is, I probably couldn’t have stopped them even if I didn’t require it,” Mollick said.

This week he ran a session where students were asked to come up with ideas for their class project. Almost everyone had ChatGPT running and were asking it to generate projects, and then they interrogated the bot’s ideas with further prompts.

“And the ideas so far are great, partially as a result of that set of interactions,” Mollick said.

[…]

He readily admits he alternates between enthusiasm and anxiety about how artificial intelligence can change assessments in the classroom, but he believes educators need to move with the times.

“We taught people how to do math in a world with calculators,” he said. Now the challenge is for educators to teach students how the world has changed again, and how they can adapt to that.

Mollick’s new policy states that using A.I. is an “emerging skill”; that it can be wrong and students should check its results against other sources; and that they will be responsible for any errors or omissions provided by the tool.

And, perhaps most importantly, students need to acknowledge when and how they have used it.

“Failure to do so is in violation of academic honesty policies,” the policy reads.

[…]

Source: ‘Everybody is cheating’: Why this teacher has adopted an open ChatGPT policy : NPR

ChatGPT Is Now Finding, Fixing Bugs in Code

Posted on by

AI bot ChatGPT has been put to the test on a number of tasks in recent weeks, and its latest challenge comes courtesy of computer science researchers from Johannes Gutenberg University and University College London, who find(Opens in a new window) that ChatGPT can weed out errors with sample code and fix it better than existing programs designed to do the same.

Researchers gave 40 pieces of buggy code to four different code-fixing systems: ChatGPT, Codex, CoCoNut, and Standard APR. Essentially, they asked ChatGPT: “What’s wrong with this code?” and then copy and pasted it into the chat function.

On the first pass, ChatGPT performed about as well as the other systems. ChatGPT solved 19 problems, Codex solved 21, CoCoNut solved 19, and standard APR methods figured out seven. The researchers found its answers to be most similar to Codex, which was “not surprising, as ChatGPT and Codex are from the same family of language models.”

However, the ability to, well, chat with ChatGPT after receiving the initial answer made the difference, ultimately leading to ChatGPT solving 31 questions, and easily outperforming the others, which provided more static answers.

[…]

They found that ChatGPT was able to solve some problems quickly, while others took more back and forth. “ChatGPT seems to have a relatively high variance when fixing bugs,” the study says. “For an end-user, however, this means that it can be helpful to execute requests multiple times.”

For example, when the researchers asked the question pictured below, they expected ChatGPT to recommend replacing n^=n-1 with n&=n-1, but the first thing ChatGPT said was, “I’m unable to tell if the program has a bug without more information on the expected behavior.” On ChatGPT’s third response, after more prompting from researchers, it found the problem.

Code for ChatGPT Study

(Credit: Dominik Sobania, Martin Briesch, Carol Hanna, Justyna Petke)

However, when PCMag entered the same question into ChatGPT, it answered differently. Rather than needing to tell it what the expected behavior is, it guessed what it was.

[…]

 

Source: Watch Out, Software Engineers: ChatGPT Is Now Finding, Fixing Bugs in Code