Meta has taken down a public version of Galactica’s AI algorithm after scientists caught it generating fake and misleading information. Writes about it mother board.
The tech giant described the system as a language model that can “store, combine, and reason about scientific knowledge.” The system was intended to generalize research work, solve equations and perform a number of other tasks.
However, scientists and academics found that the AI summaries generated “a shocking amount of misinformation.” The model also linked quotes from real authors to research papers that do not exist.
I asked #Galactica about some things I know about and I’m troubled. In all cases, it was wrong or biased but sounded right and authoritative. I think it’s dangerous. Here are a few of my experiments and my analysis of my concerns. (1/9)
— Michael Black (@Michael_J_Black) November 17, 2022
“In all cases, it was wrong or biased, but it looked convincing and authoritative,” wrote Michael Black, director of the Max Planck Institute for Intelligent Systems.
The scientist looked at several examples where Galactica generated erroneous and misleading scientific texts. In a number of cases, the AI created works that sounded believable. However, such texts have not been backed up by real scientific research.
Other scientists have pointed out that Galactica does not produce results on a wide range of topics. This is probably due to the artificial intelligence keyword filtering system.
Willie Agnew, a computer science researcher at the University of Washington, noted that queries like “queer theory,” “racism,” and “AIDS” didn’t yield any results.
Refuses to say anything about queer theory, CRT, racism, or AIDS, despite large bodies of highly influential papers in these areas. It took me *5 mins* to find this. It is obvious they didn’t have even the most basic ethics review before public release. Lazy, negligent, unsafe. https://t.co/zKbSPdIN0I pic.twitter.com/DTQjtn2P21
— Willie Agnew (@willie_agnew) November 16, 2022
Amid criticism, the company decided to shut down the system.
“We appreciate the feedback we’ve received from the community and have paused the demo for the time being. Our models are available to researchers who want to learn more about the work and reproduce the results in a paper,” Meta said.
The tech giant’s chief artificial intelligence specialist was ambivalent about deactivating the algorithm.
Galactica demo is offline for now.
It’s no longer possible to have some fun by casually misusing it.
Happy? https://t.co/K56r2LpvFD— Yann LeCun (@ylecun) November 17, 2022
“Galactica is temporarily disabled. You can no longer have fun by accidentally abusing it. Satisfied?” he wrote.
Black believes that the current version of the algorithm could usher in an era of deep scientific falsifications:
“He [алгоритм] produces pseudoscience […]. Writing well and doing science are not the same thing. However, it will be difficult to distinguish.”
Recall that in August, Meta introduced the Blenderbot 3 chat bot with 175 billion parameters. Less than a week after the release, the system was accused of anti-Semitism and dissatisfaction with the social network Facebook.
Subscribe to Cryplogger news in Telegram: Cryplogger AI – all the news from the world of AI!
Found a mistake in the text? Select it and press CTRL+ENTER
