OpenAI announced Wednesday that it will play a role in combating “hallucinations” in AI with a new method for training artificial intelligence models.
The research comes at a time when misinformation caused by AI systems is more hotly debated than ever in the run-up to the generative AI boom and the 2024 US presidential election.
Last year, OpenAI accelerated the generative AI boom by releasing ChatGPT, a chatbot powered by GPT-3 and GPT-4, becoming the fastest growing with over 100 million monthly users in two months. Reportedly set app records. to date, microsoft has invested over $13 billion in OpenAI, valuing the startup at around $29 billion.
AI’s hallucinations are similar to OpenAI’s ChatGPT and GoogleThe bard has completely fabricated the information and behaves as if he is spouting facts. One example: In his February promotional video for Bar on Google, the chatbot says: false claims About the James Webb Space Telescope. Most recently, ChatGPT cited a “fake” case in a filing in New York federal court, in which the New York state attorneys involved could face sanctions.
“Even state-of-the-art models tend to be false, and to fabricate facts in moments of uncertainty,” the OpenAI researchers wrote in their report. “These hallucinations are particularly problematic in areas that require multi-stage reasoning, as a single logic error is enough to derail larger-scale solutions.”
OpenAI’s potential new strategy for combating hoaxes: rather than rewarding correct final conclusions, train AI models to reward each correct step of reasoning in reaching an answer. The researchers say the approach is called “process monitoring” rather than “outcome monitoring,” and this strategy leads to more explainable AI because it forces models to follow a more human-like “thought” chain approach. It is said that there is a possibility.
“Detecting and mitigating logic errors and hallucinations in models is an important step in building a tuned AGI. [or artificial general intelligence]Karl Kobbe, a mathgen researcher at OpenAI, told CNBC that while OpenAI didn’t invent the process monitoring approach, the company is helping drive it forward. “The motivation behind this study is to address hallucinations in order to create models that are more capable of solving difficult reasoning problems.” “
Cobb said OpenAI has released an accompanying dataset of 800,000 human labels that it used to train the model mentioned in the research paper.
Ben Winters, senior adviser to the Electronic Privacy Information Center and leader of the AI and Human Rights Project, expressed skepticism and told CNBC that he wanted to explore the full dataset and accompanying examples.
“I doubt that this alone would significantly alleviate concerns about misinformation and misleading results in real-world use,” Winters said. “It’s definitely important whether they have plans to implement what they’ve found here,” he added. [into their products]If not, it raises some pretty serious questions about what they’re trying to expose to the public. ”
As it is unclear whether the OpenAI paper has been peer-reviewed or otherwise, Suresh Venkatasbramanian, director of the Center for Technology Responsibility at Brown University, told CNBC that the research is preliminary above all. He said he thought it was an important observation.
“This issue needs to be resolved within the research community to say anything with certainty about it,” Venkatasbramanian said. “In this world there are a lot of results that come up very regularly, but because of the general instability of how large language models work, in certain settings, models and contexts, What might work may not work in another setting, model, or context.”
Venkatasbramanian added, “Some of the hallucinations that people are concerned about are [models] Create citations and references. There is no evidence in this paper that this works for that. …I’m not saying it won’t work. This paper does not provide that evidence. ”
Cobb said the company is “likely to file.” [the paper] OpenAI did not respond to a request for comment on when, if any, the company plans to implement the new strategy into ChatGPT and other products.
“It is certainly welcome to see companies trying to devise systems to reduce these types of errors. I think it’s about interpreting it as research,” Sarah Myers-West, managing director of the AI Now Institute, told CNBC.
West further added,[OpenAI is] Although this paper publishes a small dataset of human-level feedback, it does not provide basic details about the data used to train and test GPT-4. So even though these systems already directly impact people, there is still a huge amount of uncertainty that hinders meaningful accountability efforts in the field of AI. ”
