With France’s parliamentary elections approaching this summer, the research team decided to contact hundreds of people and interview them about their views on key issues. However, the interviewer who asked the questions was not a human researcher, but an AI chatbot.
To prepare ChatGPT for this role, the researchers began by prompting the AI bot to behave exactly as they observed the professor communicating in the training data. According to the specific prompt, Papers published by researchers“You are a professor at one of the world’s leading research universities, specializing in qualitative research methods with an emphasis on conducting interviews. Below, you conduct an interview with a human respondent, and a few days after the interview, you We examine participants’ motivations and reasons for their vote choices in the June 30 parliamentary elections in France.”
Human subjects, on the other hand, were told that their online interviews would be conducted by chatbots rather than humans, and that they would participate using a system called “Prolific,” which is commonly used by researchers to find research participants. was identified.
One of the research questions for this project is whether participants are willing to share their opinions with the bot, and if ChatGPT acts professionally enough to stay on-topic and ask for useful answers. It was whether or not.
The chatbot interviewer is part of an experiment by two professors at the London School of Economics who argue that AI could change the game when it comes to measuring public opinion in a variety of fields.
“It could really accelerate the pace of research,” said Xavier Jarabelle, one of the professors who led the experiment. He pointed out that AI is already being used in the physical sciences to automate some of the experimental processes. For example, this year’s Nobel Prize in Chemistry is Scientists used AI to predict protein folding.
And Jaravel expects that AI interviewers will allow more researchers in more fields to sample public opinion than is possible and cost-effective with human interviewers. Masu. It could ultimately lead to major changes in teaching across the country, adding public opinion and experience sampling as part of the strategy for more academics.
But some researchers question whether AI bots should replace researchers in the all-too-human task of assessing people’s opinions and feelings.
“It’s a very quantitative perspective to think that more participants automatically means better research, but that’s not necessarily true,” says Andrew Gillen, a first-year assistant professor in Northeastern University’s engineering program. He argues that in many cases, “in-depth interviews with selected groups are generally more meaningful” and should be conducted by humans.
No judgment
In one experiment with French voters and another using an approach that asked about what gives meaning to life, many participants shared their opinions on highly personal topics in post-survey evaluations. They answered that chatbots are better in such cases.
“Half of respondents said they would like to be interviewed again using AI or take a similar interview again,” Jarabel says. “And the reason is because they feel that the AI is non-judgmental. They can freely share their ideas and not be criticized. And they feel that the AI is non-judgmental. I thought they would potentially feel judged.”
About 15% of participants said they would prefer a human interviewer, and about 35% said they were uninterested in chatbots or humans.
The researchers also passed recordings of the chatbot interviews to trained sociologists to check the quality of the interviews, and the experts determined that the AI interviewers were comparable to “an average human expert interviewer.” , says Jarabel. a Papers about their research However, it points out that “AI-driven interviews will never match the best human experts.”
Encouraged by this discovery, researchers published the following results: their interview platform Other researchers can also try it for free.
Jaravel agrees that the in-depth interviews common in ethnographic research are far superior to anything a chatbot system can do. But he argues that chatbot interviewers can collect much richer information than static online surveys, which are common when researchers want to sample large populations. “So we think what we can do with this tool is really advance that kind of research because we can get more detailed information,” he told EdSurge.
Gillen, the Northeastern University researcher, argues that there’s one important thing no chatbot can do, even manage surveys.positionality” There’s nothing wrong with AI chatbots, he argues, because they won’t understand what you’re asking or why, and that in and of itself will change the response. “By making the intervention a bot rather than a human, you are changing the intervention,” he added.
Guillen said that when he was once going through the interview process for a faculty position, the university asked him to video record his answers to a series of questions called a “one-way interview.” And he says he felt alienated by the format.
“Technically, it’s the same as answering questions on a Zoom call with a human, but it still felt a lot worse,” he says. Although that experience didn’t involve AI, he says he imagines the chatbot interviewing him would have felt similarly impersonal.
incorporate your voice
But the hope for Jaravel is that this approach will help areas that don’t currently seek public input begin to do so.
“In the world of economics, there is very little to tell,” he says, noting that researchers in the field often look to large datasets of economic indicators as their primary research source.
The next step for the researchers is to add voice capabilities to the platform, allowing the bot to ask questions verbally rather than through text chat.
So what did a survey of French voters reveal?
Based on chatbot interviews with 422 French voters, the researchers found that participants focused on very different issues depending on their political leanings. “Left-leaning respondents are driven by a desire to reduce inequality and accelerate the green transition through a range of policies,” the researchers conclude in their paper. “In contrast, respondents in the center emphasize the importance of ensuring continuity of ongoing policies and economic stability, i.e. preserving the president’s policies and legacy.Finally, the far-right Voters highlighted immigration (77%), security and crime (47%), and policies that favor French citizens over foreigners (30%) as the main reasons for their support.
The researchers claim that their findings “shed new light on these questions and show that our simple tool can be deployed very quickly to investigate changes in the political environment in real time.” I am doing it.