AI Chatbots Rely On Sources With Clear Biases

AI chatbots like ChatGPT and Grok can be a big help in writing essays, conducting research, and exploring complex issues. But these tools bring risks, especially when they filter facts through a political lens. And the Trump administration is now stepping into the debate. “We believe AI systems should operate free of ideological bias and avoid pushing socially engineered agendas,” said David Sacks, the administration’s AI and crypto czar, in a statement today. “We’ve introduced several proposals to ensure AI stays truth-seeking and trustworthy.”

Over the weekend, I saw this bias unfold in real time.

On Friday, a user on Elon Musk’s platform X asked Grok whether more guns make Americans safer. Grok responded flatly: “No, evidence shows more guns correlate with higher firearm homicides and violent crime rates.” The chatbot dismissed self-defense and deterrence, referring to my research –specifically my “more guns, less crime” theory – as something cited by “right-wing advocates.” Grok supported its claims by referencing Scientific American magazine and a RAND Corporation review, saying these sources show guns don’t reduce crime and instead increase violence.

Those answers are misleading and wrong.

The Scientific American article had extensive biases. Grok ignored my published rebuttal in Scientific American. In it, I noted that over two-thirds of peer-reviewed studies show that concealed carry laws do reduce crime. Melinda Wenner Moyer, a journalist affiliated with Michael Bloomberg’s The Trace, a well-known gun control advocacy outlet, wrote the article. I had provided Moyer with those studies while she prepared her piece, but she ignored them. She failed to acknowledge any of my post-1998 work and misrepresented the findings of the National Research Council’s major report on the topic.

Grok gave tremendous weight to RAND’s literature survey, claiming that RAND had surveyed 100+ studies. Eventually, Grok conceded that the number of papers studying right-to-carry laws was actually 25, showing a range of mixed results. I pointed out that the California-based think tank was highly selective in the sources it included, ignoring dozens more papers showing that these laws lowered violent crime rates and surveys of academics who have published peer-reviewed empirical research.

Even then, Grok largely ignored my responses and focused on two papers claiming that right-to-carry laws increased violent crime. The first failed to control for any variables – such as changes in policing, poverty, or economic conditions – that affect crime trends after adopting right-to-carry laws. When I pointed that out, Grok mentioned another study that demonstrated a statistical technique that could account for such factors, but that study didn’t look at right-to-carry laws. Only after a prolonged exchange did Grok acknowledge the error.

Keep reading

Unknown's avatar

Author: HP McLovincraft

Seeker of rabbit holes. Pessimist. Libertine. Contrarian. Your huckleberry. Possibly true tales of sanity-blasting horror also known as abject reality. Prepare yourself. Veteran of a thousand psychic wars. I have seen the fnords. Deplatformed on Tumblr and Twitter.

Leave a comment