Where It’s Already Gone Wrong
Netherlands’ childcare benefits scandal – 2021
Automated risk profiling and aggressive enforcement mislabelled thousands of families as fraudsters. Debt payments were incorrectly demanded from genuine cases, the system was shaken, and the political fallout triggered the government’s resignation.
Denmark’s failed welfare algorithm – 2024 to 2025
Dozens of fraud detection models monitored benefits claimants. Rights group Amnesty International reported that the algorithms risk mass surveillance and discrimination against marginalised groups. The systems remained in use as scrutiny continued into 2025.
France’s predictive policing backlash – 2025
Civil society documented predictive policing deployments and called in May 2025 for an outright ban. The evidence shows hotspot forecasting and risk tools that are opaque and likely to reproduce bias. These systems are trained on historic data which sends officers back to the same neighbourhoods that may already have been over policed, while very little is done to educate the masses on how it works and there’s no credible path to appeal.
USA expands biometric border checks – 2025
Facial comparisons run at hundreds of airports, seaports and land borders. Opt outs apparently exist but are confusing to most, and accuracy varies by demographic with transparent figures yet to surface. Human lines reportedly move slower than automated ones, turning the convenience into indirect pressure to adhere to the new technology.
Australia’s Robodebt fallout and new automation faults – 2023 to 2025
A Royal Commission found the automated debt scheme unlawful and harmful. In 2025, watchdogs flagged thousands of wrongful JobSeeker cancellations tied to IT glitches in the Target Compliance Framework. Strategies were published and apologies made, yet incentives still rewarded speed over care.
India’s ongoing biometric failures – 2025
Biometric failures and outages have blocked rations and benefits for many. Authorities are testing facial recognition to patch fingerprint failures and vice versa, but if one biometric fails and another is layered on top, error can spread across services that depend on the same ID.
Hellbound and Down 