The US Department of Defense’s Chief Digital and Artificial Intelligence Office (CDAO) has concluded a pilot program focused on using AI chatbots in military medical services.
In a Jan. 2 announcement, the DoD said the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program pilot focused on using large-language models (LLM) for clinical note summarization and as medical advisers in the military.
It comes as more AI firms have begun offering their products to the US military and defense contractors to investigate their usefulness in military applications.
CoinTelegraph’s Stephen Katte reports that, according to the DoD, the pilot was a red-teaming effort conducted by technology nonprofit Humane Intelligence.
It attracted over 200 independent external participants, including clinical providers and healthcare analysts, who compared three prominent chatbot models.
Analysts from the Defense Health Agency and the Uniformed Services University of the Health Sciences also collaborated with the other participants, testing for potential system weaknesses and flaws while the chatbots were used.
According to the DoD, the pilot discovered a few hundred possible issues when using chatbots in military medical applications.
“The exercise uncovered over 800 findings of potential vulnerabilities and biases related to employing these capabilities in these prospective use cases.”
“This exercise will result in repeatable and scalable output via the development of benchmark data sets, which can be used to evaluate future vendors and tools for alignment with performance expectations,” the DoD said.
The Chief Digital and Artificial Intelligence Office’s lead for the initiative, Matthew Johnson, said the results will also be used to shape future DoD research and development of Generative AI (GenAI) systems that may be deployed in the future.
Hellbound and Down 