Tag: technology

Open-Source AI Models Vulnerable to Criminal Misuse, Researchers Warn

Hackers and other criminals can easily commandeer computers operating open-source large language models outside the guardrails and constraints of the major artificial-intelligence platforms, creating security risks and vulnerabilities, researchers said on Thursday.

Hackers could target the computers running the LLMs and direct them to carry out spam operations, phishing content creation or disinformation campaigns, evading platform security protocols, the researchers said.

The research, carried out jointly by cybersecurity companies SentinelOne and Censys over the course of 293 days and shared exclusively with Reuters, offers a new window into the scale of potentially illicit use cases for thousands of open-source LLM deployments.

These include hacking, hate speech and harassment, violent or gore content, personal data theft, scams or fraud, and in some cases child sexual abuse material, the researchers said.

While thousands of open-source LLM variants exist, a significant portion of the LLMs on the internet-accessible hosts are variants of Meta’s Llama, Google DeepMind’s Gemma, and others, according to the researchers. While some of the open-source models include guardrails, the researchers identified hundreds of instances where guardrails were explicitly removed.

AI industry conversations about security controls are “ignoring this kind of surplus capacity that is clearly being utilized for all kinds of different stuff, some of it legitimate, some obviously criminal,” said Juan Andres Guerrero-Saade, executive director for intelligence and security research at SentinelOne.

Guerrero-Saade likened the situation to an “iceberg” that is not being properly accounted for across the industry and open-source community.

The research analyzed publicly accessible deployments of open-source LLMs deployed through Ollama, a tool that allows people and organizations to run their own versions of various large-language models.

The researchers were able to see system prompts, which are the instructions that dictate how the model behaves, in roughly a quarter of the LLMs they observed. Of those, they determined that 7.5% could potentially enable harmful activity.

Roughly 30% of the hosts observed by the researchers are operating out of China, and about 20% in the U.S.

Rachel Adams, the CEO and founder of the Global Center on AI Governance, said in an email that once open models are released, responsibility for what happens next becomes shared across the ecosystem, including the originating labs.

“Labs are not responsible for every downstream misuse (which are hard to anticipate), but they retain an important duty of care to anticipate foreseeable harms, document risks, and provide mitigation tooling and guidance, particularly given uneven global enforcement capacity,” Adams said.

A spokesperson for Meta declined to respond to questions about developers’ responsibilities for addressing concerns around downstream abuse of open-source models and how concerns might be reported, but noted the company’s Llama Protection tools for Llama developers, and the company’s Meta Llama Responsible Use Guide.

Microsoft AI Red Team Lead Ram Shankar Siva Kumar said in an email that Microsoft believes open-source models “play an important role” in a variety of areas, but, “at the same time, we are clear-eyed that open models, like all transformative technologies, can be misused by adversaries if released without appropriate safeguards.”

Keep reading

430,000-Year-Old Wooden Tools Are the Oldest Ever Found 

Early hominins in Europe were creating tools from raw materials hundreds of thousands of years before Homo sapiens arrived there, two new studies indicate, pushing back the established time for such activity. From a report:The evidence includes a 500,000-year-old hammer made of elephant or mammoth bone, excavated in southern England, and 430,000-year-old wooden tools found in southern Greece — the earliest wooden tools on record.

The findings suggest that early humans possessed sophisticated technological skills, the researchers said. Katerina Harvati, a paleoanthropologist at the University of Tubingen in Germany and a lead author of the wooden-tool paper, which was published on Monday in the journal PNAS, said the discoveries provided insight into the prehistoric origins of human intelligence. Silvia Bello, a paleoanthropologist at London’s Natural History Museum and an author on the elephant-bone study, which was published last week in Science Advances, concurred.

The artifacts in both studies, recovered from coal-mine sites, were probably produced by early Neanderthals or a preceding species, Homo heidelbergensis. Homo sapiens emerged in Africa more than 300,000 years ago, and the oldest evidence of them in Europe is a 210,000-year-old fossil unearthed in Greece. By the time Homo sapiens established themselves in Britain 40,000 years ago, other hominins had already lived there for nearly a million years.

Keep reading

Scientists mimicking the Big Bang accidentally turn lead into gold

Medieval alchemists dreamed of transmuting lead into gold.

Today, we know that lead and gold are different elements, and no amount of chemistry can turn one into the other.

But our modern knowledge tells us the basic difference between an atom of lead and an atom of gold: the lead atom contains exactly three more protons. So can we create a gold atom by simply pulling three protons out of a lead atom?

As it turns out, we can. But it’s not easy.

While smashing lead atoms into each other at extremely high speeds in an effort to mimic the state of the universe just after the Big Bang, physicists working on the ALICE experiment at the Large Hadron Collider in Switzerland incidentally produced small amounts of gold.

Extremely small amounts, in fact: a total of some 29 trillionths of a gram.

How to steal a proton

Protons are found in the nucleus of an atom. How can they be pulled out?

Well, protons have an electric charge, which means an electric field can pull or push them around. Placing an atomic nucleus in an electric field could do it.

However, nuclei are held together by a very strong force with a very short range, imaginatively known as the strong nuclear force. This means an extremely powerful electric field is required to pull out protons – about a million times stronger than the electric fields that create lightning bolts in the atmosphere.

Keep reading

AI Can Match Average Human Creativity—But We Still Hold the Edge Where It Matters Most, New Study Finds

Advances in artificial intelligence have fueled a growing belief that machines are on the verge of matching, or even surpassing, human creativity. Large language models can now write poems, spin short stories, and generate clever wordplay in seconds. To many, these outputs feel creative enough to blur the line between human imagination and machine-generated language.

However, a new large-scale empirical study suggests that while today’s most advanced AI systems can rival the average human on certain creativity measures, they still fall short of the most creative minds—and that gap remains significant.

The research, published in Scientific Reports, offers one of the most comprehensive head-to-head comparisons yet between human creativity and large language models (LLMs).

By benchmarking multiple AI systems against a dataset of 100,000 human participants, the study moves the conversation beyond anecdotes and viral examples, replacing speculation with quantitative evidence.

“Our study shows that some AI systems based on large language models can now outperform average human creativity on well-defined tasks,” co-author and Professor at the University of Montreal, Dr. Karim Jerbi, said in a press release. “This result may be surprising — even unsettling — but our study also highlights an equally important observation: even the best AI systems still fall short of the levels reached by the most creative humans.”

Keep reading

EU Targets X (Again) in Grok AI Probe

European regulators have launched a new investigation into Elon Musk’s X, focusing on alleged failures to control sexually explicit imagery generated by the company’s AI chatbot, Grok.

The case is being pursued under the European Union’s Digital Services Act (DSA), a law that grants the European Commission expansive powers to police digital platforms for potential “harms.”

In a statement, the Commission said, “The new investigation will assess whether the company properly assessed and mitigated risks associated with the deployment of Grok’s functionalities into X in the EU.”

The agency added that the review includes “risks related to the dissemination of illegal content in the EU, such as manipulated sexually explicit images, including content that may amount to child sexual abuse material.” Officials stated that these threats “seem to have materialized, exposing citizens in the EU to serious harm.”

Keep reading

Britain To Roll Out Facial Recognition in Police Overhaul

Britain’s policing system, we are told, is broken. And on Monday, the home secretary, Shabana Mahmood, announced that the fix would arrive in the form of algorithms, facial recognition vans, and a large check made out to the future.

The government plans to spend £140m ($191M) on artificial intelligence and related technology, with the promise that it will free up six million police hours a year, the equivalent of 3,000 officers.

It is being billed as the biggest overhaul of policing in England and Wales in 200 years, aimed at dragging a creaking system into the modern world.

The ambition is serious. The implications are too.

The plan is for AI software that will analyze CCTV, doorbell, and mobile phone footage, detect deepfakes, carry out digital forensics, and handle administrative tasks such as form filling, redaction, and transcription. Mahmood’s argument is that criminals are getting smarter, while parts of the police service are stuck with tools that belong to another era.

She put it plainly: “Criminals are operating in increasingly sophisticated ways. However, some police forces are still fighting crime with analogue methods.”

And she promised results: “We will roll out state-of-the-art tech to get more officers on the streets and put rapists and murderers behind bars.”

There is logic here. Few people would argue that trained officers should be buried in paperwork. Technology can help with that. The concern is what else comes with it.

Live facial recognition is being expanded aggressively. The number of police vans equipped with the technology will increase fivefold, from ten to fifty, operating across the country. These systems scan faces in public spaces and compare them to watch lists of wanted individuals.

This is a form of mass surveillance and when automated systems get things wrong, the consequences fall on real people.

Keep reading

Somali Member of Minnesota House of Representatives Uses Doctored Image of Alex Pretti in Memorial Tribute

Minnesota State Representative Mohamud Noor has come under fire for using what appears to be an AI-enhanced or filtered image of Alex Pretti, the armed man shot by a federal agent in Minneapolis over the weekend, in a social media memorial post.

The bizarrely altered photo appears to have been manipulated to make Pretti look more attractive.

Pretti, 37, was fatally shot after he engaged in a scuffle with federal agents who were in the middle of an immigration enforcement operation.

Noor, a Democrat who represents District 60B in the Minnesota House, posted a tribute to Pretti on X, writing:

“Words can’t describe our pain. Our hearts are broken, but our spirit is strong. Alex Pretti was killed standing for his neighbors. This has to stop. Our community deserves safety, transparency, and accountability. RIP Alex.”

However, the image he used bears absolutely no resemblance to the original. It appeared he used AI tools or heavy filters to “enhance” Pretti’s appearance, making him look more conventionally attractive by smoothing his skin, thickening his hair, and altering facial features.

Naturally, Noor was absolutely blasted in the replies and quote posts.

“Why are you guys trying to make him better looking through AI and filters? That’s actually twisted,” one user wrote.

Keep reading

The Tech Community’s Efforts to Dethrone OpenAI

OpenAI has made tech waves in the recent years given the prominences of the ChatGPT family of models, and the remanent of LLMs as search engine reindexing algorithms. They were a private research entity that became a titan now competing with the likes of Google. However, their story is less than glamorous.

They started out as a non-profit funded by Musk only to be insanely profit-driven. In fact, they are a cash-burn enterprise, and on top of that there are concerns based off of the localization of AI search results, privacy concerns over social prompt injecting, the suspicious death of whistleblower Suchir Balaji, and questions on whether these LLMs, particularly OpenAI are becoming digitized religions. This all put ChatGPT in the spotlight in a negative sense, and on top of the already burning fire were the Ann Altman allegations. The biggest issue, however, is that OpenAI is extremely centralized and has a business model that is based off of incentivizing data harvesting.

On the other hand, there are researchers like me and the growing cyberpunk community who have been working on AI research for many years. The straw that broke the camel’s back for me was the localization and privacy concerns that OpenAI has raised. This led me to build AI systems based off of open peering that aims to democratize LLMs and AI applications.

Keep reading

Bay Area scientist launches new company with sights on gene-edited babies

Last month, as he announced the launch, he said that Preventive has raised almost $30 million from private funding.

The funding is reportedly coming from some heavy hitters in the tech world, including OpenAI CEO Sam Altman and his husband Oliver Mulherin.

Harrington also said his team included leading experts in the fields of reproductive technology, reproductive medicine and genome-editing.

“Our goal is straightforward,” he wrote, “to determine through rigorous preclinical work whether preventive gene editing can be developed safely to spare families from severe disease.”

Harrington acknowledged the major ethical concerns around the science and the gray areas in the regulatory process, which he said, have opened the field to potentially detrimental outcomes. 

“The combination of limited expert involvement and lack of a clear regulatory pathway has created conditions for fringe groups to take dangerous shortcuts that could harm patients and stifle responsible investigation,” the researchers said, adding, “Given that this technology has the potential to save millions of lives, we do not want this to happen.”

Gene editing can only be used in in vitro fertilization to allow for the first step of genetic testing on an embryo.

“It requires IVF because you have to have the embryo in a dish,” explained Stanford law professor Henry (Hank) Greely, a leading expert on ethical, legal, and social implications in bioscience technologies.

Once a test determines an embryo has the DNA makeup of a genetic disease, for example, like Huntington’s or cystic fibrosis, scientists would then use the DNA editing technique known as Clustered Regularly Interspaced Short Palindromic Repeats, or CRISPR, to make alterations to the DNA.

Keep reading

New insight into light-matter thermalization could advance neutral-atom quantum computing

Light and matter can remain at separate temperatures even while interacting with each other for long periods, according to new research that could help scale up an emerging quantum computing approach in which photons and atoms play a central role.

In a theoretical study published in Physical Review Letters, a University at Buffalo-led team reports that interacting photons and atoms don’t always rapidly reach thermal equilibrium as expected.

Thermal equilibrium is the process by which interacting particles exchange energy before settling at the same temperature, and it typically happens quickly when trapped light repeatedly interacts with matter. Under the right circumstances, however, physicists found that photons and atoms can instead settle at different—and in some cases opposite—temperatures for extended periods.

Implications for quantum computing

These so-called prethermal states are fleeting on human timescales, but they can last long enough to matter for neutral-atom quantum computers, which rely on interactions between photons and atoms to store and process information.

“Thermal equilibrium alters quantum properties, effectively erasing the very information those properties represent in a quantum computer,” says the study’s lead author, Jamir Marino, Ph.D., assistant professor of physics in the UB College of Arts and Sciences. “So delaying thermal equilibrium between photons and atoms—even for a matter of milliseconds—offers a temporal window to preserve and process useful quantum behavior.”

All quantum computers store and process information using qubits—the most basic units of quantum information and analogous to the binary bits used in classical computers. While classical bits can exist either as a 1 or a 0, qubits have the ability to exist in a superposition of two states at once, allowing for infinitely more complex calculations.

Keep reading