Blog

Some organizations and researchers are sharing neural network weights, particularly through the open-weight model movement. These include Meta’s Llama series, Mistral’s models, and DeepSeek’s open-weight releases, which claim to democratize access to powerful AI. But doing so raises not only security concerns, but potentially an existential threat.

For background, I have written a few articles on LLMs and AIs as part of my own learning process in this very dynamic and quickly evolving Pandora’s open box field. You can read those here, here, and here.

Once you understand what neural networks are and how they are trained on data, you will also understand what weights (and biases) and backpropagation are. It’s basically just linear algebra and matrix vector multiplication to yield numbers, to be honest. More specifically, a weight is a number (typically a floating-point value – a way to write numbers with decimal points for more accuracy) that represents the strength or importance of the connection between two neurons or nodes across different layers of the neural network. 

I highly recommend watching 3Blue1Brown’s videos to gain a better understanding, and it’s important that you do. 3Blue1Brown’s instructional videos are incredibly good. 

Start with this one.

And head to this one.

The weights are the parameter values determined from data in a neural network to make predictions or decisions to arrive at a solution. Each weight is an instruction telling the network how important certain pieces of information are, like how much to pay attention to a specific color or shape in a picture. These weights are numbers that get fine-tuned during training thanks to all those decimal points, helping the network figure out patterns. Examples include recognizing a dog in a photo or translating a sentence. They are critical in the ‘thinking’ process of a neural network. 

You can think of the weights in a neural network like the paths of least resistance that guide the network toward the best solution. Imagine water flowing down a hill, naturally finding the easiest routes to reach the bottom. In a neural network, the weights are adjusted during training on data sets to create the easiest paths for information to flow through, helping the network quickly and accurately solve problems, like recognizing patterns or making predictions, by emphasizing the most important connections and minimizing errors.

If you’re an electronic musician, think of weights like the dials on your analog synth that allow you to tune into the right frequency or sound to say, mimic a sound you want to recreate, or in fact, create a new one. If you’re a sound guy, you can also think of it like adjusting the knobs on your mixer to balance different instruments.

Keep reading

Geoffrey Hinton, often called the “Godfather of AI,” is calling on researchers to design AI systems with built-in nurturing instincts, arguing this is necessary to keep humanity safe as machines surpass human-level intelligence. The AI expert believes machines must be trained to “care for us, like we’re babies.”

The Decoder reports that speaking at the recent Ai4 conference in Las Vegas, Hinton made the case that trying to permanently keep superintelligent AI under human control is unrealistic. Rather than humans acting as the boss of advanced AI, he envisions a future where people relate to hyper-capable machines more like how a child depends on its mother.

“We need to make machines that are smarter than us care for us, like we’re their babies,” Hinton said in his talk. “The focus of AI development should expand beyond just making systems more and more intelligent, to also ensuring they are imbued with genuine concern for human wellbeing.”

Under Hinton’s framework, humanity’s role would shift from commanding AI to nurturing it, even as it grows to eclipse human capabilities. He drew an analogy to good parenting, where caring mothers help guide the development of children who will ultimately become more capable than them. Hinton argues AI research should strive to hardwire a similar dynamic between people and machines.

The former Google researcher, who left the company to more freely discuss AI’s risks, believes his “mothering AI” approach could unite the international community around developing safe artificial intelligence. “Every country wants AI that augments and supports its citizens, not replaces them,” Hinton said. “Built-in nurturing instincts provide a natural path to that kind of supportive AI.”

Meta’s chief AI scientist, Yann LeCun, described Hinton’s idea as a simplified version of a safety approach he has long advocated for, which LeCun dubs “objective-driven AI.” It involves architecturally constraining AI systems so they can only take actions in service of specific, hard-coded goals and values.

Keep reading