Despite being wild Despite the success of ChatGPT and other large-scale language models, the artificial neural networks (ANNs) underpinning these systems may be headed in the wrong direction.
For one thing, ANN is “extremely power-hungry.” Cornelia Vermüller, a computer scientist at the University of Maryland. “And another problem is that [their] Lack of transparency. A system like this is so complex that no one really understands what it does or why it works so well. This makes it nearly impossible for humans to use symbols to reason by analogy with objects, ideas, and the relationships between them.
Such shortcomings can be attributed to the current structure of ANNs and their constituent individual artificial neurons. Each neuron receives inputs, performs computations, and produces outputs. His modern ANN is an elaborate network of these computational units trained to perform a specific task.
However, the limitations of ANNs have long been apparent. For example, consider an ANN that distinguishes between circles and rectangles. One way to do this is to have the output layer have two neurons, one representing a circle and one representing a square. If you want the ANN to also identify the color of the shape (such as blue or red), you need four output neurons, one for the blue circle, one for the blue square, one for the red circle, and one for the red square. More functions means more neurons.
This cannot be how our brains perceive the natural world with its many variations. “We should propose that there is a neuron for every combination,” he said. Bruno Olshausen, a neuroscientist at the University of California, Berkeley. “So in your mind you’re thinking: [say,] A purple Volkswagen detector. “
Instead, Olshausen and others They argue that information in the brain is represented by the activity of numerous neurons. The perception of the purple Volkswagen is thus encoded not as the action of a single neuron, but as the action of thousands of neurons. The same set of neurons firing in different ways can represent completely different concepts (like a pink Cadillac).
This is the starting point for a radically different computational approach known as hyperdimensional computing. The point is that concepts such as a car, its make, model, color, or all of them are represented in his single entity: a hyperdimensional vector.
A vector is simply an ordered array of numbers. For example, a 3D vector consists of three numbers. X, yes, and z Coordinates of a point in 3D space. A hyperdimensional vector (hypervector) can be, for example, an array of 10,000 numbers representing points in a 10,000-dimensional space. These mathematical objects and the algebra that manipulates them are flexible and powerful enough to push modern computing beyond some of its current limitations and to facilitate new approaches to artificial intelligence. .
“This is the most exciting thing in my entire career,” said Olshausen. For him and many others, hyperdimensional computing promises a new world where computing is efficient and robust, and machine decisions are completely transparent.
Entry into high-dimensional space
To understand how hypervectors enable computing, let’s go back to the red circle and blue square image. First, we need a vector representing the variables SHAPE and COLOR. Next, we also need a vector of values that can be assigned to the variables (CIRCLE, SQUARE, BLUE, RED).
Vectors must be distinct. This distinguishability can be quantified by a property called orthogonality. Orthogonality means at right angles. In 3D space there are three vectors that are orthogonal to each other. X direction, another direction yes, and one-third z. There are 10,000 such mutually orthogonal vectors in a 10,000-dimensional space.
