Hewitt Crane was a practical minded kind of guy. To help the world get a better feel for just how much oil it used in a year, he came up the unit he called the cubic mile of oil (CMO), to considerable acclaim. Crane was actually one of the pioneers of computing. He was an early developer of magnetic core RAM, eye-tracking devices, pen input devices, and invented the first all-magnetic computers still finding extensive use for fail-safe systems in the military. Today, another kind of device he presciently envisioned back in 1960 is starting to attract attention — the neuristor.
A neuristor is the simplest possible device that can capture the essential property of a neuron — that is, the ability to generate a spike or impulse of activity when some threshold is exceeded. A neuristor can be thought of as a slightly leaky balloon that receives inputs in the form of puffs of air. When its limit is reached, it pops. The only major difference is that more complex neuristors can repeat the process again and again, as long as spikes occur no faster than a certain recharge period known as the refractory period. A neuristor uses a relatively simple electronic circuit to generate spikes. Incoming signals charge a capacitor that is placed in parallel with a device called a memristor. The memristor behaves like a resistor except that once the small currents passing through it start to heat it up, its resistance rapidly drops off. When that happens, the charge built up on the capacitor by incoming spikes discharges, and there you have it — a spiking neuron comprised of just two elementary circuit elements.
The details of trying to build neuristors have been largely unglamorous. In fact, most people trying to create artificial brains with neuristor-like elements have been content to do so by simulating them on regular old digital computers. The Spaun computer comprised of 2.5 million neurons is a recent example. A group at HP Labs has come up with an improved method of fabricating a neuristor made from niobium dioxide (NbO2). For now, their prototype devices are too inefficient to be used in large numbers on a single chip, but the group is developing ways to make them more compatible with current fabrication methods. This new study does not deny long-standing efforts of the neuromorphic community to build analog neurons into chips, it is just the next step towards getting people on board with neuristor hardware instead of neuristor simulation.
When spikes beat bits
In a good electronics course, the first and simplest computer you usually build is an analog device that uses just a handful of simple amplifier chips to map and and solve a rudimentary equation. The fact that digital computers waste a lot of energy just to do a small amount of computation was not lost on the energy-conscious mind of Hewitt. Taking a cue from the brain, he formulated ideas for building devices that would exchange spikes rather than bits. In principle, for spiking networks, the only thing that absolutely requires energy is the spike. A spike doesn’t need any ordered reference structure like a bit — it can become whatever is desired.A group of fireflies exchanging flashes, crows cawing, or locusts emitting seasonal calls, are all examples of spiking networks. The individual integrates these spikes of information, emitted in the local currency by its peers, and feeds spikes back to system as the behavior of the group evolves. What the spikes eventually come to represent is in many ways indefinably unique to the particular unit that generates them.
While for machines this would make them a bit more difficult to interpret or reproduce, it makes them infinitely easier to program — the hardware is the software. If it becomes apparent that the system needs to internalize more information about say, a visual scene, it just tosses in some more neuristors and lets them competitively integrate or die according to the properties of the network.
When a newborn first opens their eyes to the world, those first rays of light modify the founding circuitry of the retina and brain that has been roughed out according to the genes. Initially random, spiking activity begins to acquire meaning relative to external world. In this way, spikes can be considered a baggage-free way to represent sensory input or motor output, among other things. It is certainly possible to represent bits with spikes, and spikes with bits, but the conversion is usually imperfect. For example, if we put an obstacle in the way of a fly in an experimental setup, the fly can make an adjustment to its flight path in about 30ms. A single neuron in the fly brain detects the visual stimulus captured by photoreceptors contacting its input neurites, and signals the flight muscle contacted directly by its output neurites. In this context we might then ask, how much information in bits per second is transmitted by this neuron?
If for the sake of argument, we suppose that this particular neuron can spike at a maximum of 500Hz, it follows that in the 30ms behavioral window there would be 15 intervals, each 2ms long, where a spike may or may not occur. There are then 215, or 32736 possible spike “words” that the neuron could in theory transmit. The bit rate of this neuron, at least in the given window would therefore be 15 bits per 30ms, or more conveniently, 500 bits per second. That number is just what the neuron is transmitting to the flight muscle in its termination zone. In fact each of the neuron’s 100 or more input neurites may be said to be integrating information at a roughly similar bit rate, we just cannot map a real world function to them as easily at that level of detail.
In principle the requirement that spikes fall within windows is artificial. Bats can reportedly detect Doppler shifts in echos down to the nanosecond level using spikes. Spikes with that kind of temporal precision, particularly when considered in the context of a network, could have much higher corresponding bit rates. Brains, or in particular neurons, use spikes in entirely different ways depending on what kind job the neuron does and what kind of network it is in. Neurons controlling the eye muscles need to fire quickly and often to maintain eye position. On the other hand, neurons that control the movements of food through the gut need to spike rhythmically on cue and be able to put out a constant volley of spikes at each peak in the rhythm to drive the gut muscle properly — these neurons might be likened to performing the “wave” much like you would do at a football game.
Next page: Are neuristors the future of prosthesis and robotics?
Posts
No comments:
Post a Comment