Scholars in many different disciplines have hypothesized that the human brain is merely a very complex computer. Regardless of the validity of this statement, the human brain is certainly not the type of computer with which most people are familiar. Computers which are found in our homes and offices are serial -- they only do one computation at a time, but do sequences of computations very quickly.
To perform even the simplest functions, a serial computer requires hundreds or thousands of computations. Depending on the speed of the computer's processor, this process can be very fast (billions of computations per second). Even though serial computers are very fast, they have their limitations. For instance, imagine a very complex activity such as driving a car through a large city during rush hour. We could probably program a computer to do this, but the processing needed would have to be at least as fast as today's fastest computers.
The way a brain holds and processes information is vastly different from a serial computer. Our neurons do not transmit information very fast. Even for a reflex that only involves three neurons, it may take several tenths of a second. However, we are somehow able to process the information that it takes to drive a car using our relatively slow neurons. This is because our nervous system does not process serially. Instead, the human brain is an example of a massively-parallel system -- rather than executing computations one-by-one in series, the human brain makes many, many computations simultaneously when performing a neural process.
Our brains are part of a nervous system. The basic unit of transmission in the nervous system is a cell called a neuron. Neurons are considered to have 3 main parts: an axon, a dendrite, and a soma. The soma is the part of the neuron that contains the nucleus, organelles and cytoplasm. The axon is the long part of the neuron over which the signal (or impulse) is propagated. Dendrites are small, branching projections off of the soma which receive impulses. (insert Rebecca's neuron drawing here)
When a neuron passes a signal to another neuron, the signal is passed over a small space called a synapse. The axon terminals at the end of a presynaptic neuron (neuron sending the signal) receive the electrical signal from the axon. The signal stimulates the release of a chemical (a neurotransmitter) which diffuses across the synapse to the dendrites and soma of the postsynaptic neuron (neuron receiving the signal). Depending on the proteins in the membrane of the postsynaptic neuron that the neurotransmitter binds to, the neurotransmitter can either stimulate or inhibit the neuron.
Neurons always maintain a potential (an electrical gradient of ions) across their membranes. When a postsynaptic neuron is stimulated or inhibited, the potential of the neuron's membrane changes. If the neuron is stimulated enough, the potential of the membrane near the axon can reach a threshold. Once the threshold of the membrane potential is reached, an impulse (action potential) is transmitted along the axon to the next neuron.
Most of the time, a presynaptic neuron does not only affect one postsynaptic neuron. The axon terminals usually attach to many other neurons. A single neuron will only release one neurotransmitter. However, because the proteins in the membrane of the postsynaptic neuron determines whether they will stimulated or inhibited, a single presynaptic neuron can inhibit one cell and stimulate another. Likewise, a postsynaptic neuron is usually affected by more than one presynaptic neuron. If different presynaptic neurons release different neurotranmitters, the combined affects of the stimuli and inhibition may cancel each other out. In addition, usually, one presynaptic axon terminal cannot stimulate a postsynaptic neuron enough to reach threshold by itself. If many of them stimulate the postsynaptic neuron at the same time without too many inhibiting the postsynaptic neuron, then threshold may be reached.
As neurons in the nervous system interconnect, they form large clusters and ultimately form the brain. These large clusters inspired the development of neural networks. These networks model the connectivity of the brain with one major assumption. In neural networks, it is assumed that the interactions between neurons in the brain can be represented mathematically. The connections are like neurons. The connections have different strengths, in terms of numerical values. Depending on the connection "strength," they will transmit stronger or weaker signals to the nodes. The connection strength might work in the same way that different presynaptic neurons might release different amounts of neurotransmitter into a synapse. Perhaps the strength might represent how many synapses a presynaptic neuron has with a postsynaptic neuron. A node is like the synapse and the membrane of the postsynaptic neuron. Depending on the amount a node is stimulated or inhibited will affect the way it signals the next layer of nodes along another connection. This resembles the way the membrane of a postsynaptic neuron is stimulated or inhibited. Depending on how quickly a membrane is stimulated and where it is stimulated affects whether an action potential is produced in that neuron.
While the overall structure of a neural network resembles the physiology of the brain, further similarities are limited. For instance, physiologically, there is no explanation of a mechanism for learning. Connections from neuron to neuron can change in the brain. However, it is not known how this relates to learning. Whatever the mechanism for learning is, it is most likely not back propagation. Back propagation is the method by which neural networks change their connection strengths and it will be explained in the section Training a Neural Network. More importantly, the most complex neural networks may contain a few thousand or so connections. The human brain contains upwards of a billion neurons. Due to its relatively small size, at best a neural network can learn to distinguish between a couple of things. We are able to differentiate vast amounts of stimuli from our senses, not to mention all of the processes in our brains that we are not consciously aware of. Because our nervous system is so vast, we have no idea how such large networks may work. Neural networks may never be capable of duplicating a system the size of our brain. Still, it is impressive enough that neural networks mimic at least some minor brain processes.
In the above
diagram, the dots represent nodes and the lines between them represent the connections
between nodes. A given input (usually some numerical value) will stimulate the
input layer. Input at a node could be anything that can be represented mathematically.
For instance, this information could be a pixel of a picture, a letter of a
word, etc. Each input node can receive a different value depending on the nature
of the input. At the input node, this value will be transmitted to the nodes
at the hidden layer along connections. It can either be transmitted to all of
the hidden layer nodes, or just a few, depending on how the individual network
is set up. The "strength" or "weight" of the connection can also alter the value
from the input node. Some connections are weighted differently than others because
sometimes the information represented in that connection is more or less important.
A given hidden layer node will receive values from all of the input nodes to which it is connected. At this point, the different values of the connections can stimulate or inhibit the node. The values of the connections will undergo a calculation which produces a new value. This value in turn will be transmitted along new connections to the output layer in the same way that the input was transmitted to the hidden layer.
A neural network does not start out capable of identifying a picture or reading a word, but miraculously, a neural network can learn to do these activities. A serial system of computing would not be able to perform these activities as well, especially under a variety of conditions. For instance a neural network can identify an object even if the appearance of that object is altered. Any alteration in the appearance of an object would most likely confuse a serial computer, or at least the serial computer would get it wrong more often.
A neural network can have any number of nodes, any number of layers of nodes (usually more than two), and almost any number of connections between nodes. Once a particular network is set up with these parameters and some functions to be performed at the nodes, a network can be trained.
For the most part, a network is trained by changing the weights of the connections between nodes. These weights can be randomly chosen or individually chosen. Usually, a computer program randomly generates values for connection weights. Then, the network is given an input, and it is allowed to process the information through its nodes to produce an output. Perhaps the input will be a letter, and you want the network to tell you whether it is the letter "P" or the letter "R". Initially, the output will be wrong quite frequently. However, the weights of the connections can then be changed, and the network can try again. Over the course of hundreds of attempts and subtle changes in the weights of the connections, the network can learn to identify the letter very accurately.
A common type of training in neural
networks is
back propagation. With back propagation, a program is set up to automatically
change the weights of connections whenever the output is wrong, and strengthens
connections whenever the output is correct. When the connections are changed,
the program changes the values of certain connections backward from the output
nodes to the hidden layer to the input nodes. After further trials, the values
of different connections may be changed. This continues until the output is
usually correct within a certain degree of freedom. The network never produces
outputs that are correct 100% of the time. Once trained, the neural network
is a probabilistic system which is correct most of the time.
Once trained, a network can be exposed to an input that it has never "seen" before, and it will usually still be right. For instance, a network trained to recognize the letter "R" can be shown an "R" that is written differently, and it will probably still identify it as "R". This is because the values in the network are probabilistic. Even when "R" is written differently, the picture will recognize that it is similar to that of the original "R." The network will recognize that it is probably an "R" and will answer accordingly.
Because the physiology of the nervous system has yet to explain most mechanisms for learning and processing information, even physiologists recognize that other areas of inquiry are important in understanding these processes. Physiology can explain much of the sensation of input, and the filtering of input into the brain. However once this information enters the brain, physiologists are at a loss to explain what happens until some output occurs.
(insert diagram here)Despite this lack of physiological information, neural networks are valuable models for psychological studies of brain activities. Simply for the fact that they learn even if the mechanism of learning is different than that of a brain. They have been useful in understanding what we learn when we say we learn something. In a neural network, the network does not learn every detail about something. Rather, the network is able to recognize recurrent patterns between similar things. We may be able to learn more details than a neural network, but our brain is also more complicated than a neural network. Still, it is likely that when we actually identify something, we recognize broad patterns rather than details.
Neural networks are also useful tools in studying learning deficiencies. When someone has dyslexia, suffers a stroke, or perhaps suffers from head trauma, the usual processes of learning and comprehension are disrupted. For instance, when someone has a stroke, blood is cut off to part of the brain and that part of the brain no longer functions. In many neural networks, if they are trained to perform a task and some of the connections are taken out (mimicking a stroke), the resulting mistakes made by the neural network are often similar to that of a stroke victim. Cases of dyslexia, and other disorders have also been modeled with neural networks. As neural networks are developed that better model brain activity, they will continue to be useful tools in studying neuropsychology.
Also, neural networks are important in cognitive science. Perhaps a network can be explained as a complex reflex where a behavior simply results from an input. However, it seems likely that the randomness of the network may show that behavior cannot be reduced to simply reflexes of brain physiology. Rather, the patterns that emerge in these networks may represent cognition. Behavior may only be explainable as a result of emergent patterns in neural networks and in brains. While the connections and neurons are important and necessary for the emergent properties. The emergent patterns are entirely new entities which determine output and behavior.
Bernstein, Douglas A., Alison Clark-Stewart, Edward J. Roy, and Christopher D. Wickens. Psychology. New York: Houghton Mifflin Company, 1997.
McClelland, James L., David E. Rumelhart. Parallel Distributed Processing. Vol. 1 & 2. Cambridge, MA: MIT Press, 1986.
Randall, David, Warren Burggren, Kathleen French. Eckert Animal Physiology: Mechanisms and Adaptations. New York: W.H. Freeman and Company, 1997.