Connectionism

Single-Layer_Neural_Network-Vector-Blank

It is the name for the computer modelling approach to information processing based on the design or architecture of the brain. Connectionist computer models are based on how computation occurs in neural networks where neutrons represent the basic information processing structures in the brain.

All connectionist models consist of four parts:

  • units: they are what neutrons are to the biological neural network, the basic information processing structures. Most connectionist models are computer simulations run on digital computers. Units in such models are virtual objects and are usually represented by circles. A unit receives input, it computes an output signal and then it sends the output to other units. This is called activation value. The purpose of the unit is to compute an output activation.
  • connections: connectionist models are organised in layers of units, usually three (3). A network however, is not simply an interconnected group of objects but an interconnected group of objects that exchange information with one another. Network connections are conduits. The conduits through which information flows from one member of the network to the next are called synapses or connections and are represented with lines. (in biology synapses are the gaps between neutrons, the fluid-filled space through which chemical messengers -neurotransmitters- leave one neutron and enter another)
  • activations: activation value in connectionist models are analogous to a neuron’s firing rate or how actively it is sending signals to other neurons. There is a big variability between the least active and the most active neutrons expressed in a scale fro 0 to 1
  • connection weights: The input activations to a unit are not the only values it needs to know before it can compute its output activation. It also needs to know how strongly or weakly an input activation should affect its behaviour. The strength or weakness of a connection is measured by a connection weight. They range between -1 to 1. Inhibitory connection reduce a neuron’s level of activity; excitatory connections increase it.

Yet, the behaviour of a unit is never determined by an input signal sent via a single connection, however strong or weak that connection might be. It depends on its combined input. That is the sum of each input activation multiplied by its connection weight. The output activation of a unit represents how active it is, not the strength of its signal.

Connectionist networks consist of units and connections between units and have some very interesting features like emergence of behaviour. This does not reduce to any particular unit (liquidity in water). Graceful Degradation and Pattern Completion are two ways in which activations are spread through a network. They are not classical computers, their behaviour does not arise from an algorithm, they learn to behave the way they do.

 

References

Robert Stufflebeam, 2006. Connectionism: An Introduction (pages 1-3), in CCSI (Consortium on Cognitive Science Instruction) supported by the Mind Project, full article available here

Image available here

Learning Rules

hebbs rule

Hebb’s rule: is a neuroscience theory where an increase in synaptic efficacy arises from the presynaptic cell’s repeated and persistent stimulation of the postsynaptic cel (…) Hebbian theory concerns how neurons might connect themselves to become engrams (=means by which memories are stored thus biophysical/biochemical changes in the brain in response to external stimuli) (…) The theory attempts to explain associative or Hebbian learning, in which simultaneous activation of cells leads to pronounced increases in synaptic strength between those cells, and provides a biological basis for errorless learning methods for education and memory rehabilitation. In the study of neural networks in cognitive function, it is often regarded as the neuronal basis of unsupervised learning.

512px-Kernel_Machine

Back-propagationis a method used in artificial neural networks to calculate the error contribution of each neuron after a batch of data (in image recognition, multiple images) is processed  [=computing systems inspired by the biological neural networks that constitute animal brains, these systems learn to do tasks by considering examples](…) Backpropagation is sometimes referred to as deep learninga term used to describe neural networks with more than one hidden layer (layers not dedicated to input or output)

Boltzmannexamplev1

Boltzmann machine: is a type of stochastic recurrent neural network [a stochastic or random process is a mathematical object usually defined as a collection of random variables] (…) They were one of the first neural networks capable of learning internal representations, and are able to represent and (given sufficient time) solve difficult combinatoric problems (…) Boltzmann machines with unconstrained connectivity have not proven useful for practical problems in machine learning or inference, but if the connectivity is properly constrained, the learning can be made efficient enough to be useful for practical problems

 

References & Images

 

Connectomics

creativebrain_connectivity

Connectomics uses advanced brain imaging techniques to identify and map the intricate web of white matter (communication lines) that link gray matter (neural brain volume). Mapping such networks occurs at the level of synaptic connections. This research began in the 70’s but has recently gained interest thanks to technical and computational advances that automate the collection of electron-microscopy data and offer the possibility of mapping even large mammalian brains. “Connectome” was coined in analogy with the “genome”—the entirety of an organism’s hereditary information—studied by biologists. To imagine how the story of the connectome will unfold over the next few decades, it’s helpful to recall the history of the genome. Connectomics is more challenging than genomics; the structure of the brain is extraordinarily complex. With an electron microscope, the branches of neurons can be seen clearly, even when they are tightly packed together in the brain.

People with high creative capacity have more connections between their left and their right hemispheres of their cerebral cortex.

 

References

  • Highly Creative People Have Well-Connected Brain Hemispheres, full article and image available here
  • The big data challenges of connectomics, available here
  • Connectomics: Tracing the Wires of the Brains, available here
  • video

Seven Sins of Memory (2003)

seven-sins

Our self is based on memories of past experiences while the retrieval, recollection and reconstruction of the past is reciprocally influenced by the self. Memory’s imperfection is classified in this book in seven sins (intended here as in transgressions fatal to spiritual progress/ ways in which the normal, everyday operations of our mind may occasionally produce suboptimal or flawed memory experiences):

  • Forgetting: 01 transience/ 02 absent-mindedness/ 03 blocking
  • Distortion: 01 misattribution/ 02 suggestibility/ 03 bias
  • Intrusive memories: 01 persistence

seven-sins

  • Image 01 available here
  • Image o2 available here

See also; Joseph LeDoux, 2002, Synptic Self: In the absence of learning and memory processes the self would be an impoverished expression of our genetic constitution

Internet as a memory source

internet-memory

An interesting research is conducted by B. Sparrow, J. Liu and D.M. Wegner in 2011 and presented in  ‘Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips’. Their experiments focus on whether online access to search engines has become a primary transactive* memory source. They have conducted four experiments proving that:

01. when we are faced with a gap in our knowledge, we are primed to turn to the computer to rectify the situation (…) 02. when people don’t believe that they will need information for a later exam, they do not recall it at the same rate as when they do believe they will need it (…) 03. believing that one won’t have access to the information in the future enhances memory for the information itself, whereas believing the information was saved externally enhances memory for the fact that the information could be accessed at lest in general (…) 04. people don’t remember “where” when they know “what”but do remember where to find the information when they don’t recall it (…) people expect information to remain continuously available.

The results of the study suggest that processes of human memory are adapting to the advent of new computing and communication technology. In ” The Internet as a Memory Source: How the Brain is Keeping Up” the author uses this information to focus to the neurological/biological implications of this development. Is the existence of internet and its use as an external memory source changing the way our brains form synapses? For it seems that we no longer store information in the long term memory but rather its location.

Sparrow et al. use an interesting phrase: “we are becoming”, they say, “symbiotic with our computer tools, growing into interconnected systems”. It’s almost as if remembering through these systems is not any different that sharing memories with other individuals, plus through internet we have access to a vast range of information at any point.

*TRANSACTIVE memory: a combination of memory stores held directly by individuals and the memory stores they can access because they know someone who knows that information.

Image available here.

Neuroplasticity

NeuroplasticityThe brain’s natural ability to form new connections in order to make up for  for injury or changes in the environment. The ability of the brain to reorganize pathways between neurons as a result of new experiences. (definition extracted from Stanford Webpage)

neuroplasticity-02

Image showing the neural connections in the brain of a newborn, a 3 month old, a 15 month old and a two year old child. 

The hippocampus is at the front of the brain and was examined in Magnetic Resonance Imaging (MRI) scans on 16 London cabbies. The tests found the only area of the taxi drivers’ brains that was different from the 50 other “control” subjects was the left and right hippocampus (…) The posterior hippocampus was also more developed in taxi drivers who had been in the career for 40 years than in those who had been driving for a shorter period (…) “This is very interesting because we now see there can be structural changes in healthy human brains.”

BBC News World edition, Taxi drivers’ brains ‘grow’ on the job, Tuesday, 14 March, 2000, full article available here 

A cab driver’s hippocampus — the part of the brain that holds spatial representation capacity — is measurably larger than that of a bus driver. By driving the same route every day, the bus drivers don’t need to exercise this part of the brain as much. The cabbies, on the other hand, rely on it constantly for navigation. If you were to restrict certain senses — like vision, for instance — your brain would make a similar adaptation. This great survival machine will rewire itself, opening neuro pathways to heighten the power of other senses to keep you from falling off a cliff or get eaten by a tiger.

Daniel Honan, Neuroplasticity: You Can Teach An Old Brain New Tricks, Big Think, full article available here

Neuroplasticity is what allows us to take our experiences, then learn from them and form new memories. Huge changes are occurring in the brain during these early stages of cognitive development, but the truth is that our neural networks continue to build on each other until the day we die (…) The more often neural pathways fire, the stronger the connections will become. This is called long-term potentiation, and it is the basis of all learning and memory formation (…) The big implication here is that if our brain changes itself based on our experiences, then by changing our experiences we can actively reshape our brains

See also Synaptic Pruning

Educational Research, Cognitive Science & Neuroscience

ercp-n

Notes from MIT’s Online Education Policy Initiative Report, Pages 6-10

ER: Constructivism: Dewey, experiential learning, Piaget, Vygotsky, Montessori, inquiry and discovery. Active learning, teaching laboratories, Amos Eaton (1824), active instruction, Mazur, peer learning, all-hands-on courses, mini-lectures, simulations experiments. Online counterparts are flipped classroom. Project -based learning, video disks, personal computers and calculators. Papert’s Constructionism, a refinement of constructivism, development of Lego Mindstorms, robot design, prototyping technologies. Problem-based learning, imprecisely defined problems, self-directed learning peer learning, teamwork, internships, work-study programs, blurred boundaries between college and workplace. Student-centered education, reflection, discussion, interdisciplinary thinking, self-paced learning, Bloom, students in small cohorts. Online counterparts are Peer2Peer University where peer is the primary instructor.

CS: level of the brain, Ebbinghaus, how memories form and persistmind wandering, task-unrelated thoughts, make students curious, retrieval practice, engaging repeatedly in recall activities
is called interpolated testing, block of practice right after students have learned a topic, contrast between storage strength and retrieval strength, concept of desirable
difficulties,generation effect, generation of answers can help learning even if they are wrong, and feedback is effective even if it is corrective. Cognitive load theory, “compression” of new information, novices should be given worked examples
rather than open-ended problems. Impact of context, the context of the learning reflect the context in which that information will likely be used.

N: level of the neurons, initial encoding, integration of memories, consolidation, synaptic and system levels, sleep, blocked learning may be associated with saturation at the synapse during a process known as long-term potentiation, cognitive load has been shown to be measurable using pupillary dilation, activation of sensorimotor brain regions would enhance understanding of torque and angular momentum, MRI shows more active training
methods correlated not only with better test performance but also with greater stimulation of the predicted brain regions

Image available here