Executive Control is core to what most people recognise as true intelligence. For example, the ability to attend to relevant cues and maintain task dependent information whilst ignoring distracting details and taking appropriate actions. Working Memory (WM) is a core component of Executive Control. “Working memory is a short-term repository… Read More »Towards Biologically Inspired Executive Control
Check out this fascinating research meeting with Numenta and guest speaker Max Bennett. Max describes this thalamocortical microcircuit theory. More detail can be found in his paper, published here: https://www.frontiersin.org/articles/10.3389/fncir.2020.00040/full#h1
We’re very happy to report that we recently published a preprint on AHA, an ‘Artificial Hippocampal Algorithm’ for Episodic Machine Learning. It’s the culmination of a multi-year research project and is a starting point for the next wave of developments. This article describes the motivation for developing AHA and a… Read More »AHA! an ‘Artificial Hippocampal Algorithm’ for Episodic Machine Learning
The standard definitions of Episodic and Semantic Memory hide some important subtleties that impact the development of Machine Learning algorithms for AI (particularly those building Episodic Learning capability). This article explores this issue and provides a context for upcoming articles work to take us a step closer to ‘animal-like’ machine learning
Artificial neural networks (ANNs) – are conceptually simple; the combination of inputs and weights in a classical ANN can be represented as a single matrix product operation followed by an elementwise nonlinearity. However, as the number of learned parameters increases, it becomes very difficult to train these networks effectively. Most… Read More »Biologically-plausible learning rules for artificial neural networks
Originally published in March 2019 in an electronic journal in Japanese Introduction The purpose of this essay is to survey the relationship between decision making and large-scale cerebral networks with regard to conscious access, a purported neural correlate of consciousness, and to provide clues for computational modelling and general understanding… Read More »Cerebral networks for conscious access and decision making
We are exploring the nature of equivariance, a concept that is now closely associated with the capsules network architecture (see key papers Sabour et al, and Hinton et al). Machine learning representations that capture equivariance must learn the way that patterns in the input vary together, in addition to statistical clusters in… Read More »Understanding Equivariance
Competitive learning is a branch of unsupervised learning that was popular a long, long time ago in the 1990s. Older readers may remember – the days before widespread use of GSM mobile phones and before Google won the search engine wars! Although competitive learning is now rarely used, it is worth… Read More »Convolutional Competitive Learning vs. Sparse Autoencoders (1/2)
ML Today Today’s Machine Learning has demonstrated unprecedented performance in what seems like every application thrown at it. Almost all the success has been based on advanced memory systems that can learn to recognise an input based on a large number of training examples. This is the equivalent to memory… Read More »The case for Episodic Memory in Machine Learning
Reproduced left to right from [8,10,1] Within a 5 day span in October, 4 papers came out that take a significantly different approach to AI hierarchical networks. They are all inspired by biological principles to varying degrees. It’s exciting to see different ways of thinking. Particularly at a time… Read More »New approaches to Deep Networks – Capsules (Hinton), HTM (Numenta), Sparsey (Neurithmic Systems) and RCN (Vicarious)