Check out this fascinating research meeting with Numenta and guest speaker Max Bennett. Max describes this thalamocortical microcircuit theory. More detail can be found in his paper, published here: https://www.frontiersin.org/articles/10.3389/fncir.2020.00040/full#h1
We’ve just undertaken a review and refresh of our research roadmap! The topics and approach we will take in the coming year are all laid out in a new page: Research Roadmap Our primary topics for the coming year include: Continual Few-Shot Learning (CFSL) via our Episodic memory system Using… Read More »Research Roadmap: 2020-2021
The Whole Brain Architecture Initiative (WBAI) aims to foster research into architectural approaches to general intelligence. They have held a series of events – the Hackathons – that encourage and support the development of new models which provide working solutions to questions about the possible architecture of general intelligence in… Read More »5th WBAI Hackathon
Artificial neural networks (ANNs) – are conceptually simple; the combination of inputs and weights in a classical ANN can be represented as a single matrix product operation followed by an elementwise nonlinearity. However, as the number of learned parameters increases, it becomes very difficult to train these networks effectively. Most… Read More »Biologically-plausible learning rules for artificial neural networks
One of our key projects is a memory system that can learn to associate distant cause & effect while only using local, immediate & unsupervised credit assignment. Our approach is called RSM – Recurrent Sparse Memory. We recently uploaded a preprint describing RSM. This is the first of several blog… Read More »Learning partially-observable higher-order sequences using local and immediate credit assignment
We’ve uploaded a new paper to arXiv presenting our algorithm for biologically-plausible learning of distant cause & effect using only local and immediate credit assignment. This is a big step for us – it ticks almost all our requirements for a general purpose representation. The training regime is unsupervised &… Read More »Learning distant cause and effect using only local and immediate credit assignment
As AI/ML researchers, we have obviously pondered the risks of AI. We even wrote about it. But what might surprise you is the risks that keep AI/ML folk awake at night aren’t the ones you hear about in the media. We’re not worried about runaway “paperclip maximizers” or “skynet”-style machine… Read More »AI is already harming us – but not the way you think
We recently talked about Capsules networks and equivariances. NB: If you’re not familiar with Capsules networks, read this first. Our primary objective with Capsules networks is to exploit their enhanced generalization abilities. However, what we’ve found instead raises new questions about how generalization can be measured and whether Capsules networks are… Read More »Predictive Capsules Networks – Research update
We are exploring the nature of equivariance, a concept that is now closely associated with the capsules network architecture (see key papers Sabour et al, and Hinton et al). Machine learning representations that capture equivariance must learn the way that patterns in the input vary together, in addition to statistical clusters in… Read More »Understanding Equivariance
This is the second part of our comparison between convolutional competitive learning and convolutional or fully-connected sparse autoencoders. To understand our motivation for this comparison, have a look at the first article. We decided to compare two specific algorithms that tick most of the features we require: K-Sparse autoencoders, and… Read More »Convolutional Competitive Learning vs. Sparse Autoencoders (2/2)