In the last blog we looked at the phenomenon of “jagged intelligence” — the paradox that AIs are simultaneously superhuman and stupid in adjacent skills. We identified a lack of continual, fast learning and embodiment as possible causes of this weirdness, and noted that embodiment is also a “learning problem”, because current ML models can’t learn… Read More »An unexpected success in zero-shot, few-shot and streaming RL
Introduction Will highly intelligent machines inevitably think like us? Our experiences with AI that is simultaneously incredibly powerful and incredibly flawed are evidence that not all forms of intelligence are the same. Until recently, all human-level intelligence was human. Now, AI seems to threaten our monopoly. But truly human-level and human-like artificial general… Read More »Not like us: Why AI feels alien, and how to fix it
Authors: Hoang Long Dang and David RawlinsonUPDATE: Full HRM-Agent paper now available. When the Hierarchical Reasoning Model was released, we were very excited about its combination of small size and impressive reasoning abilities. These qualities seem to be enhanced by recurrent processing, but with a surprisingly simple gradient update based on the converged recurrent… Read More »HRM-Agent: Using the Hierarchical Reasoning Model in Reinforcement Learning
Since the arrival of Chat GPT, I’ve felt that progress in AI has been largely incremental. Of course, even Chat GPT was/is mostly just a scaled up Transformer model, which was introduced by Vaswani et al in 2017. The belief that AI is advancing relatively slowly is a radical and unpopular opinion… Read More »Why I’m excited about the Hierarchical Reasoning Model
Check out this fascinating research meeting with Numenta and guest speaker Max Bennett. Max describes this thalamocortical microcircuit theory. More detail can be found in his paper, published here: https://www.frontiersin.org/articles/10.3389/fncir.2020.00040/full#h1
We’ve just undertaken a review and refresh of our research roadmap! The topics and approach we will take in the coming year are all laid out in a new page: Research Roadmap Our primary topics for the coming year include: Continual Few-Shot Learning (CFSL) via our Episodic memory system Using… Read More »Research Roadmap: 2020-2021
Research Roadmap The diagram above shows our research roadmap for the coming year (July 2020-June 2021). You can compare to last year’s diagram (below) to see our progress. Red filling indicates completion; yellow indicates areas we expect to make progress this year. The map has two principal stages – improved… Read More »Research Roadmap (Aug 2020)
The Whole Brain Architecture Initiative (WBAI) aims to foster research into architectural approaches to general intelligence. They have held a series of events – the Hackathons – that encourage and support the development of new models which provide working solutions to questions about the possible architecture of general intelligence in… Read More »5th WBAI Hackathon
Artificial neural networks (ANNs) – are conceptually simple; the combination of inputs and weights in a classical ANN can be represented as a single matrix product operation followed by an elementwise nonlinearity. However, as the number of learned parameters increases, it becomes very difficult to train these networks effectively. Most… Read More »Biologically-plausible learning rules for artificial neural networks
One of our key projects is a memory system that can learn to associate distant cause & effect while only using local, immediate & unsupervised credit assignment. Our approach is called RSM – Recurrent Sparse Memory. We recently uploaded a preprint describing RSM. This is the first of several blog… Read More »Learning partially-observable higher-order sequences using local and immediate credit assignment