Themes - Cerenaut

Research Themes

Biological Plausibility

To force a separation between our work and the dominant trajectory of Machine Learning we self-impose "Biological Plausibility" constraints. That doesn't mean we need to do everything exactly as it is done in "Wet Brains": Rather, we aim to exclude the most implausible features of ML and develop biologically-plausible alternatives with similar computational qualities.

Mental Simulation

Rather than focus on building control and planning systems explicitly, later moving towards more abstract reasoning systems, we take the approach of building a sophisticated simulation of the world first, and then exploiting this simulation for planning and goal-directed behaviour. Our strategy will be to learn to build predictive models, extend predictions many steps into the future in Generative mode, and then learn to control these predictions resulting in the ability to evaluate action choices over many time steps. This is our definition of mental simulation.

Sparse Coding

In practice, we have found that sparse representations have many desirable properties including combinatoric representational capacity and the ability to represent novel combinations of concepts without additional learning.

Disentangled latent variable learning

Capsules networks learn to describe observations using only a few parameters of a subset of cells. This bottleneck forces Capsule networks to disentangle the latent variables and entities in the world and describe them efficiently. These representations are more powerful and flexible than conventional neural models.

Attention

Attetional filtering has recently delivered state of the art results in a number of tasks especially natural language modelling. However, the dominant approach to attentional filtering still relies on deep backpropagation to learn attentional strategies. To avoid biologically implausible time-travel we will use the Bellman equation (a key part of discounted reinforcement learning) to associate delayed rewards with their causes.

Embodiment

Rodney Brooks' early work on the subsumption archhitecture and the importance of embodiment in robotics still resonates with us today. Many of the capabilities and features we intend to demonstrate only fit into an agent-based paradigm, which means we must develop an environment for it to operate in. Fortunately, these days there are a number of excellent (mostly RL-based) simulated environments to play with, such as the OpenAI Gym and the AnimalAI Olympics.

Credit Assignment

One of the most biologically-implausible aspects of conventional ML is the use of deep-backpropagation as the core mechanism for credit assignment (i.e. learning network parameters). We then face the considerable challenge of building entirely different learning systems that are still able to perform as effectively as those trained with deep-BP.

Continual Learning

One of the key constraints we apply is that our methods must learn continually even in nonstationary environments (i.e. the statistics of the data may change at any time, and learning effectiveness must not be impaired by this). Many ML methods fail to function well in nonstationary environments and therefore cannot learn continually. The criterion of "lifelong" learning is also often used, meaning that learning cannot stop.

Sparse Rewards

We assume that external feedback can only take the form of sparse (rare) feedback, whether in the form of sparse labels or rewards. In addition we require that our methods function with only rewards (the quality of a state or action) rather than the ideal state or action. This matches the conditions in which people learn.

Unsupervised or Self-Supervised Learning

Given only sparse global rewards, the vast majority of learning must be either unsupervised or self-supervised. One of the most promising learning rules may be Predictive Coding - learning to predict the next input, whatever it is. The rule needs only local information and naturally learns the dynamics of observed sensor input.

Semi-Symbolic Representations

One of the key inspirations for our work was Jeff Hawkins' elimination of the symbol-grounding problem by redefining it as an incremental process of accumulating invariances. Later, researchers generalized this to include notions of equivariance (Hinton) and disentanglement (Y. Bengio). Whichever qualities are produced in the process of abstraction, the transformation must be incremental (and reversible) not discrete.

Hierarchical Planning

Having established that representations must be symbolic to varying degrees, and building on the effective functioning of deep neural networks, it is natural that representations should be hierarchical - that is, concepts at different levels in a graph should have varying scope and specificity. For efficiency, the same representation and hierarchy should be used for planning - meaning that plans would not exist as a series of steps but as a set of features in the graph that are incrementally unfolded an developed to match observed reality. This is the process by which we hope to translate mental simulations to effective action in simulated environments.