Skip to content

Sparse Distributed Representations

New approaches to Deep Networks – Capsules (Hinton), HTM (Numenta), Sparsey (Neurithmic Systems) and RCN (Vicarious)

  Reproduced left to right from [8,10,1] Within a 5 day span in October, 4 papers came out that take a significantly different approach to AI hierarchical networks. They are all inspired by biological principles to varying degrees. It’s exciting to see different ways of thinking. Particularly at a time… Read More »New approaches to Deep Networks – Capsules (Hinton), HTM (Numenta), Sparsey (Neurithmic Systems) and RCN (Vicarious)

SDR-RL (Sparse, Distributed Representation with Reinforcement Learning)

Erik Laukien is back with a demo of Sparse, Distributed Representation with Reinforcement Learning. This topic is of intense interest to us, although the problem is quite a simple one. SDRs are a natural fit with Reinforcement Learning because bits jointly represent a state. If you associate each bit-pattern with… Read More »SDR-RL (Sparse, Distributed Representation with Reinforcement Learning)

“Quantum computing” via Sparse distributed coding?

An interesting article by Gerard Rinkus comparing the qualities of sparse distributed representation and quantum computing. In effect, he argues that because distributed representations can simultaneously represent multiple states, you get the same effect as a quantum superposition. The article was originally titled “sparse distributed coding via quantum computing” but… Read More »“Quantum computing” via Sparse distributed coding?

Sparse Distributed Representations (SDRs)

TL;DR: An SDR is a Sparse Distributed Representation, described below SDRs are biologically plausible data structures SDRs have powerful properties SDRs have received a lot of attention recently There are a few really great new resources on the topic: Presentation by Subutai Ahmad of Numenta Older introductory presentation by Jeff Hawkins Excellent… Read More »Sparse Distributed Representations (SDRs)

Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function

This post is about a fantastic new paper by Michael R. Ferrier, titled:   Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function   The paper was posted to the NUPIC mailing list and can be found via:   http://numenta.org/community-content.html   The paper itself… Read More »Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function