Capsules: Alternative to Pooling

Recently, Geoffrey Hinton and his colleagues made the article about capsules available. He has been known to heavily criticize the use of pooling and back propagation.

“A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or object part.” The nodes of inputs and outputs are vectors, instead of scalars as in neural networks. A cheat sheet comparing the traditional neurons and capsules is as follow:

v2-8cfd484b3283010c6cf1be594d1588b7_r

Based on the capsule, the authors suggested a new type of layer called CapsNet.

Huadong Liao implemented CapsNet with TensorFlow according to the paper. (Refer to his repository.)

  • Sara Sabour, Nicholas Frosst, Geoffrey E Hinton, “Dynamic Routing Between Capsules,” arXiv:1710.09829 (2017). [arXiv]
  • “浅析 Hinton 最近提出的 Capsule 计划” (2017). [Zhihu] (in Chinese)
  • “如何看待Hinton的论文《Dynamic Routing Between Capsules》?” (2017) [Zhihu] (in Chinese)
  • Github: naturomics/CapsNet-Tensorflow [Github]
  • Nick Bourdakos, “Capsule Networks Are Shaking up AI — Here’s How to Use Them,” Medium (2017). [Medium]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a free website or blog at WordPress.com.

Up ↑

%d bloggers like this: