From c24712c2d1c66dfd27df4bb8b688c92773ef8d25 Mon Sep 17 00:00:00 2001 From: Fernando Espinosa Date: Sat, 4 Nov 2017 23:04:31 -0700 Subject: [PATCH 1/2] Update index.html --- posts/2015-09-NN-Types-FP/index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/posts/2015-09-NN-Types-FP/index.html b/posts/2015-09-NN-Types-FP/index.html index 60a21cc..8160d9a 100644 --- a/posts/2015-09-NN-Types-FP/index.html +++ b/posts/2015-09-NN-Types-FP/index.html @@ -127,7 +127,7 @@

Representations are Types

A layer \(f_1\) followed by a layer \(f_2\). The output representation of \(f_1\) is the input of \(f_2\).

In the case of very simple neural network architectures, where there’s just a linear sequence of layers, this isn’t very interesting. The representation of one layer’s output needs to match the representation of the next layer’s input – so what? It’s a trivial and boring requirement.

-

But many neural networks have more complicated architectures where this becomes a more interesting constraint. For a very simple example, let’s imagine a neural network with multiple similar kinds of inputs, which performs multiple, related tasks. Perhaps it takes in RGB images and also grayscale images. Maybe it’s looking at pictures of people, and trying to predict age and gender. Because the similarities between the kinds of inputs and between the kinds of tasks, it can be helpful to do all of this in one model, so that training data helps them all. The result is multiple input layers mapping into one representation, and multiple outputs mapping from the same representation.

+

But many neural networks have more complicated architectures where this becomes a more interesting constraint. For a very simple example, let’s imagine a neural network with multiple similar kinds of inputs, which performs multiple, related tasks. Perhaps it takes in RGB images and also grayscale images. Maybe it’s looking at pictures of people, and trying to predict age and gender. Because of the similarities between the kinds of inputs and between the kinds of tasks, it can be helpful to do all of this in one model, so that training data helps them all. The result is multiple input layers mapping into one representation, and multiple outputs mapping from the same representation.

From f77cfc2d1c5be69e0a7c45d95cc819072abc103d Mon Sep 17 00:00:00 2001 From: Fernando Espinosa Date: Sun, 5 Nov 2017 15:39:01 -0800 Subject: [PATCH 2/2] Update index.html --- posts/2014-10-Visualizing-MNIST/index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/posts/2014-10-Visualizing-MNIST/index.html b/posts/2014-10-Visualizing-MNIST/index.html index 1686cde..d8fe158 100644 --- a/posts/2014-10-Visualizing-MNIST/index.html +++ b/posts/2014-10-Visualizing-MNIST/index.html @@ -284,7 +284,7 @@

The MNIST Cube

Exploring this visualization, we can see some glimpses of the structure of MNIST. Looking at the pixels \(p_{18,16}\) and \(p_{7,12}\), we are able to separate a lot of zeros to the bottom right and a lot of nines to the top left. Looking at pixels \(p_{5,6}\) and \(p_{7,9}\) we can see a lot of twos at the top right and threes at the bottom right.

-

Despite minor successes like these, one can’t really can’t understand MNIST this way. The small insights one gains feel very fragile and feel a lot like luck. The truth is, simply, that very little of MNIST’s structure is visible from these perspectives. You can’t understand images by looking at just two pixels at a time.

+

Despite minor successes like these, one can’t really understand MNIST this way. The small insights one gains feel very fragile and feel a lot like luck. The truth is, simply, that very little of MNIST’s structure is visible from these perspectives. You can’t understand images by looking at just two pixels at a time.

But there’s lots of other perspectives we could look at MNIST from! In these perspectives, instead of looking a face straight on, one looks at it from an angle.

The challenge is that we need to choose what perspective we want to use. What angle do we want to look at it from horizontally? What angle do we want to look at it from vertically? Thankfully, there’s a technique called Principal Components Analysis (PCA) that will find the best possible angle for us. By this, we mean that PCA will find the angle that spreads out the points the most (captures the most variance possible).

But, what does it even mean to look at a 784-dimensional cube from an angle? Well, we need to decide which direction every axis of the cube should be tilted: to one side, to the other, or somewhere in between?