Introduction

Hello, and welcome back. In this week we’re going to go over the basics of neural network programming.

It turns out that when you implement a neural network there are some techniques that are going to be really important.

For example, if you have a training set of \(m\) training examples, you might be used to processing the training set by having a for-loop step through your \(m\) training examples. But it turns out that when you’re implementing a neural network, you usually want to process your entire training set without using an explicit for-loop to loop over your entire training set. So, you’ll see how to do that in this week’s materials.

Another idea, when you organize the computation of, in your network, usually you have what’s called a forward pause or forward propagation step, followed by a backward pause or what’s called a backward propagation step. And so in this week’s materials, you also get an introduction about why the computations, in learning and neural networks, can be organized in this forward propagation and a separate backward propagation.

For this week’s materials I want to convey these ideas using logistic regression in order to make the ideas easier to understand. But even if you’ve seen logistic regression before, I think that there’ll be some new and interesting ideas for you to pick up in this week’s materials.

So with that, let’s get started.

Theory

An example of binary classification problem

Logistic regression is an algorithm for binary classification. So let’s start by setting up the problem. Here’s an example of a binary classification problem. You might have an input of an image, like that, and want to output a label to recognize this image as either being a cat, in which case you output \(1\), or not-cat in which case you output \(0\), and we’re going to use \(y\) to denote the output label.

Let’s look at how an image is represented in a computer.

To store an image your computer stores three separate matrices corresponding to the red, green, and blue color channels of this image. So if your input image is 64 pixels by 64 pixels, then you would have three 64 by 64 matrices corresponding to the red, green and blue pixel intensity values for your images.11 Although to make this little slide I drew these as much smaller matrices, so these are actually 5 by 4 matrices rather than 64 by 64.

So, to turn these pixel intensity values- into a feature vector, what we’re going to do is unroll all of these pixel values into an input feature vector \(x\). So to unroll all these pixel intensity values into feature vector, what we’re going to do is define a feature vector \(x\) corresponding to this image as follows.

We’re just going to take all the pixel values 255, 231, and so on, until we’ve listed all the red pixels. And then eventually 255, 134 and so on until we get a long feature vector listing out all the red, green and blue pixel intensity values of this image. If this image is a 64 x 64 image, the total dimension of this vector \(x\) will be 64 x 64 x 3 because that’s the total numbers we have in all of these matrices. Which in this case, turns out to be 12,288, - that’s what you get if you multiply all those numbers. And so we’re going to use \(n_x\) = 12288 to represent the dimension of the input features \(x\)22 And sometimes for brevity, I will also just use lowercase \(n\) to represent the dimension of this input feature vector.

So in binary classification, our goal is to learn a classifier that can input an image represented by this feature vector \(x\), and predict whether the corresponding label \(y\) is 1 or 0, that is, whether this is a cat image or a non-cat image.

Notation

Let’s now lay out some of the notation that we’ll use throughout the rest of this course. A single training example is represented by a pair, (x,y) where \(x\) is an x-dimensional feature vector33 \(x \in \mathbb{R}^{n_x}\), and \(y\), the label, is either 0 or 144 \(y \in \left\{ 0,1 \right\}\). Your training sets will comprise \(m\) training examples. ⊕(x⁽¹⁾, y⁽¹⁾): 1st training example;  (x⁽²⁾, y⁽²⁾), 2nd training example;  (x^(m), y^(m)), last training example. And so, your training sets will be written \((x^{(1)}, y^{(1)})\) which is the input and output for your first training example; \((x^{(2)}, y^{(2)})\), for the second training example, up to \((x^{(m)}, y^{(m)})\), which is your last training example. And then, that altogether is your entire training set.

⊕m_train: training examples
m_test: test examples