COMP-4360 Machine Learning Assignment 2

Due: Wednesday, 7th March 2012 at 15:30

The following list contains additional information and comments about the project.

  1. Tue., Feb. 21 14:44:00 CST 2012: Assignment opened.

Introduction

This page describes the Neural Network assignment of the course COMP 4360 Machine Learning.

Neural Networks

In this assignment you will implement a simple back-propagation neural network using sigmoid activation functions. In this part of the assignment, you must implement a back-propagation network as described in class using sigmoid activation function without a momentum term. Implement a learning rate α=0.1. The input and hidden layer as well as the hidden and output layer are fully connected as shown in . Use an activation level of 0.1 for 0 and 0.9 for 1. Don’t forget to add a constant -1 as offset term to the inputs and connect it to all nodes in the hidden and output layer to act as offset term.

4-Bit Counter

In the first part of the assignment, you will implement a simple 4-bit counter using a neural network. The network for the counter consists of 4 input nodes, nh hidden nodes, and 4 output nodes. The input to the counter is a number between 0 and 15 in binary (e.g., 11=%1011) and the output of the counter network is the binary presentation of the input number + 1 (e.g., 12=%1100). The architecture of the network for the counter is shown in Figure 1.

Figure 1: Architecture of the Counter Neural Network

Try different number of nodes in the hidden layer from 1 to 7 hidden nodes. Randomly select four of the possible sixteen input patterns as validation set. Compare the accuracy of your system on the test set and the training set. Determine the best number of hidden nodes.

Show the results of your tests in a table similar to this one

Number of Hidden Nodes

MSE Training

Bit Errors Training

MSE Testing

Bit Errors Testing

1

0.0270298

1

0.024398/1

1

2

0.0205392

1

0.034855/1

1

3

0.0158417

0

0.062132/0

0

4

0.0152964

0

0.062132/0

0

5

0.0150203

0

0.059700/0

0

6

0.0150032

0

0.059992/0

0

7

0.0150006

0

0.060026/0

0

In this test, the network with five hidden nodes performed best.

HuroCup Sprint Vision Processing

In the HuroCup Sprint event, a robot has to walk forwards and backwards in a 50cm lane for 3.0m. This year, we would like to use a novel target pattern as shown in the picture below and in this sprint marker youtube video.

The idea is that the robot can identify the direction of the track by looking at the relative shift of the yellow/black and green/red line.

Training Data

The input to the network will be a set of images with associated offsets and distance along the centre line information. The offset and the distance are encoded in the filenames. You can download the images here.

For example, the image file named

        sprint_050r_106.ppm

corresponds to a picture taken when the robot was 50cm to the right of the centre line and 106cm along the centre line to the target.

This scenario is shown in the drawing below.

Some sample images are shown below.

sprint_0150l_150.ppm

sprint_000l_0050.ppm

sprint_050r_0060.ppm

PPM File Format

There are a total of 1400 images in the training set. Each image is in the RGB PPM file format. The PPM file format consists of three header lines

P6\n

320 240\n

255\n

which represents a magic header (P6 for RGB colour images), the image width and height in ASCII on the next line, and the number of colours in ASCII per RGB band. Following this header is a width * height * 3 sized array of binary data which corresponds to RGB pixels. Each channel (R,G,B) is one byte.

Create a neural network that is able to predict the offset and distance to the target given a set of training images.

Unless otherwise specified use 50% of the images for training and test the accuracy of your algorithm with the remaining 50%. Use a uniform distribution to sample from the set of images to create the training and testing sets.

Convert the image date into training data by scaling the RGB channels to be between 0 and 1 by dividing by 255.

Subsampling

The original size of the images is 320x240, which would result in a very large input vector and number of nodes. Therefore, you should subsample the image first and reduce it to a 32x24 sized image. The easiest way to subsample the image is to simply take every nth pixel only.

Discretization of the domain

Each network uses seven output nodes for the direction (105l,100l,050l,000l,050r,100r, and 150r) and five output nodes for the distance along the centre line (200cm, 150cm, 100cm, 50cm, and 0cm).

In the first experiments the output is determined by the node with the maximum output resulting in a discretization of 50cm in the offset and 50cm in the distance along the centre line.

Colour Images

Create a network with 32*24*3 input nodes in the first layer. Convert the pixels into training data by scaling the RGB channels to be between 0 and 1 by dividing by 255.

Greyscale Images

Rerun the experiments described above and compare the performance of the network when using greyscale images. Convert every RGB pixel into a greyscale value (Grey=(R+G+B)/3) and create a network with 32*24 input nodes and the same number of output nodes as in the experiments using colour images.

Number of Hidden Nodes

Compare the performance of the networks for colour and greyscale images using from 1 to 7 hidden nodes.

Learning Curve

Show the dependence of the ANN to the number of training examples used. That is, show in a table the performance of the colour image network using 1, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, and 300 training examples.

Submission

To hand in your assignment:

  1. create a directory to hold all of your data. The directory name should be 4550_a<num>_< login_name > .
  2. Inside of the directory comp_<course_number>_assignment_<num>_< login_name > , create a directory source .
  3. put your source code for your assignment solution (it's ok if you have more than one source file) into the directory source .
  4. Your program must compile by just running the command make in the source directory.
  5. place a copy of the electronic handin honesty declaration (modified to include your own name and student number) at the top level above your source directories -understand that doing so and handing it in from your account both securely identifies you and holds you to the responsibilities of a paper honesty declaration.
  6. Write a readme.txt file to explain anything you feel is necessary. This may include special features/bugs of your program. What parts of the assignment did you implemement. Making things clear here for the marker is in your own best interest.
  7. Once you're happy that you have everything and it all works the way it should, move outside the directory itself (i.e. make sure you can see the name of the directory in the list of files if you do an ls).
  8. at the unix prompt, type: handin <course_number>_ a<num>_< login_name >. For example: handin 4360_a3_umsmith

You should receive a response saying how many bytes were handed in - all of the files you put in your handin directory are compressed for handin, so the number reported may seem unusually small.

If you change your mind about what is handed in or think you've made a mistake, you can enter the handin command as often as you want, each new handin will erase the old. Remember there is a hard deadline on the due date, at which point the handin system will be disabled.

When marking your assignments, the marker will get a copy of everything you placed in the directory. They will start a telnet unix session, cd into your source directory. Depending on the programming language that you have used they will either: (a) execute make , followed by ./a.out [args] in case of C or C++, or (b) javac *.java followed by java ass2 [args] in case of a Java submission.

You will get an email back to the cc account you used to do the handin after the marker is finished. Please ensure the mail on this account is not forwarded elsewhere.