Exercise M1

Consider a simple "neural network" function F: Rm -> Rn, defined as F(X) = W2T(W1X), where T(yi,j) = tanh(yi,j).

Given a set of k "training samples" (X1,Y1), where X1 is an m by k matrix and Y1 an n by k matrix, and a set (X2,Y2) of k testing samples of the same dimensions, write a procedure (or give a series of matlab steps) to find the weight matrices W1 and W2, so that the norm(F(X1) - Y1) is minimized, and compare this with the norm(F(X2) - Y2).

A matlab program is provided that implements a sample non-linear function to generate test data sets (X1,Y1) and (X2,Y2); typically, X1 and X2 can be random, with Yi the output of the sample non-linear function on input Xi. For this exercise, you can assume m is no larger than 8, n is no larger than 4, and k no larger than 20, though you can use larger values of k, if desired, to get a better fit.