java,neural-network,backpropagation

If you move from binary classification to multiclass classification, you'll have to generalize your backpropagation algorithm to properly handle more than two classes. The main differences to binary classification are that the update changes to: with: being the new score where the argument y (output) is chosen that yields the...

machine-learning,artificial-intelligence,neural-network,backpropagation

I agree with the comments that this model is probably not the best for your classification problem but if you are interested in trying to get this to work I will give you the reason I think this oscillates and the way that I would try and tackle this problem....

artificial-intelligence,neural-network,backpropagation,reinforcement-learning,temporal-difference

The backward and forward views can be confusing, but when you are dealing with something simple like a game-playing program, things are actually pretty simple in practice. I'm not looking at the reference you're using, so let me just provide a general overview. Suppose I have a function approximator like...

machine-learning,neural-network,classification,backpropagation

A Feed-Forward Neural Network is a type of Neural Network architecture where the connections are "fed forward", i.e. do not form cycles (like in recurrent nets). The term "Feed forward" is also used when you input something at the input layer and it travels from input to hidden and...

lua,neural-network,backpropagation,training-data,torch

why the predictionValue variable is always the same? Why doesn't it get updates? First of all you should perform the backward propagation only if predictionValue*targetValue < 1 to make sure you back-propagate only if the pairs need to be pushed together (targetValue = 1) or pulled apart (targetValue =...

java,algorithm,artificial-intelligence,neural-network,backpropagation

I finally found the problem. For the XOR, I didn't need any bias and it was converging to the expected values. I got exactly the output when you round the final output. What was needed is to train then validate, then train again until the Neural Network is satisfaying. I...

neural-network,backpropagation

Genetic algorithm would be a good candidate here. A chromosome could encode the weights of the neural network. After evaluation you assign a fitness value to the chromosomes based on their performance. Chromosomes with higher fitness value have a higher chance to reproduce, helping to generate better performing chromosomes in...

artificial-intelligence,neural-network,backpropagation,encog

There are several different variants of the RPROP algorithm. Encog has been modified to support more of them since the publication of the book. The book focuses on the classic RPROP, as defined by a paper by Reidmiller. Subsequent papers made additional algorithms. This accounts for some differences between Encog's...

machine-learning,threshold,backpropagation

What you call "thresholds" are actually biases in affine transformations denoted by neurons f(w,x,b) = g(<w,x>+b) Biases should not be used as a constants as you suggest but rather - trained just as any other parameter in the network. Usually one simply adds a hypothethical "bias neuron" always equal to...

matlab,runtime-error,backpropagation

In the l == 1 case, you illegally try to use net.layerOutput{l-1} 0 is not positive. The input layer needs to use the inputs, not inter-layer connections....

matlab,neural-network,linear-regression,backpropagation,perceptron

A neural network will generally not find or encode a formula like t = a + b*X1 + c*X2, unless you built a really simple one with no hidden layers and linear output. If you did then you could read the values [a,b,c] from the weights attached to bias, input...

machine-learning,classification,backpropagation

Because of mathematics. We'd really like to minimize number of misclassifications, but this target would be non-smooth (and not even continuous), and thus hard to optimize. So, in order to optimize it, we use smooth "proxies": sum-of-squares penalizes your mistakes in a continuous way. Namely, a (very) tiny deviation in...

machine-learning,neural-network,backpropagation

Note: it makes little sense to ask for the best method here. Those are two different mathematical notations for exactly the same thing. However, fitting the bias as just another weight allows you to rewrite the sum as a scalar product of an observed feature vector x_d with the weight...

neural-network,backpropagation,gradient-descent

If you have 5 hidden layers (assuming with 100 nodes each) you have 5 * 100^2 weights (assuming the bias node is included in the 100 nodes), not 100^5 (because there are 100^2 weights between two consecutive layers). If you use gradient descent, you'll have to calculate the contribution of...

computer-vision,artificial-intelligence,neural-network,backpropagation,sobel

Handwriting is composed by strokes, usually the strokes are filled with a solid color: with these assumption a stroke is well described by its edges while its color or the background color is not so useful in describing it. The edges are basically described by boolean values: a pixel is...

unit-testing,neural-network,backpropagation

I'm in the middle of doing something similar for my degree. What you are looking for is integration tests, not unit tests. Unit test only tell you if the code works the way you want it to. To check if the algorithm actually works correctly, you should write integration tests...

machine-learning,neural-network,backpropagation,feed-forward

In short, yes it is a good approach to use a single network with multiple outputs. The first hidden layer describes decision boundaries (hyperplanes) in your feature space and multiple digits can benefit from some of the same hyperplanes. While you could create one ANN for each digit, that kind...

matlab,machine-learning,neural-network,octave,backpropagation

Start with a larger range when initialising weights, including negative values. It is difficult for your code to "cross-over" between positive and negative weights, and you probably meant to put * 2 * epsilon_init - epsilon_init; when instead you put * 2 * epsilon_init * epsilon_init;. Fixing that may well...

Ok, I understand why you thought that, but 'y' is the input sum and the output only depends on it, if you wanna find the output, it's very simple, you just need apply it by the activation function phi, in this case I think we should use the phi because...

c#,neural-network,aforge,backpropagation

You can make normalize your input data and denormalize output data. For instance inputdata/100 and than outputdata*100. Try please.

machine-learning,artificial-intelligence,neural-network,classification,backpropagation

The "units" are just floating point values. All computations happening there are vector multiplications, and thus can be parallelized well using matrix multiplications and GPU hardware. The general computation looks like this: double v phi(double[] x, double[] w, double theta) { double sum = theta; for(int i = 0; i...