Menu
  • HOME
  • TAGS

Multiclass Neural Network Issue

java,neural-network,backpropagation

If you move from binary classification to multiclass classification, you'll have to generalize your backpropagation algorithm to properly handle more than two classes. The main differences to binary classification are that the update changes to: with: being the new score where the argument y (output) is chosen that yields the...

Neural Network Error oscillating with each training example

machine-learning,artificial-intelligence,neural-network,backpropagation

I agree with the comments that this model is probably not the best for your classification problem but if you are interested in trying to get this to work I will give you the reason I think this oscillates and the way that I would try and tackle this problem....

Neural Network and Temporal Difference Learning

artificial-intelligence,neural-network,backpropagation,reinforcement-learning,temporal-difference

The backward and forward views can be confusing, but when you are dealing with something simple like a game-playing program, things are actually pretty simple in practice. I'm not looking at the reference you're using, so let me just provide a general overview. Suppose I have a function approximator like...

What is the difference between Backpropogation and feed-forward Neural Network

machine-learning,neural-network,classification,backpropagation

A Feed-Forward Neural Network is a type of Neural Network architecture where the connections are "fed forward", i.e. do not form cycles (like in recurrent nets). The term "Feed forward" is also used when you input something at the input layer and it travels from input to hidden and...

Torch Lua: Why is my gradient descent not optimizing the error?

lua,neural-network,backpropagation,training-data,torch

why the predictionValue variable is always the same? Why doesn't it get updates? First of all you should perform the backward propagation only if predictionValue*targetValue < 1 to make sure you back-propagate only if the pairs need to be pushed together (targetValue = 1) or pulled apart (targetValue =...

Trouble Understanding the Backpropagation Algorithm in Neural Network

java,algorithm,artificial-intelligence,neural-network,backpropagation

I finally found the problem. For the XOR, I didn't need any bias and it was converging to the expected values. I got exactly the output when you round the final output. What was needed is to train then validate, then train again until the Neural Network is satisfaying. I...

i need a way to train a neural network other than backpropagation

neural-network,backpropagation

Genetic algorithm would be a good candidate here. A chromosome could encode the weights of the neural network. After evaluation you assign a fitness value to the chromosomes based on their performance. Chromosomes with higher fitness value have a higher chance to reproduce, helping to generate better performing chromosomes in...

Bug in Resilient Backpropagation?

artificial-intelligence,neural-network,backpropagation,encog

There are several different variants of the RPROP algorithm. Encog has been modified to support more of them since the publication of the book. The book focuses on the classic RPROP, as defined by a paper by Reidmiller. Subsequent papers made additional algorithms. This accounts for some differences between Encog's...

Thresholds in backpropagation

machine-learning,threshold,backpropagation

What you call "thresholds" are actually biases in affine transformations denoted by neurons f(w,x,b) = g(<w,x>+b) Biases should not be used as a constants as you suggest but rather - trained just as any other parameter in the network. Usually one simply adds a hypothethical "bias neuron" always equal to...

Subscript indices must be real positive integers, and they are (Matlab) [duplicate]

matlab,runtime-error,backpropagation

In the l == 1 case, you illegally try to use net.layerOutput{l-1} 0 is not positive. The input layer needs to use the inputs, not inter-layer connections....

Any Ideas for Predicting Multiple Linear Regression Coefficients by using Neural Networks (ANN)?

matlab,neural-network,linear-regression,backpropagation,perceptron

A neural network will generally not find or encode a formula like t = a + b*X1 + c*X2, unless you built a really simple one with no hidden layers and linear output. If you did then you could read the values [a,b,c] from the weights attached to bias, input...

Use number of misclassificatios as objective function for back propagation

machine-learning,classification,backpropagation

Because of mathematics. We'd really like to minimize number of misclassifications, but this target would be non-smooth (and not even continuous), and thus hard to optimize. So, in order to optimize it, we use smooth "proxies": sum-of-squares penalizes your mistakes in a continuous way. Namely, a (very) tiny deviation in...

In neural networks, why is the bias seen as either a “b” parameter or as an additionnal “wx” neuron?

machine-learning,neural-network,backpropagation

Note: it makes little sense to ask for the best method here. Those are two different mathematical notations for exactly the same thing. However, fitting the bias as just another weight allows you to rewrite the sum as a scalar product of an observed feature vector x_d with the weight...

How can I add concurrency to neural network processing?

neural-network,backpropagation,gradient-descent

If you have 5 hidden layers (assuming with 100 nodes each) you have 5 * 100^2 weights (assuming the bias node is included in the 100 nodes), not 100^5 (because there are 100^2 weights between two consecutive layers). If you use gradient descent, you'll have to calculate the contribution of...

Why do I must use Sobel Operator?

computer-vision,artificial-intelligence,neural-network,backpropagation,sobel

Handwriting is composed by strokes, usually the strokes are filled with a solid color: with these assumption a stroke is well described by its edges while its color or the background color is not so useful in describing it. The edges are basically described by boolean values: a pixel is...

Unit testing backpropagation neural network code

unit-testing,neural-network,backpropagation

I'm in the middle of doing something similar for my degree. What you are looking for is integration tests, not unit tests. Unit test only tell you if the code works the way you want it to. To check if the algorithm actually works correctly, you should write integration tests...

FeedForward Neural Network: Using a single Network with multiple output neurons for many classes

machine-learning,neural-network,backpropagation,feed-forward

In short, yes it is a good approach to use a single network with multiple outputs. The first hidden layer describes decision boundaries (hyperplanes) in your feature space and multiple digits can benefit from some of the same hyperplanes. While you could create one ANN for each digit, that kind...

Neural Network not fitting XOR

matlab,machine-learning,neural-network,octave,backpropagation

Start with a larger range when initialising weights, including negative values. It is difficult for your code to "cross-over" between positive and negative weights, and you probably meant to put * 2 * epsilon_init - epsilon_init; when instead you put * 2 * epsilon_init * epsilon_init;. Fixing that may well...

Derivation of the Backpropagation Algorithm for Neural Networks

algorithm,backpropagation

Ok, I understand why you thought that, but 'y' is the input sum and the output only depends on it, if you wanna find the output, it's very simple, you just need apply it by the activation function phi, in this case I think we should use the phi because...

Aforge BackPropagation Using

c#,neural-network,aforge,backpropagation

You can make normalize your input data and denormalize output data. For instance inputdata/100 and than outputdata*100. Try please.

What are units in neural network (backpropagation algorithm)

machine-learning,artificial-intelligence,neural-network,classification,backpropagation

The "units" are just floating point values. All computations happening there are vector multiplications, and thus can be parallelized well using matrix multiplications and GPU hardware. The general computation looks like this: double v phi(double[] x, double[] w, double theta) { double sum = theta; for(int i = 0; i...