python,optimization,machine-learning,scipy,theano

Each call to train_fn is not necessarily a single training epoch. I'm not exactly sure how fmin_cg is implemented, but in general, conjugate gradient methods may call the cost or gradient function more than once per minimziation step. This is (as far as I understand it) required sometimes to find...

Just for reference in case anyone stumbles upon this: This doesn't occur anymore for me. I'm not exactly sure what fixed it, but I think the main difference is that I avoid any multithreading and forks (without exec). This caused many similar problems, e.g. Theano CUDA error: an illegal memory...

As pointed out by user1615070, I just had to install numpy and scipy in my virtualenv before installing Theano (to not use the system versions).

Theano does not support optional parameters. By specifying the function's input parameters as ins=[y,c] you are telling Theano that the function has two 1-dimensional (vector) parameters. As far as Theano is concerned, both are mandatory. When you try to pass None in for c Theano checks that the types of...

python,multithreading,machine-learning,theano

The most direct answer to your question is: you can't parallelize training in the way you desire. BLAS, OpenMP, and/or running on a GPU only allow certain operations to be parallelized. The training itself can only be parallelized, in the way you want, if the training algorithm is designed to...

Keras uses numpy arrays containing the theano.config.floatX floating point type. This can be configured in your .theanorc file. Typically, it will be float64 for CPU computations and float32 for GPU computations, although you can also set it to float32 when working on the CPU if you prefer. You can create...

There is no way to change this value in code running in the same process. The best you could do is to have a "parent" process that alters, for example, the THEANO_FLAGS environment variable and spawns children. However, the method of spawning will determine which environment the children operate in....

For the update, it always use the previous value (the value before the Theano function call). So you found the right thing. For momentum, I think it is normal that there is a delay....

In theano you have to use T.sum(), T.neq(), T.argmax(),T.grad() for symbolic computation with theano variables like T.matrix. You can't use built in sum() for example. If you use theano you have to follow theano's own methods because theano uses a different form of computing to utilize the gpu's architecture. However...

python,debugging,neural-network,theano

Use .eval() to evaluate the symbolic expression Use Test Values ...

python,environment-variables,spyder,theano

You need to tell the sudoers file which Environmental Variables to keep when using the sudo command. To edit the sudoers file run. sudo visudo Then add the following line to the end of it. Defaults env_keep = "LD_LIBRARY_PATH CPATH LIBRARY_PATH" Then export your variable. export LD_LIBRARY_PATH="/path/to/library" Now you should...

python,numpy,theano,deep-learning

Unless I'm mistaken in my interpretation of your code, I don't think what you're trying to do makes sense. If I understand correctly, in model() you are computing a weighted sum over your image pixels using dot(X, w), where I assume that X is an (nimages, npixels) array of image...

I was able to get it working. I am pasting some code here which might help some one. Not very elegant - but works. def shuffle_in_unison(a, b): #courtsey http://stackoverflow.com/users/190280/josh-bleecher-snyder assert len(a) == len(b) shuffled_a = np.empty(a.shape, dtype=a.dtype) shuffled_b = np.empty(b.shape, dtype=b.dtype) permutation = np.random.permutation(len(a)) for old_index, new_index in enumerate(permutation): shuffled_a[new_index]...

KnownFailureTest are a valid return value for nosetests. When Theano started, we where creating tests for features to implement and raised KnownFailureTest in them until we implement them. We do not do that anymore as we end up with to much questions from people about this. So this cause too...

I call Py_Finalize earlier now, not via std::atexit, and so far, I did not have this crash anymore. So I guess that is the solution.

python,numpy,canopy,theano,keras

You specified the wrong output dimensions for your internal layers. See for instance this example from the Keras documentation: model = Sequential() model.add(Dense(20, 64, init='uniform')) model.add(Activation('tanh')) model.add(Dropout(0.5)) model.add(Dense(64, 64, init='uniform')) model.add(Activation('tanh')) model.add(Dropout(0.5)) model.add(Dense(64, 2, init='uniform')) model.add(Activation('softmax')) Note how the output size of one layer matches the input size of the...

There was some problem in my laptop only. It is running fine on a different computer.

Unfortunately, Visual Studio is explicitly required to run some of the CUDA tools. As of February 2015, your best bet is to download VS 2013 Community Edition (http://www.visualstudio.com/en-us/products/visual-studio-community-vs.aspx) and CUDA 6.5 (https://developer.nvidia.com/cuda-downloads). This is a great guide for installing CUDA on Windows 7 and 8.

So, what I did instead was to install a local version of libc and to change LD_LIBRARY_PATH in my .bashrc file to point to it See this answer for why that can't work. Why my setting of LD_LIBRARY_PATH didn't force my new (local) libc to use the new (local)...

This error is often solved by running the following, as mentioned in the documentation. conda install mingw libpython Since you've now installed your own installation of GCC, you may need to play around with environment variables, especially PATH to get things working....

You only need to rearrange your axes to get this to work: >>> import numpy as np >>> a = np.random.rand(3, 4, 5) >>> b = np.random.rand(5, 6) >>> np.allclose(np.einsum('ikj,jl->ikl', a, b), np.dot(a, b)) So with that in mind: >>> a = np.random.rand(3, 5, 4) >>> b = np.random.rand(6, 5)...

python,numpy,parallel-processing,theano

I am not familiar with bitwise operations but this here gives me the same result as your code and is vectorized. import numpy as np # make sure it is a numpy.array v = np.array(v) # vectorized computation N = (v >> 7) & 255 ...

This is a very similar problem to that described in another StackOverflow question. The problem is that you are using a symbolic cast operation which turns the shared variable into a symbolic variable. The solution is to cast the shared variable's value rather than the shared variable itself. Instead of...

Theano shared variables behave more like ordinary Python variables. They have an explicit value that is persistent. In contrast, symbolic variables are not given an explicit value until one is assigned on the execution of a compiled Theano function. Symbolic variables can be thought of as representing state for the...

The error by cuda are returned async. So part of the error message can be unrelated. Here is the first line: MemoryError: error freeing device pointer 0x0000000500C60000 (the launch timed out and was terminated) The answer is in the second part: the launch timed out and was terminated You GPU...

The result of a print operation must be reachable (via some path in the computation graph) from an output of the Theano function. If you want to print an expression that is not used then a simple solution is to just include the result of that expression in the outputs...

This is just a way to tell Theano "create a shared variable and initialize its value to be zero in the same shape as p." This RMSprop method is a symbolic method. It does not actually compute the RmsProp parameter updates, it only tells Theano how parameter updates should be...

I think this is related to pymc3 setting theano.config.compute_test_value = 'raise': https://github.com/pymc-devs/pymc3/blob/master/pymc3/model.py#L395 You can explicitly set it back to ignore to get rid of the error....

No, this is not possible. NumPy arrays need to be rectangular in every pair of dimensions. This is due to the way they map onto memory buffers, as a pointer, itemsize, stride triple. As for this taking up space: np.array([[1,2,3], [4,5]]) actually takes up more space than a 2×3 array,...

You're using an operation class as if it were an operation function. Instead of D = T.nlinalg.AllocDiag(eigen_values) try D = T.nlinalg.AllocDiag()(eigen_values) or D = T.nlinalg.alloc_diag(eigen_values) ...

You need to understand how to vectorise your code. For example: Ui=np.zeros(dim) for k in range(m): Ui+=t.dot(I[i,k],W[k,:]) can be implemented as: Ui = I[None, i] * W Learn on numpy broadcasting. This is a really powerful way of thinking and it do computation faster and with less memory. This work...

python,arrays,numpy,theano,mnist

The problem is probably similar to that of this question. You're trying to create a matrix of pixel values with a row per image. But each image has a different size so the number of pixels in each row is different. You can't create a "jagged" float typed array in...

Use Theano flag: blas.ldflags=-lblas -lgfortran If you do not know how to use Theano flag, check this page: http://deeplearning.net/software/theano/library/config.html Your problem is discussed on that page: http://deeplearning.net/software/theano/install_ubuntu.html...

neural-network,convolution,theano,conv-neural-network

I deduce from this that you intend to have tied weights, i.e. if the first operation were are matrix multiplication with W, then the output would be generated with W.T, the adjoint matrix. In your case you would thus be looking for the adjoint of the convolution operator followed by...

X - u should work exactly as you write it by broadcasting: import theano import theano.tensor as T n = 10 m = 20 X = T.arange(n * m).reshape((n, m)) u = T.arange(0, n * m, m).reshape((n, 1)) r = X - u r.eval() Similar to your updated question, you...

It is correct to think that updates should be a list (or dictionary) of key value pairs where the key is a shared variable and the value is a symbolic expression describing how to update the corresponding shared variable. These two lines create the pairs: zgup = [(zg, g) for...

python,neural-network,gpu,theano

I solved this by updating to the bleeding edge (latest version on GitHub) version. http://deeplearning.net/software/theano/install.html#bleeding-edge-install-instructions Somewhat odd, but a solution nonetheless....

The traceback indicate that in the function equal_compuations(), we didn't cover all case, when doing some comparison. I have a PR with a fix for it here: https://github.com/Theano/Theano/pull/1928 thanks for the report. Your [edit] section, indicate me that you cut some of the errors message. If this happear during optimization...

You need to specify the outputs_info to be None for the output which you don't want to feed back into f. For further information, see the scan documentation. Below is an example which should do what you want. import theano import theano.tensor as T import numpy as np theano.config.exception_verbosity='high' theano.config.optimizer='None'...

numpy,pycuda,theano,deep-learning

The previous answer is partial. The suggestion should not work, as the work around is the one used in the final compiled code. There is optimization that will do this transformation automatically. The title of the question isn't the same as the content. They differ by the axis argument. I'll...

While I believe there's no conv1d in theano, Lasagne (a neural network library on top of theano) has several implementations of Conv1D layer. Some are based on conv2d function of theano with one of the dimensions equal to 1, some use single or multiple dot products. I would try all...

arrays,python-3.x,numpy,theano

What you're printing is a numpy.ndarray. By default they format themselves like this when printed. The output array is a floating point array because, by default, Theano uses floating point tensors. If you want to use integer tensors then you need to specify a dtype: a = T.vector(dtype='int64') Or use...

This work: import theano a=theano.tensor.matrix() idxs=(a<0).nonzero() new_a=theano.tensor.set_subtensor(a[idxs], 0) Do not forget, Theano is a symbolic language. So the variable a isn't changed in the user graph. It is the new variable new_a that contain the new value and a still have the old value. Theano will optimize this to work...

Please find the answer to this question given by pascal at https://groups.google.com/forum/#!topic/theano-users/cNnvw2rUHc8 The failures are correct because the input values are not being provided at the time the function is being called, since it is symbolic. The answer is to use T.minimum(x,threshold)...

I found a solution that works with both Numpy and Theano: c = a[:, :, np.newaxis] * b[:, np.newaxis, :] ...

In general theano.scan is the solution for recursive situations. In a case like yours theano.scan should be used to replace a Python for loop, not in addition to the for loop. It's difficult to see exactly what you're trying to achieve but the extensive use of set_subtensor and inc_subtensor suggests...

This is using a Python feature call "lambda". lambda are unnamed python function of 1 line. They have this forme: lambda [param...]: code In your example it is: lambda prior_result, A: prior_result * A This is a function that take prior_result and A as input. This function, is passed to...

I resolved this by looking through m-x packages-list-packages and seeing that there was (for some reason) an old deprecated version of jedi installed alongside another version of jedi. I deleted all the deprecated installations I had, and the error has gone away, but jedi doesnt seem to be able to...

python,windows,visual-studio,installation,theano

I found a bit of hack to fix it here Essentially it entails finding the cuda_ndarray.cuh file in the <install-dir>/andbox\cuda and adding #include <algorithm>. It still leaves a warning warning C4273: 'round' : inconsistent dll linkage. Which according to this reported issue is due to a conflict between Python and...

I found an awkward workaround import theano import theano.tensor as T import numpy as np vec = T.vector() compare = T.isinf(vec) out = vec[(1-compare).nonzero()] v = [ 1., 1., 1., 1., np.inf, 3., 4., 5., 6., np.inf] v = np.asarray(v) out.eval({var:v}) array([ 1., 1., 1., 1., 3., 4., 5., 6.])...

So the solution is to use theano.clone with the replaces keyword argument. For example, in the exponentiation example, you can define a step function as follows: def step(p, a): replaces = {prior_result: p, A: a} n = theano.clone(next_result, replace=replaces) return n ...

This was a bug in Theano. It has been fixed by Frédéric Bastien via https://github.com/Theano/Theano/pull/2992 More here: https://groups.google.com/forum/#!topic/theano-users/nTRfigJD19w...

opencl0:0 is correct. Could you confirm that pyopencl works? You may have a problem with your opencl (or drivers/cl compiler). However, I think Theano does not quite work with OpenCL at the moment. The current state is there is partial support, enough for "hello world", but not enough to run...

as written in the comment above, i think this is caused by tensor not implicitly importing signal or even signal.conv, hence you have to do the import yourself to use it: In [1]: import theano In [2]: theano.tensor Out[2]: <module 'theano.tensor' from '/usr/local/lib/python2.7/site-packages/theano/tensor/__init__.pyc'> As you can see importing theano also...

python,logistic-regression,theano

p_y_given_x and y_pred are symbolic variable (just python object from Theano). Those python variable that point to the Theano object do not get updated. They just represent the computation we want to do. Think like in pseudo-code. They will be used when compiling the Theano function. It is only then...

You can run scan with an arbitrary number of steps using the n_steps parameter. To pass y_t to the next computation -- and assuming x_t.shape == y_t.shape -- you can use outputs_info=[h0, x_t] as an argument to scan and modify the step function to one_step(h_tm1, y_tm1, W_ih, W_hh, b_h, W_ho,...

amazon-web-services,amazon-ec2,theano,deep-learning,caffe

If you follow the instructions here https://github.com/deeplearningparis/dl-machine then you can set up an AMI image with Theano and Torch. There is also a PR on the config to have caffe by default as well (if you need it, just checkout the branch and run the install script as soon as...

python,installation,theano,nvcc

Theano is designed to work (almost) identically on both CPU and GPU. You don't need a GPU to use Theano and if you don't have a Nvidia GPU then you shouldn't try installing any GPU-specific stuff at all.

python,optimization,neural-network,theano

I don't know whether this is faster, but it may be a little more concise. See if it is useful for your case. import numpy as np import theano import theano.tensor as T minibatchsize = 2 numfilters = 3 numsamples = 4 upsampfactor = 5 totalitems = minibatchsize * numfilters...

The C in CNN stands for convolution. In order to perform a convolution you need variables that taken together form some sort of spatial/temporal/in any way continuous extent, on which a group structure holds, such as translation in space, translation in time, rotations, or something more exotic. This is not...

The number of dimensions must be fixed at the time the Theano function is compiled. Part of the compilation process is to select operation variants that depend on the number of dimensions. You could always compile the function for a high-dimensional tensor and just stack your inputs such that they...

python,matlab,numpy,matplotlib,theano

Matlab and python has a different way to store arrays in memory. Matlab saves an array column-first, while python uses row-first method. Consider, for example, a 2-by-2 matrix M = [1, 2 3, 4] In memory, matlab save the matrix as [1 3 2 4] while python's order is [1...

First that section of your code do nothing. Just remove this line T.set_subtensor(self.pooled_out[i], self.pooled_out[i]) Without full code, I can't test my solution, but I think this would do what you want: pmin = self.pooled_out.min(axis=[2,3], keepdims=True) pmax = self.pooled_out.max(axis=[2,3], keepdims=True) normalized_pooled_out = (self.pooled_out - pmin)/pmax Then normalized_pooled_out contain the symbolic variable...