I was just working on something like this yesterday for a scientist's data. Put all of the common parts of your computation into a function. Put all of the variable stuff (filenames, etc) in a list (or create a list from reading from a file). Then loop over your list,...

Is it critical that you have to use that distance function? I think the dist should be much more efficient. #Making your zip.table a data.table helps us with speed library(reshape) library(data.table) setDT(zip.table) #Calculate distance matrix and put into table form setorder(zip.dist,zip) zip.dist <- dist(zip.table[,.(longitude=abs(longitude),latitude)]) zip.dist <- as.matrix(zip.dist) zip.dist <- melt(zip.dist)[melt(upper.tri(zip.dist))$value,]...

This uninformative error is due to the way you have organized your data vector. It is 2 rows by n columns, and PyMC expects it to be n rows by 2 columns. The following modification makes this code (almost) work for me: xy = MvNormal('xy', mu=mean, tau=precision, value=data.T, observed=True) I...

python,pandas,matplotlib,correlation

You may be able to do what you want with glue" http://www.glueviz.org/en/stable/

Pearson Correlation is defined to be a number relating one sequence (or vector) of values to another (look it up). As far as I know there is no roughly equivalent definition for a group of vectors to another, but you could do something like take the average vector (of the...

r,data-visualization,heatmap,correlation,r-corrplot

You can try the heatmap.2 function from the gplots package which I like for heatmaps and it will give something very similar to the graph you are after (I rounded to second digit for the example below. Use as many digits as you want): Some data manipulation initially: mycor <-...

python,numpy,scipy,correlation

"Correlation distance" is not the same as the correlation coefficient. A "distance" between two equal points is supposed to be 0. (If you search for "correlation distance", note that there is yet another concept, the "distance correlation", which is not the same as the "correlation distance".)

The specific problem with your code is that in your line corrcoef([X,Y]) you just lumped your X and Y into one variable. You can definitely get the answer that you want out of this matrix (the off-diagonal terms are the correlation between the columns of X and your Y) but...

Try r2 <- matrix(0, ncol=3, nrow=2, dimnames=list( paste0('y',1:2), paste0('x',1:3))) r2[] <- paste(round(res$r[4:5,1:3],2), round(res$P[4:5,1:3],4), sep="; ") Update You could create a function like below f1 <- function(df){ df1 <- df[order(colnames(df))] indx <- sub('\\d+', '', colnames(df1)) indx1 <- which(indx[-1]!= indx[-length(indx)]) indx2 <- (indx1+1):ncol(df1) r2 <- matrix(0, ncol=indx1, nrow=(ncol(df1)-indx1), dimnames=list(colnames(df1)[indx2], colnames(df1)[1:indx1])) r1 <-...

The correlation coefficient is a number representing the similarity between 2 images in relation with their respective pixel intensity. As you pointed out this function is used to calculate this coefficient: Here A and B are the images you are comparing, whereas the subscript indices m and n refer to...

Here is a simple solution: sim.cor <- function(R, Marginal, n, K) { res <- numeric(length = K) for(i in 1:K) res[i] <- cor(ordsample(n, Marginal, R))[1,2] res } where n is the sample size and K is the number of times you want to repeat. So, in your example, you can...

database,r,matrix,export,correlation

You might just look at the structure of the rcorr object. Ill create one with: > rcx=rcorr(x) then look at its stucture with: > str(rcx) List of 3 $ r: num [1:10, 1:10] 1 0.0503 0.0309 0.0462 0.0719 ... ..- attr(*, "dimnames")=List of 2 .. ..$ : NULL .. ..$...

This should be fast: cA <- A - rowMeans(A) cB <- B - rowMeans(B) sA <- sqrt(rowMeans(cA^2)) sB <- sqrt(rowMeans(cB^2)) rowMeans(cA * cB) / (sA * sB) ...

For matrix m, you could do: data.frame(row=rownames(m)[row(m)], col=colnames(m)[col(m)], corr=c(m)) # row col corr # 1 60516 60516 1.000000000 # 2 45264 60516 -0.370793010 # 3 02117 60516 -0.082897940 # 4 60516 45264 -0.370793012 # 5 45264 45264 1.000000000 # 6 02117 45264 0.005145601 # 7 60516 02117 -0.082897941 # 8...

r,regression,correlation,weighted-average

The answer can be found in that CERN paper: ftp://ftp.desy.de/pub/preprints/cern/ppe/ppe94-185.ps.gz the procedure is a generalised least square regression. See the equation (2) page (1) for the result....

So us we discussed in the comments, you have to compare "apples with apples", thus both data sets have to be compared by unique dates. First approach will be to give same weight to each event, count them and compare to "model.births" ## Aggrgating "model.weather" by date and counting events...

You could try indx <- which(Rvalue==1 & Pvalue < 0.05 & !is.na(Pvalue), arr.ind=TRUE) d1 <- data.frame(rN=row.names(Rvalue)[indx[,1]], cN=colnames(Rvalue)[indx[,2]], Pval=signif(Pvalue[indx], digits=4)) head(d1,2) # rN cN Pval #1 41700 41699 0 #2 41701 41699 0 Update Not sure why you are getting the same result when you change the cutoff. It may be...

r,date,time-series,correlation

This is a little late to the party, but the below is a pretty compact solution with dplyr and rollapply from (zoo package). library(dplyr) library(zoo) dt<-seq(as.Date("2013/1/1"), by = "days", length.out = 20) df1<-data.frame("ABC",dt,rnorm(20, 0,3),rnorm(20, 2,4) ) names(df1)<-c("name","date","x","y") df2<-data.frame("XYZ",dt,rnorm(20, 2,5),rnorm(20, 3,10) ) names(df2)<-c("name","date","x","y") df<-rbind(df1,df2) df<-df %>% group_by(name)%>% arrange(date) %>% do({ correl...

Let's use this correlation formula : You can implement this for X as the M x N array and Y as the other separate time series array of N elements to be correlated with X. So, assuming X and Y as A and B respectively, a vectorized implementation would look...

spring,logging,correlation,spring-amqp,mdc

Inject another custom message converter into the AmqpInvokerServiceExporter. Set the MDC (from the header) in fromMessage(), clear it when the reply is mapped (in toMessage)....

You need to use cbind, not as.matrix, or the approach that I show below. Always identify the R packages you are using. You claim that your data set a 'vector data set'. I doubt that. I am assuming it is a matrix. If it is a matrix, you can do...

You can just take the first column of your matrix and suppress the color labels: corrplot(cor(iris[,1:4])[1:4,1, drop=FALSE], cl.pos='n') ...

I had a similar problem and use="pairwise.complete.obs" solved it

r,ggplot2,replication,correlation,confidence-interval

You can add the chart and axis titles yourself, but this code does what I think you're looking for using ggplot2 and the 'psychometric' package: library(ggplot2) library(psychometric) corSamp <- function(x) { # return the correlation between price and carat on diamonds for a given sample size index <- sample(1:nrow(diamonds), x)...

python,opencv,numpy,fft,correlation

Yup, just like most things I've thought to accomplish by looping over the array: numpy has a built-in solution. [numpy.nonzero][1] numpy.nonzero(a) Return the indices of the elements that are non-zero. Returns a tuple of arrays, one for each dimension of a, containing the indices of the non-zero elements in that...

@William Lisowski debugged your code in a comment, but you can simplify the whole procedure. Create the tuples beforehand using the user-written command tuples (ssc install tuples). clear set more off *----- example data ----- sysuse auto keep mpg weight price gen time = _n tsset time *----- what you...

matlab,correlation,cross-correlation

First, Let me transpose your code in R2014b: load carbig; data = [Displacement Horsepower Weight Acceleration MPG]; % Truncate the data, to follow-up with your sample code data = data(1:100,:); nans = sum(isnan(data),2) > 0; [wx, wy, r, U, V,] = canoncorr(X(~nans,1:3),X(~nans,4:5)); OK, now the trick is that the vectors...

matlab,time-series,correlation,matrix-inverse

You might be confusing the Correlation matrix of a random vector (multivariate random variable), and the autocorrelation matrix of a random process (stochastic process)... So if your serie is a vector autoregressive model of order 1 (which it seems to be, so h' is your coefficient matrix), then indeed E[y(t-1)*y(t-1)']...

Changes of scale will not change the correlation coef: > out2[,2] <- out[,2]*10 > cor(out2) [,1] [,2] [1,] 1.0 0.5 [2,] 0.5 1.0 plot(out2) > lm(out2[,2]~out2[,1]) Call: lm(formula = out2[, 2] ~ out2[, 1]) Coefficients: (Intercept) out2[, 1] -5.732e-16 5.000e+00 ...

arrays,loops,for-loop,correlation

for (int index = 0; index < xpoints.length; index++) { double x = Double.parseDouble(xpoints[index]); double y = Double.parseDouble(ypoints[index]); sumnationxy += (x * y); } ...

You seem to have a slight misunderstanding of how cross-correlation works. Cross-correlation takes one signal, and compares it with shifted versions of another signal. If you recall, the (unnormalized) cross-correlation of two signals is defined as: s and h are two signals. Therefore, we shift versions of the second signal...

python,numpy,matplotlib,heatmap,correlation

You can simply insert an extra singleton dimension in order to turn your (n,) 1D vector into a (1, n) 2D array, then use pcolor, imshow etc. as normal: import numpy as np from matplotlib import pyplot as plt # dummy correlation coefficients coeffs = np.random.randn(10, 10) row = coeffs[0]...

correlation,windows-applications,visual-studio-cordova

You can use native project to create app package for Windows store. Refer this on using windows native project to create store package. You can also try Visual Studio 2015 CTP6 which has a fix for this issue.

matlab,correlation,categorical-data

That's pretty simple to do. In terms of two-dimensional problems, data that exhibit a positive correlation are those data points where there is clearly an increase in the output for every increase in the independent value r. Similarly, for negative correlation, there is a decrease in the output for every...

Following the advice from @knedlsepp, and using filter as in the movingstd, I found the following solution, which is quite fast: function Cor = MovCorr1(Data1,Data2,k) y = zscore(Data2); n = size(y,1); if (n<k) Cor = NaN(n,1); else x = zscore(Data1); x2 = x.^2; y2 = y.^2; xy = x .*...

You could get the "interim" output using dplyr and tidyr: library(dplyr) library(tidyr) cors <- df %>% spread(Item_Id, price) %>% group_by(Location_Id) %>% do(correlation = cor(.[, -(1:2)], use = "pairwise.complete.obs")) The way that this works is that the spread function (from tidyr) spreads the As, Bs, Cs etc into their own columns:...

If you've problems using transfer properties step to get the JSON value from your response, you can use a groovy test step to achieve your goal. So create a groovy test step to parse your response, get your value and set it as a property (for example at testCase level)...

r,performance,for-loop,matrix,correlation

Your code is slow mainly because your do rbind, which creates a new matrix and copies all the data from the previous one. This generates a huge overhead. A simple solution is to create the matrix before the loop, and then fill it : output = matrix(0, nrow=nrow(A)*ncol(B), ncol=4) for(i...

r,statistics,correlation,vegan

You want the anova() method that vegan provides for cca(), the function that does CCA in the package, if you want to test effects in a current model. See ?anova.cca for details and perhaps the by = "margin" option to test marginal terms. To do stepwise selection you have two...

p-values is used for testing the hypothesis of no correlation and it is actually one of the outputs of the correlation analysis not a parameter to change. by definition: Each p-value is the probability of getting a correlation as large as the observed value by random chance, when the true...

r,statistics,time-series,correlation,xts

What about using rollapply in different way? As you dont supply the complete dataset, here a demonstration how I mean it: set.seed(123) m <- matrix(rnorm(100), ncol = 10) rollapply(1:nrow(m), 5, function(x) cor.mean(m[x,])) [1] -0.080029692 -0.038168840 -0.058443824 0.005699772 -0.014459878 -0.021569173 As I just figured out, you can also use the function...

I have four suggestions for you, depending on what exactly you are doing to compute your matrices. I'm assuming the example you gave is a simplified version of what needs to be done. First Method - Adjusting the inner loop index One thing you can do is change your j...

An option using data.table library(data.table) res <- setDT(df1)[, lapply(.SD[,-1, with=FALSE], function(x) cor(x,Col1)) , by=Group1] setnames(res,2:4, paste0('Col1Cor', names(res)[-1])) You can specify the use option in ?cor for removing the "missing values" EDIT: here is what the data looks like that is giving me the error: http://postimg.org/image/g9rfkamod/...

You can use complete.cases to get a logical vector of complete rows (TRUE = complete); then subsetting inside ad-hoc function used for testing too library(gtools) df <- data.frame(temp=rnorm(100, 10:30), prec=rnorm(100, 1:300), humi=rnorm(100, 1:100)) df$prec[c(1:10, 25:30, 95:100)] <-NA df$humi[c(15:19, 20:25, 80:90)] <-NA my.fun <- function(x,y) { my.df <- data.frame(x,y) my.df.cmpl <-...

You can try library(Hmisc) rcorr(as.matrix(df), type='spearman')$P ...

python,matrix,pandas,yahoo,correlation

This is a start which answers questions 1-3 (you should only have one question per post). import pandas.io.data as web import datetime as dt import pandas as pd end_date = dt.datetime.now().date() start_date = end_date - pd.DateOffset(years=5) symbols = ['AAPL', 'IBM', 'GM'] prices = web.get_data_yahoo(symbols=symbols, start=start_date, end=end_date)['Adj Close'] returns = prices.pct_change()...

If 'd1', 'd2', ...'d13' are the datasets and the columns are the in the same order, we can place the dataset in a list and get the cor for the specified columns. There are options in ?cor to compute the covariances in the presence of missing values. Here, I used...

Ok folks I've found it. Many thanks @user456789123 for helping me with 2. So the code I used: proc sort data=test; by varGroup; run; Which will help in step 2. that I will need to get the Cronbach's alpha for each Group. proc corr data=test alpha outp=stats; var A B...

With only one observation per day, it is not possible to compute a correlation. But you can compute the correlations on a moving window, e.g., with rollapply. # Convert the data to time series library(zoo) d <- zoo( airquality, sprintf( "%02i-%02i", airquality$Month, airquality$Day ) ) # Compute the correlations r...

To get the correlation between all row combinations, try cor(t(df1), t(df2)) ...

Correlation isn't affecting by linear transformation of the underlying variables. So the most direct way to get what you want could be: out <- as.data.frame(mvrnorm(10, mu = c(0,0), Sigma = matrix(c(1,0.56,0.56,1),, ncol = 2), empirical = TRUE)) out$V1.s <- (out$V1 - min(out$V1))*1000+10 out$V2.s <- (out$V2 - min(out$V2))*200+30 Now the data...

The problem is the NAs in the data - you need to keep them in until you've paired up the observations, but themn tell cor() and cor.test() what to do with the NA data - slightly confusingly they both have different ways of specifying that the NAs should be removed....

matlab,time-series,covariance,correlation,matrix-inverse

For p=2, expression for y_tminus1 should be (see expressions after Eq.1 in the paper) y_tminus1 = Y(end-2:end-1).' Your mult is OK. For Eq. 20 you need to take expectation E(mult). For this you need to generate multiple paths and take an average over them. For the RHS of Eq. 25,...

python,arrays,numpy,scipy,correlation

Correlation (default 'valid' case) between two 2D arrays: You can simply use matrix-multiplication np.dot like so - out = np.dot(arr_one,arr_two.T) Correlation with the default "valid" case between each pairwise row combinations (row1,row2) of the two input arrays would correspond to multiplication result at each (row1,row2) position. Row-wise Correlation Coefficient calculation...

cv::Rect and cv::Mat::copyTo cv::Mat img=cv::imread("image.jpg"); cv::Mat imgTranslated(img.size(),img.type(),cv::Scalar::all(0)); img(cv::Rect(50,30,img.cols-50,img.rows-30)).copyTo(imgTranslated(cv::Rect(0,0,img.cols-50,img.rows-30))); ...

The problem is likely NA values in your data set. When you set use="complete.obs" and you apply that to more than two columns, it only uses rows where all of those columns are not missing. If you only wanted to skip missing values for the unique pairs of columns, set...

matlab,correlation,curve-fitting,data-fitting

As am304, with such a data set I would strongly suggest to fit you data initially in the Y-X referential, then only calculate the equivalent in the X-Y referential if you really need the polynomial coefficients this way. One very useful function (I use it extensively) in the curvefit toolbox...

You can put your records into a data.frame and then split by the cateogies and then run the correlation for each of the categories. sapply( split(data.frame(var1, var2), categories), function(x) cor(x[[1]],x[[2]]) ) This can look prettier with the dplyr library library(dplyr) data.frame(var1=var1, var2=var2, categories=categories) %>% group_by(categories) %>% summarize(cor= cor(var1, var2)) ...

Suppose you want to subset the genes based on the prefix and do the cor on each subset res <- lapply(split(1:nrow(mat1), gsub("_.*", "", rownames(mat1))), function(i) cor(mat1[i,])) data set.seed(29) mat1 <- matrix(sample(0:80, 15*3, replace=TRUE), ncol=3, dimnames=list(paste(rep(c('V18', 'V19'), c(8,7)), LETTERS[1:15], sep="_"), paste0("AC", 1:3))) ...

Regarding the last portion of your question: rcorr binds matrices sample1 and sample2 by columns and uses the combined matrix to calculate rank correlation coefficients. If you gave different names to the genes in sample1 and sample2, e.g.: colnames(sample1) <- sprintf('sample1.%s',colnames(sample1)) colnames(sample2) <- sprintf('sample2.%s',colnames(sample2)) you would see that you have...

matlab,line,data-modeling,correlation,data-fitting

It would be easier to diagnose with a sample dataset. At a guess, the problem is that your first line should be: maxx = max(X); minx = min(X); The way you had it minx=min(Y) distorts your fitx and fity values Edit: Thank you for submitting the sample data. What you...

python,pandas,group-by,correlation

You pretty much figured out all the pieces, just need to combine them: In [441]: df.groupby('ID')[['Val1','Val2']].corr() Out[441]: Val1 Val2 ID A Val1 1.000000 0.500000 Val2 0.500000 1.000000 B Val1 1.000000 0.385727 Val2 0.385727 1.000000 In your case, printing out a 2x2 for each ID is excessively verbose. I don't see...

javascript,jquery,triggers,correlation,prefix

Try This: $("[class$='-dummy']").on('click', function () { var btnClass = $(this).attr('class').match(/([\w-]+-dummy)+/)[0].replace('-dummy', ''); alert(btnClass); $("." + btnClass).trigger("click"); }); Demo: https://jsfiddle.net/tusharj/gc3d5yy5/ Hope this helps...

This is taken from the seqanswers thread about the same thing. Suppose your working directory contains a directory of your cuffdiff output, say cuffdiff_out. Then you can run this to find the correlation of the FPKM values. cuff_data <- readCufflinks("cuffdiff_out/") m <- fpkmMatrix(genes(cuff_data)) cor(m[, 1], m[, 2]) ...

you can just call cor on your table like: cor(dataframe) ...

regex,jmeter,session-cookies,correlation

If you are already using HTTP Cookie Manager you don't need to correlate the cookie via Regular Expression Extractor. If you need to get the cookie value for some reason - you can just add the following line to user.properties file (lives under /bin folder of your JMeter installation) CookieManager.save.cookies=true...

You can use outer to perform the test between all pairs of columns. Here X and Y are data frames expanded from df, consisting of 8 columns each. outer(df[, c(1,3)], df[, c(2,4,5,6)], function(X, Y){ mapply(function(...) cor.test(..., na.action = "na.exclude")$estimate, X, Y) }) You even get output on the same form...

matlab,correlation,matrix-inverse,determinants,fminsearch

How about sqrt(det(Gamma)) for the sqrt-determinant and inv(Gamma) for inverse? But if you do not want to implement it yourself you can look at yulewalkerarestimator UPD: For estimation of autocovariance matrix use xcov also, this topic is a bit more explained here...

I think you want a correlation matrix, try this: cor(yourdataframe) EDIT: I think I misunderstood, if you want to correlate Age with every other column try this: apply(yourdataframe, 2, cor, x = AGE) ...

Sorry, I noticed you said you looked at the documentation in your question. My fault for not seeing that. As I understand it, with X and Y being your original data matrices, A and B are the sets of coefficients that perform a change of basis to maximally correlate your...

r,time-series,correlation,bootstrapping

Probably all what you need is to use acf function in stats package. It will do correlations for many lags as you prefer. library(stats) # for the use of "acf" function health.d <- health.d[!duplicated(health.d),] health.d$lnincome <- log(health.d$Income + 1) health.d <- health.d[(health.d$sex == 1 & health.d$married == 0),] #First Difference...

r,correlation,continuous,categorical-data

Since y is not dichotomous, it doesn't make sense to use biserial(). From the documentation: The biserial correlation is between a continuous y variable and a dichotmous x variable, which is assumed to have resulted from a dichotomized normal variable. Instead use polyserial(), which allows more than 2 levels. polyserial()...

java,performance,ocr,correlation,coefficients

Nowadays, it's hard to find a CPU with a single core (even in mobiles). As the tasks are nicely separated, you can do it with a few lines only. So I'd go for it, though the gain is limited. In case you really mean cross-correlation, then a transform like DFT...

batch-file,correlation,esper,nesper

There is batch expression window that can compare events and release batches. It is described in [1]. [1] http://esper.codehaus.org/esper-5.0.0/doc/reference/en-US/html_single/index.html#view-win-exprbatch