KDB Query with upper in where clause
Try "upper[cola]=`ABC". Round brackets are not used for function arguments, they are used for precedence etc. Square brackets are used in function arguments. The reason round brackets worked in the second case is because the parser simply ignored them. Would be the same as "select upper cola from table" In...
Pivot table in kdb+/q
This is a self-contained version that's easier to use: tt:1000#0!trades_agg piv:{[t;k;p;v] / controls new columns names f:{[v;P]`${raze " " sv x} each string raze P[;0],'/:v,/:\:P[;1]}; v:(),v; k:(),k; p:(),p; / make sure args are lists G:group flip k!(t:.Q.v t)k; F:group flip p!t p; key[G]!flip(C:f[v]P:flip value flip key F)!raze {[i;j;k;x;y] a:count[x]#x 0N;...
Installing qPython in Win7
Looks like this is the culprit: g:/r/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: cannot find -lpython27 And yes I would say you need to install python-dev to get the python27 libraries for linking to....
kdb+ historical database partitioning design - daily data on thousands of symbols
As @GilbertLeBlanc says, 50m is certainly not much and if that was the expected size it would simply be best (in this use-case) to splay the table (i.e. no partioning at all) along with using attributes on the columns that you will use for filtering (p# org# on ticker since...
KDB: apply dyadic function across two lists
You have asked for the cross product of your argument lists, so the correct answer is raze F ./: xList cross yList ...
(kdb+/q) append to dictionary
I think it doesn't like the compound assignment groupBy,:{...} Try groupBy:groupBy,{...} The difference is that in the first case it's trying to directly alter the data in memory without creating a copy, whereas in the second case it is creating a copy of the data in memory and then re-assigning...
Downloading FTP files in Q (KDB)
use a system call to curl without a file destination -- its default destination is stdout, so the file contents will be returned to q as the return value of system data:system"curl ftp://wherever/whatever"...
kdb+: replace null integer with 0
Using the fill operator (^) Example Table: q) tbl:flip`a`b!(2;0N)#10?0N 0N 0N,til 3 a b --- 0 2 1 1 1 1 1 1 Fill nulls in all columns with 0: q)0^tbl a b --- 0 2 1 1 1 1 0 1 1 0 Fill nulls only in selective columns...
How to push tables through socket
Let's understand the simple process of doing this: We have one server 'S' and one client 'C'. When 'C' calls .u.sub function, that function code connects to 'S' using its host and port and call a specific function on 'S' (lets say 'request') with subscription parameters. On getting this request,...
Dictionary assignment in defined namespace failing
From code.kx: A context is actually a sorted dictionary whose domain is a list of symbols with the names of the entities defined in the context If you change your code to the following, it will work as expected: q).foo:(`symbol$())!() q).foo.bar:`a`b`c!1 2 3 q).foo.bar a| 1 b| 2 c| 3...
KDB / Q Resetting random elements on newly created list (numerical vs symbol/char)
To make it generic would be a matter of using the booleans to index rather than using them to multiply (as you currently are). However, for non numerical lists you'd need to reset values to nulls rather than to zero. These nulls would have to be the correct null corresponding...
Aggregate rdb and hdb in kdb
Say this function lives on the an independant q instance and you want all columns, for a day d + today for sym s... notice the column rearrangement. mergeFun:{[d;s] rdb:hopen`::5011; hdb:hopen`::5012; t:hdb({[d;s] `time`sym xcols delete date from select from table where date=d, sym=s};d;s),rdb({[s] select from table where sym=s};s); hclose each...
Saving a kdb+ “unmappable” table to disk
You can manually save these tables using the following steps: q)t:([] a:1 2 3; f:(1 2.0;(3 4.;5 6.); 7. 8.)) q)t a f --------- 1 1 2 2 3 4 5 6 3 7 8 q)`:t/ set t k){$[@x;.[x;();:;y];-19!((,y),x)]} 'type q.q)) q.q))\ q) Fails as nested type. q)`:t/ set select...
Updating point-in-time (bitemporal) table
You can use 'upsert' in following way: reference: http://code.kx.com/wiki/Reference/upsert Step 1. I am adding status column in 'new' table (otherwise it will be null for new rows in final table) q) new:update stamp:`timestamp$.z.z from new Step 2. Make those columns of 'new' as primary keys whose modification is criteria for...
How do I use the k() function to receive data in kdb/q
The k() function will block until it receives data from the remote socket. So if your function looked like this: compute_long_running_function:{[] r:til 1000; neg[.z.w] r } the function result will be sent over the loopback connection and be retrieved by conn.k() The alternative is you just do: results=conn.k("compute_long_running_function[]"); which will...
How to append a single element to a list keyed in a dict
You can do dict[`b],:8 but it will automatically overwrite the dictionary. A more robust way would be @[dict;`b;,;8] or @[`dict;`b;,;8] where the latter automatically overwrites the dictionary, while the former creates a modified copy of the dictionary (in case you don't want it to overwrite the original yet)...
Construct q command to get metadata for all tables
q)trade:([] sym: 10?`4; time:10?.z.t; prx:10?100f; sz:10?10000); q)quote:([] sym: 10?`4; time:10?.z.t; bPrx:10?100f; aPrx:10?100f; bSz:10?10000; aSz:10?10000); q)testTable:update `s#a from ([] a:til 10; b: 10?`3; c:10?.z.p); q)raze {update table:x from 0!meta x}'[tables[]] c t f a table -------------------- sym s quote time t quote bPrx f quote aPrx f quote bSz j quote...
kdb+/q: Prevent initialised tables from returning a comma
Array access with [0] or first is the correct way if you want an "atomic" variable. myvar:first exec name from test; ...
KDB+ / Q Table creation with dynamic number of columns
A table is a dictionary from a dictionary of symbol keys to a list of equal length vectors. Dynamically create the column names using "string til". q){ `id xkey update id:i from flip (`$"vals",/:string til y)!(y#x)?\:`8 }[3;4] id| vals0 vals1 vals2 vals3 --| ----------------------------------- 0 | lkamnmmm nfnbfmkm kiblpojl onhlghno...
In q, how to add a symbol to a symbol
In certain scenarios like this one you can use sv (http://code.kx.com/wiki/Reference/sv) q)` sv `:c:/q,a,`table `:c:/q/partofurl/table Failing that, you can always string and append q)`$":c:/q/",string[a],"/table" `:c:/q/partofurl/table ...
How to get positions of element matches in a list in kdb?
I think you should use ? for this: q)2 5 3 1?3 5 2 1 http://code.kx.com/wiki/Reference/QuestionSymbol#find...
kdb: inter-process bulk insert
use this: q)h (insert; `TEST;DATA) Syntax for calling function on a service from other service using handle: h(function;func_parameters) ...
How to use .year inside function in KDB?
As per this page on code.kx, this behavior is a quirk of q. To get around this, you can use the cast function. q)f:{[x] :`year$x} q)f[myDate] 2014i ...
kdb+ conditional insert: only insert when column value doesn't exist
First thing is to have proper attributes (sort,group) on the target column which will make function faster. Now there are 2 scenarios I can think of: a) Table is keyed and target column is keyed column : In this case normal insert will work in way like your conditional insert....
How to get the total size of a table in kdb+?
-22! returns the size in bytes of in-memory objects. e.g. q)t:([] a:til 1000) q)-22!t 8031 q)/ 1000 longs = 1000*8 bytes + a small header q)t:([] a:til 2000) q)-22!t 16031 If you are interested in how memory management in kdb works I recommend this tutorial: http://www.timestored.com/kdb-guides/memory-management (Disclaimer: I wrote it.)...
kdb q: '{ error because of newline?
A space before the last parenthesis. CSVsave:{[filename;table] filename: $[-11h = type filename;filename;`$":", filename]; @[hdel;filename;()]; h: hopen filename; (neg h) csv 0: table; hclose h; }; I'd also suggest trying a kdb IDE. Rather than having to continuously save load. e.g. qStudio ...
Repeat last statement in the Q shell
On Linux or OS X you should use rlwrap. It is included in most Linux package repositories. On OS X I installed rlwrap via MacPorts, which requires Xcode to be installed. On Windows the q console comes with this functionality and you don't need to install anything....
How to work with date and time in KDB
to remove the date, you can use the cast operator, $. To reference only the time, you can prefix $ with `time as shown below. q).z.z 2015.02.23T14:10:33.523 q)`time$.z.z 14:10:30.731 q)t:([]ts:10#.z.N;ti:.z.t-til 10) q)exec `time$ts-ti from t 00:00:00.000 00:00:00.001 00:00:00.002 00:00:00.003 00:00:00.004 00:00:00.005.. You can see more examples here. http://code.kx.com/wiki/Reference/DollarSign...
KDB+ / Q Better webinterface
Have you tried doth package by simon? This is probably not even close to what you describe, but it could be a starting point to implement something custom.
Kdb+/q: Looping over initialised list
What you're doing may work but it's far from best practice. Loops and indices are not the way to go. What you're looking for is essentially test:([] name:`symbol$(); balance:`int$()); insert[`test;(`John;1001)]; insert[`test;(`Jane;2002)]; q)select sum f[balance] from test balance ------- 30.03 ...
(kdb+/q) Pivot table: date in rows, symbols in columns, last price as values
The function piv requires the key and pivot arguments to be a list. So modifying your naive call as follows gives us a result: q) piv[`q;(),`date;(),`sym;`price;f;g] date | obhmprice oijbprice mkjkprice nihdprice ldegprice mbgnprice jmmip.. ----------| -----------------------------------------------------------------.. 2009.01.08| 72.35531 28.9323 23.88535 12.21371 2.417089 49.45298 98.14.. 2009.01.07| 83.59946 6.036849 21.47751 78.8127...
How can I start kdb instance in Windows?
q trade.q -p 5001 This video tutorial demonstrates downloading and installing kdb on windows: http://www.timestored.com/kdb-guides/getting-started-kdb...
KDB+ / Q table creation with foreign key
database,table,foreign-keys,q,kdb
You can use the same syntax on the column values list: q)T3:1!flip ((`id`f1 )!((1 2 3 4 5);(`T1$2 2 2 4 4))) q)T3~T2 1b Update: Again for this case we can use the same syntax on the list - q)A:enlist`T1$1 2 3 4 5 q)meta flip (`id`v1`v2)!(B,A) c | t...
Running an R bat file that connects to KDB to query data
I think you should point it to 64bit version of R. "C:\Program Files\R\R-3.1.2\bin\x64\R.exe" CMD BATCH --vanilla --slave "C:\Users\Desktop\NEW TASK Executables\execute1.R" ...
kdb+: use string as variable name
You could use "set" but it will create a global: q){(`$"test") set 1;test}[] 1 q)test 1 or (as noted by user2393012 in the comments): @[`.;`test;:;1] If you want to avoid globals you could use some sort of namespace/dictionary/mapping: q){d:()!();d[`$"test"]:1;d`test}[] 1 ...
kdb 2-way asof join (merge bid/ask tables)
You don't need aj to achieve what you want. uj combined with fills will do the job: b: 1!`time`sym`bs`bprc xcol bid / convert bid to a keyed table a: 1!`time`sym`as`aprc xcol ask / convert ask to a keyed table ba: `time xasc b uj a / join them and sort...
Use a function in the groupby clause of a functional select
1. Yes you can, just function should be allowed/compatible to group by clause. 'xbar' is allowed. Syntax is: (function;param1;parma2;...param n) Here is one ex. q)t:([]id:til 20;v:til 20) q)select by 5 xbar id from t q) // functional form q)?[`t;();(enlist `id)!enlist (xbar;5;`id);()] 2. Trick to get functional form of simple query...
No such host is known using AquaQ
Terry you were right actually. It turns out q requires this "/" in paths not "\". I had "\" in the path for the environment variables. Johnny Press answered this for me here
Why does KDB Ticker Plant need a log file?
It is just to restore the data mainly in case of RDB process crash which ensures system stability. You can assume it as a backing store or as a permanent storage. First let's understand the tickerplant architecture: Tickerplant takes data from a source (feed handler) , logs it in file...
q: `': The pipe is being closed`
@[loopSizes; 0; {0N!"error thrown, returning zero";}] Not sure why you were setting z to 0 in the 2nd argument of @ - just 0 suffices and gets passed to loopSizes. (so long as loopSizes is a monadic function). 0N! prints to console. Also see -1! to print to standard out....
timer doesn't work in qStudio
Two reasons: Show displays the value in the kdb console. There's an issue with how kdb handles remote calls involving slashes. As detailed here: http://www.timestored.com/qstudio/help/faq You should replace \t 500 with system "t 500"; and the timer will now be set. ...
How to select first record prior/after a given timestamp in KDB?
Select last record before the given timestamp: q)select from Price where timestamp<2014.04.14T09:30,stock=`GOOG,i=last i Select first record after the given timestamp: q)select from Price where timestamp>2014.04.14T09:30,stock=`GOOG,i=first i ...
How to remove element from list in KDB?
You can use except as well: q)x except 1#() a b --- 1 2 3 4 ...
How to join splayed table in KDB?
You need to map the splayed StockPrices table into memory. This can be done by using a select query: q)(`::6060)"aj[`sym`time;select from trade;quote]" / bad 'splay q)(`::6060)"aj[`sym`time;select from trade;select from quote]" / good sym time prx bid ask ------------------------------------------- aea 01:01:16.347 637.7554 866.0131 328.1476 aea 01:59:14.108 819.5301 115.053 208.1114 aea 02:42:44.724...
type char to type num mapping
You can use .Q.t. q).Q.t abs type `test "s" q).Q.t abs type 5i "i" Edit: Or even better just use .Q.ty This seems to return upper case for atoms and lower case for lists. q).Q.ty `test "S" q).Q.ty `test`test2 "s" ...
How to publish to KDB Ticker Plant from Java effectively
java,kdb,ticker,q-lang,exxeleron-q
There's not enough information in this message to give a fully qualified response, but having done the same with Java+KDB it really comes down to eliminating the possibilities. This is common sense, really, nothing super technical. make sure you're inserting asynchronously Verify it's exxeleron q java that is causing the...
kdb - qStudio multiple line execution
For running multi-line in QStudio, use semicolon for line ending. For ex, following 2 lines will not run together in qstudio using Ctrl+E: a:1 b:2 `type error. If you look in history tab, it sends command (a:1b:2) So use a:1; b:2; Now '\l' doesn't work with semicolon ending, it gives...
how to create bulk object with char arrays for KDB+
A string column in kdb is considered as a general type (list of list of char). q)type each flip ([]a:string`aa`bb;b:"ab") a| 0 <-- string column b| 10 <-- char column So your kdb table definition should've been // table_def:([] name: ()) Your previous definition is expecting the column values to...
Quicksort in Q/KDB+
Lets do it step by step . I assume you have basic understanding of Quick Sort algo. Also, there is one correction in code you mentioned which I have corrected in step 5. Example list: q)x: 1 0 5 4 3 Take a random element from list which will act...
How to join multiple tables
Another solution particular to your problem which is also little faster than your solution: a (,')/(b;c)...
kdb+: group by and sum over multiple columns
How about this? get select first time, avg colA, sum colB, first colC by sums colC<>prev colC from table ...
Use value to reference a variable in a function
value will always work on global scope. If you really need this, maybe make use of workspace variables, e.g. .a.b:1 ... I don't have a q instance to hand to test if that works but i'm almost sure it does....
General position find function in q
matrix,position,find,kdb,q-lang
How's this look? I think it should work for all two-dimensional lists, ragged or rectangular, and also for vectors. (I haven't worked out a version for arbitrary dimensions yet.) q)position:{{$[type x;enlist each where x;raze flip each flip(til count x;raze each .z.s each x)]}x=y} q)t 1 -1 1 -1 3 4...
Select value from mixed-type row in KDB
I almost don't want to give a solution here because mixing strings with integers in a single column is a terrible idea. Completely destroys performance and prevents any of the benefits that kdb offers. First and foremost, re-think your setup. If you insist on keeping it as is, you can...
Difference when you query from KDB HDB and KDB RDB
Off the top of my head: Partitioned HDB tables will have a "virtual" date column RDB tables (generally) won't have a `date column The virtual "i" column behaves differently for Partitioed HDB tables (http://code.kx.com/wiki/DotQ/DotQDotind) HDB tables (unless stored flat/serialised) are not immediately pulled entirely into memory, data is read on...
Passing Date Vector from R to KDB
I usually generate a list of dates and convert them into KDB readable by using sub function dates <- seq(as.Date("2014/11/01"), as.Date("2014/12/01"), by = "day") dates <- sub('-',".",dates);dates <- sub('-',".",dates); dates [1] "2014.11.01" "2014.11.02" "2014.11.03" "2014.11.04" "2014.11.05" "2014.11.06" "2014.11.07" "2014.11.08" "2014.11.09" "2014.11.10" [11] "2014.11.11" "2014.11.12" "2014.11.13" "2014.11.14" "2014.11.15" "2014.11.16" "2014.11.17" "2014.11.18"...
Cannot allocate memory for a column of compound floats on a partitioned table
I think this is all just a combination of the free-32bit memory limitation, the fact that your row counts are possibly large and the fact that (unavoidably) something must be pulled entirely into memory when retrieving data from a column, whether it is the column itself that gets entirely pulled...
Inconsistent behaviour with each and peach in KDB
That was not an inconsistent behavior of peach and each. First, Functions in kdb has implicit parameters as x,y,z if not specified any. So f:{x+y} is equivalent to f:{[x;y] x+y} But f:{[a;b] a+b} will not have x,y,z as implicit parameter For more details, see section Implicit Parameters in http://code.kx.com/wiki/JB:QforMortals/functions#Implicit_Parameters Peach...
kdb+ 32-bit, what are the consequences of the 4GB restriction?
database-connection,port,32bit-64bit,kdb
Correct, a kdb+ instance is a whole process in its own right, with a port number set at command line by yourself. You can't do peach over a number of handles: handles:hopen each someList; {x"do some work"} peach handles You'll get noupdate, so would seem you can't do this manually....
checking in list from kdb instance
Something like the following will work: ans:hdl({select count i by date,sym from trade where date=xxx, sym in x};res) @mollmerx is correct about having res defined locally and not on the remote process, although my query above is preferable to a functional select. ...
KdB+ - HTML5 websocket deserializer not working for Long data type
As rightly pointed out in the comment by user2393012, the serializer for KdB+ maps 0N(null) to the min value of the data type. Hence a null value of Long type gets serialized to -9223372036854775808. The deserializer is handling this integer min correctly for 16 bit and 32 bit values as...
KDB Excluding rows that have nulls
Either use simple list functions for easy to read code or functional form for speed. The functional form will be faster as it doesn't scan all the entire columns just those that pass each stage of the filter. q)t:flip `a`b`c`d`e!flip {5?(x;0N)} each til 10 q)t a b c d e...
KDB+ / Q Accessing root namespace from namespace
Context (namespace) is just a dictionary so you can use dictionary syntax. q) \d .seed q.seed) `.[`n] Reference: http://code.kx.com/wiki/JB:QforMortals/workspace_organization Check section: A Context is a Dictionary ...
How to convert character array into list in kdb
You are missing 'each' here. Use: select enlist each column_with_character_arr from table ex: q) t:([]v:("abc";"xyz")) q) select enlist each v from t Why length error on select (),v from t? Because comma(,) in select query separates different columns we want in output, i.e. select col1,col2 from tbl So in query...
Unable to use launched kdb+tick demo
That error in ticker.bat is the process attempting to execute an html file as code. Check what arguments are in ticker.bat then check that source file. I think you'll find you've download html instead of the raw text version.
KDB projectiopn multiple variable
Here is my understanding of your doubt: You have a list of inputs as L:(inputs1;inputs2;...) , where inputs1 is a list of (date;symbol;starttime;endtime) and you want to apply 'test' function on each input list in 'L'. For this, KDB provide 'dot' operator. Ex: q) f:{[a;b;c] a+b+c} q) f . (1...
kdb+/q: Apply iterative procedure with updated variable to a column
No need to use global vars for this - can use scan instead - see here. Example -- Generate a table - q)t:0N!([] time:5?.z.p; sym:5?`3; price:5?100f; size:5?10000) time sym price size ----------------------------------------------- 2002.04.04D18:06:07.889113280 cmj 29.07093 3994 2007.05.21D04:26:13.021438816 llm 7.347808 496 2010.10.30D10:15:14.157553088 obp 31.59526 1728 2005.11.01D21:15:54.022395584 dhc 34.10485 5486 2005.03.06D21:05:07.403334368 mho...
How to apply max function for each row in KDB?
You can try using | q)update x|0.5 from myTable ...
Using flip inside each not working in KDB
Remember that table is a list of dictionary. q)show each ([]a:1 2 3;b:4 5 6); a| 1 b| 4 a| 2 b| 5 a| 3 b| 6 http://code.kx.com/wiki/JB:QforMortals2/tables#Overview The values of each dictionary are atoms, not lists. So you can't flip a dictionary into a table that way, you have...
KDB: user-defined aggregation function on tables
You can use : q)select enlist MySum x2 from data Why 'select Mysum x2 from data' doesn't work? From KDB WIKI: "The following functions receive special treatment within select count,first,last,sum,prd,min,max,med,avg,wsum,wavg,var,dev,cov,cor " For Details, read: http://code.kx.com/wiki/Reference/select Section: Special functions within select ...
Executing jar from q
Solved it by writing a small Java program that executes the full java command in cmd: q -> runs small Java program 1 -> runs cmd commands to open the dependent Java class I think the problem was that the jar had external dependencies which also needed to be loaded,...
KDB+ / Q unique random values with variable for amount
This will give you num unique numbers between 0 and 9. q)(neg num)?10 ...
How to kill KDB queries in RDB or HDB?
You can set client query time out in your service: param: '-T ' reference: http://code.kx.com/wiki/JB:QforMortals2/commands_and_system_variables#Timeout_.28-T.29 From wiki: The timeout parameter (note upper case) is an int that specifies the number of milliseconds any call from a client will execute before it is timed out and terminated. The default value is...
how to efficiiently select first or last rows from a KDB table stored on disk
.Q.ind - it takes a table and (long!) indices into the table - and returns the appropriate rows http://code.kx.com/wiki/DotQ/DotQDotind