Date function and Selecting top N queries in DocumentDB
Some thoughts/suggestions below: Please take a look at this idea on how to store and query dates in DocumentDB (as epoch timestamps). http://azure.microsoft.com/blog/2014/11/19/working-with-dates-in-azure-documentdb-4/ To get top N results, set FeedOptions.MaxItemCount and read only one page, i.e., call ExecuteNextAsync() once. See https://msdn.microsoft.com/en-US/library/microsoft.azure.documents.linq.documentqueryable.asdocumentquery.aspx for an example. We're planning to add TOP to...
How does nesting structures affect DocumentDB query performance?
database,data-structures,nosql,query-performance,azure-documentdb
There isn't a single "correct" answer to this. Choosing whether to represent relationships as a single embedded document (aka de-normalizing) or as references as you would in a RDBMS (aka normalizing) depends highly on your use-case / scenario. Typically you will want to de-normalize for read-heavy scenarios and normalize for...
ConnectionPoolTimeoutException when reaching the max pool size while inserting documents
java,apache-httpclient-4.x,azure-documentdb
You are seeing ConnectionPoolTimeoutException because the DocumentClient does not automatically close the stream when createXXXXXXXX() is called. This behavior was intended to support streaming blob attachments for createAttachment(). To close the stream, you can call close() to close the stream or .getResource() to return the resource and then close the...
How can I perform an UPSERT using Azure DocumentDB?
Yes, a store procedure would work great for upsert. There are even code samples available on DocumentDB's Github: Upsert (Optimized for Insert): https://github.com/Azure/azure-documentdb-js/blob/master/server-side/samples/stored-procedures/upsert.js Upsert (Optimized for Replace): https://github.com/Azure/azure-documentdb-js/blob/master/server-side/samples/stored-procedures/upsertOptimizedForReplace.js Bulk Import / Upsert: https://github.com/Azure/azure-documentdb-hadoop/blob/master/src/BulkImportScript.js ...
Cannot read property '_self' of undefined in DocumentDB
node.js,azure,azure-documentdb
Here are a few tips: Looking at the exception you are getting: client.createCollection(database._self,collectionDefinition, function(err,coll ^ TypeError: Cannot read property '_self' of undefined database is undefined because you are getting an error passed in to the callback. Looks like you the error message you are receiving is: { code: 401, body:...
How to create a auto incremented column in Documentdb
c#,auto-increment,azure-documentdb
DocumentDB doesn't include auto-increment functionality out of the box. As Gaurav mentioned in Do we have Identity Column in DocumentDB, the id field is special in that if a value isn't provided by the application - DocumentDB will assign a GUID automatically. If you need auto-increment functionality - one potential...
DocumentDB stored procedure blocked
I too was experiencing this so I came up with a workaround. Whenever I get this out of resources response, I actually remove and recreate the stored procedure. I created a library that takes care of this automatically based upon the response. It also takes care of automatic delay from...
DocumentDb “where” clause with mathematical expression
Here's the gist... Today, all JSON properties in DocumentDB get automatically indexed by a Hash index; which means queries with equality operators (e.g. WHERE d.name= "george") are extremely fast. On the other hand, range queries (e.g. WHERE d.lastUpdateTime > 14) require a range index to operate efficiently. Without a range...
Is it possible to pause a Azure DocumentDB?
There is not current way of pausing the service. Anyway, as almost all things in Azure, even when paused you would still be paying for it.
DocumentDB Query Requires Unexpected High RUs
Queries using != will require a scan, since all index entries have to be looked up to eliminate the values that are not equal to 5. This will lead to a high RUs. Whenever possible, please try rewriting the query as an equality or a range query. If the number...
Azure DocumentDB Query Performance: ID vs SelfLink
There will be no perceptible performance difference between SelfLinks and IDs. You can use IDs directly within your application in place of SelfLinks.
How to find nearest points using latitude and longitude from a DocumentDB collection?
You'll need a geospatial function like ST_NEAR or ST_DISTANCE. This is not currently available - you can check the status of the feature here. Short-term, especially if have < 1000 documents, you might be able to use an existing geospatial library e.g., System.Spatial and perform processing client-side. Alternatively, you can...
Network timeout using node.js client at about 5 seconds
The stored procedure above queues document creates in a while loop until the API returns false. Keep in mind that createDocument() is an asynchronous method. The boolean returned represents whether it is time to wrap up execution right there and then. The return value isn't "smart" enough to estimate and...
Best practice regarding Azure DocumentDB auth key?
security,azure,azure-documentdb
You are doing the right thing by storing your connection strings in your Azure Website environment so they are not in your web.config and therefore not in your source control system. And your local development process works well for you only because you are using the emulator which doesn't require...
Want to understand async
c#,async-await,asp.net-web-api2,azure-documentdb
you can only use await inside a method if that method is async and async methods need to return Task, Task<T> or void although void returning async methods are reserved for event handlers because the exceptions thrown within them are swallowed and you cannot await their completion or chain subsequent...
How to get list of all collections in a DocumentDB document?
The short answer is: you can't retrieve a list of collections inside a document. You can, however, retrieve a list of collection inside a database. Here's a look at the DocumentDB resource model: Looking at the DocumentDB resource model - databases contain collections, which in turn contain stored procedures, triggers,...
Fire trigger from DocumentDB stored procedure
stored-procedures,triggers,azure-documentdb
Triggers can not be called from the server-side SDK (e.g. from inside another trigger or sproc).
Currently Using MySQL, Looking at DocumentDB
mysql,mongodb,azure-documentdb
Based on the provided information you have the choice of several schema designs (with JSON as examples). I've made some assumptions, such as that more than one tank can be on one map and map data is only linked to a single map. You have to tweak it for your...
Determining size of a JSON document stored in DocumentDB
Yes - you can calculate the size of the document query response, so that all the system properties (e.g. _rid, _ts) are included. You will want to use UTF-8 encoding to get the correct size. You will also want to factor in an additional ~10% for indexing storage costs....
search by substring in documentDB
c#,search,like-operator,azure-documentdb
Wildcards like SQL's LIKE '% %' has not been implemented in DocumentDB yet. Please voice your opinion and vote for this feature on DocumentDB's feedback forum....
DocumentDB TransientFaultHandling Best Practice
You can find sample code using the TransientFaultHandling Nuget package in the Github repo for the DocumentDB Data Migration Tool: https://github.com/Azure/azure-documentdb-datamigrationtool/blob/master/DocumentDb/Microsoft.DataTransfer.DocumentDb.FunctionalTests/DocumentDbHelper.cs Which looks something like this: private static IReliableReadWriteDocumentClient CreateClient(IDocumentDbConnectionSettings connectionSettings) { return new DocumentClient(new Uri(connectionSettings.AccountEndpoint), connectionSettings.AccountKey)...
DocumentDB - query result order
At the moment, you'll need to sort the collection on the client or stored procedures (although I'd I'd recommend doing so on the client since stored procedures have bounded execution). To answer your questions: 1) I don't believe there is a guaranteed default ordering 2) The Stored Procedure would result...
How to get DocumentDB Disk Usage without Azure Portal
You can use client.ReadDocumentCollectionAsync to get this information, e.g., ResourceResponse<DocumentCollection> response = await client.ReadDocumentCollectionAsync(collectionLink); Console.WriteLine(response.CollectionSizeQuota); Console.WriteLine(response.CollectionSizeUsage); See https://msdn.microsoft.com/en-us/library/microsoft.azure.documents.client.documentclient.readdocumentcollectionasync.aspx and https://msdn.microsoft.com/en-us/library/dn799209.aspx for more details. The equivalent in the REST API is the...
createUserDefinedFunction : if already exists?
You can check for existing UDFs by running query using queryUserDefinedFunctions. Example: List<UserDefinedFunction> udfs = client.queryUserDefinedFunctions( myCollection.getSelfLink(), new SqlQuerySpec("SELECT * FROM root r WHERE [email protected]", new SqlParameterCollection(new SqlParameter("@id", myUdfId))), null).getQueryIterable().toList(); if (udfs.size() > 0) { // Found UDF. } ...
Inserting Sub-Array into Azure's Document DB
asp.net,asp.net-mvc,azure-documentdb
Set organization.Videos to a non-null value. Document db simply preserves what you stored. Apparently, you previously stored null.
Microsoft Azure DocumentDB vs Azure Table Storage
azure-table-storage,azure-documentdb
Both are NOSql technologies, but they are massively different. Azure Tables is a simple Key/Value store and does not support complex functionality like complex queries (most of them will require a full partition/table scan anyway, which will kill your performance and your cost savings), custom indexing (indexing is based on...
DocumentDb -> Is Index a reserved word?
No, index is not reserved. You should be able to use this.
DocumentDB queries with arrays
You can combine the JOIN operator , which is used to form cross products with nested array elements, with the IN operator. SELECT docs FROM docs JOIN tags IN docs.tags WHERE tags IN ("B", "C") Note that because you are creating a cross product, that you will get a result...
documentdb AND get record count
Until the implementation of the "count" keyword, you should do your query in a store procedure on the server. Take care to not get all columns/properties in your query if you want only a count. Select only the id like dc.CreateDocumentQuery(update.SelfLink, "SELECT c.id FROM c")
Document Db Usage on Azure
Though DocumentDB allows you to store files (they are stored as attachments), I would not recommend using it. Here are my reasons: In the current version, the maximum size of an attachment is 2 MB. You can't really stream attachments. You would need to first read the attachment contents in...
Unable to use result of Async DocumentDB operation CreateDocumentAsync
c#,azure,async-await,task-parallel-library,azure-documentdb
That's because you're not awaiting an asynchronous method: This: var doc = _applicationResource.Save(data); Needs to be: var doc = await _applicationResource.Save(data); Your method should look as follows: [HttpPost] [Route("")] public async Task<IHttpActionResult> CreateNewApplication(dynamic data) { if (data == null) { return BadRequest("data was empty"); } try { var doc =...
DocumentDb and how to create folder?
You could define a sequential naming convention and create a range index on the collection indexing policy. In this way, if you need to retrieve a range of documents, you can do it in this way, which will leverage the indexing capabilities of docdb efficiently. As a recommendation, you can...
Azure DocumentDB Multi-Tenant Architecture
azure,multi-tenant,multi-tenancy,azure-documentdb
There's no such thing as a one-size-fits-all answer when it comes to partitioning / sharding tenant data. Generally, how you partition data depends on your application's query patterns as well as the resource requirements per tenant (in terms of both storage and throughput). Just keep in mind that collections are...
Getting “A task was cancelled” while trying to create a document in DocumentDB
The issue was caused by not getting async calls right. I was calling an async method i.e. CreateDocumentAsync() incorrectly. I posted a separate question on async programming and the answers I got helped me troubleshoot the issue. Here's that post: Want to understand async...
Azure Document Db Worker Role
azure,azure-worker-roles,azure-documentdb
The solution for me to get rid of the security exception and null reference exception was to disable intellitrace. Once I did that, I was able to deploy and attach debugger and see everything working. Not sure what is between the null in intellitrace and the DocumentClient, but hopefully it's...
Exception when adding range index to Azure DocumentDB collection
You have to include "/" as an additional IncludedPath like: var collection = new DocumentCollection { id = "myCollectionID" }; collection.IndexingPolicy.IncludedPaths.Add(new IndexingPath { IndexType = IndexType.Range, Path = "/\"TimeStamp\"/\"Epoch\"/?", NumericPrecision = 7 }); collection.IndexingPolicy.IncludedPaths.Add(new IndexingPath { IndexType = IndexType.Hash, Path = "/" }); this._client.CreateDocumentCollectionAsync(database.SelfLink, collection); Alternatively, if you want to...
Query in DocumentDB for Searching
So with a slight modification to your schema my document now looks like this - {id : "123", name : "sai", marks : [ {Subject : "English", Grade : 50}, {Subject : "Maths", Grade : 60}, {Subject : "Science", Grade : 90} ] } Then a custom IndexingPolicy IncludePath has...
How to insert into documentDB from Excel file containing 5000 records?
c#,database,excel,bulkinsert,azure-documentdb
Update: As of 4/8/15, DocumentDB has released a data import tool, which supports JSON files, MongoDB, SQL Server, and CSV files. You can find it here: http://www.microsoft.com/en-us/download/details.aspx?id=46436 In this case, you can save your Excel file as a CSV and then bulk-import records using the data import tool. Original Answer:...
C# Azure DocumentDB Authorization Key Permissions
The client is constructed with the authorization key and has no more information about it than you at that point. It's just a string. You may be able to leverage the DocumentClient.ReadPermissionAsync method to get permissions of a resource. I have not tried this first hand, but may be good...
Azure Search scalability
azure,azure-documentdb,azure-search
Interesting that you're using Azure Search as your primary storage, as it's not built to be a database engine. The storage is specifically for search content (type typical pattern is to use Azure Search in conjunction with a database engine, such as SQL Database or DocumentDB, for example), using results...
NodeJS, DocumentDB (Via Docooment) Bulk Insert
node.js,azure-documentdb,docooment
I'd recommend going with one of the following options: DocumentDB's Data Migration Tool The easiest way to bulk import data from a CSV in to DocumentDB is to use DocumentDB's migration tool. You can find details on how to use it here. Programatically via Stored Procedures (Database-Side Scripting) If you'd...
How to import bulk data in documentDB from excel?
c#,excel,insert,bulkinsert,azure-documentdb
Update (4/8/15): DocumentDB just released a data import tool, which supports JSON files, MongoDB, SQL Server, and CSV files. You can find it here: http://www.microsoft.com/en-us/download/details.aspx?id=46436 Original Answer: DocumentDB doesn't support bulk importing from excel files just yet... However, you can leverage DocumentDB's store procedures to make bulk import a bit...
How to get DocDbWrapperScript.js?
You can find the DocDbWrapperScript.js on the DocumentDB JS Github here: https://raw.githubusercontent.com/Azure/azure-documentdb-js/master/server-side/DocDbWrapperScript.js...
Not getting Downloaded - “Azure DocumentDB Data Migration Tool”
mongodb,azure,migration,azure-documentdb
Are you downloading from http://www.microsoft.com/en-us/download/details.aspx?id=46436? You will need to temporarily enable popups for the download to initiate.
Why are my primary and secondary DocumentDB keys not showing in the Azure portal?
azure,primary-key,azure-documentdb
The account can take several minutes to provision, during which time the keys will not be available (since the account has finished creating). If, in the portal, you choose to Browse | DocumentDB accounts, you will see the status (creating, updating, online). Once your account is in an online state,...
Should my Azure DocumentDB document classes inherit from Microsoft.Azure.Documents.Document?
This behavior is attributed to how JSON.NET deals with properties on dynamic objects. It effectively ignores them unless you decorate them with the JsonProperty attribute. You can either work with plain POCO or you can extend from Resource (shown below), which is a static object that Document itself extends. public...
DocumentDB Java SDK behind a proxy
Following pull request merge, if you want to use a proxy you need to get a release version greater than 1.0.1. As the release is not available at this moment, you could build github source and use the sdk snapshot version. Then you just need to setup your proxy through...
Exporting data from Azure DocumentDB
There is no built-in export option in the database service itself. However, the DocumentDB Data Migration Tool, published by the DocumentDB team, is available and provides both import and export capabilities (export is to JSON). Everything is documented on the DocumentDB area of the Azure site, specifically here. This isn't...
How to query in DocumentDB based on inner json object value?
You can use DocumentDB's JOIN to create a cross product on documents with array elements. For example, the following query creates a cross-product on documents with it's students property to query on the students.name: select doc.id from doc join students in doc.students where students.name = 'sunny' returns the following dataset:...
Unable to return type Microsoft.Azure.Documents.Document from web api
web-api,asp.net-web-api2,azure-documentdb
First, when you await for a function that returns a Task<T>, you get a value of type T. That's why you put the await word in front of the function call. Second, what makes Web API very easy to use is that you don't need to work on the HTTP...
Can Azure Search integrate seamlessly with Azure DocumentDB?
azure,azure-documentdb,azure-search
I can not comment below the current thread, so I will add a new reply. I am a Program Manager with Azure Search and I can confirm Daron's comments about this being a top request. There is also a fair amount of voting for it from our UserVoice page (http://feedback.azure.com/forums/263029-azure-search/suggestions/6328680-auto-indexing-of-docdb)....
How is the pricing setup for Azure DocumentDB?
I had a hard time getting information on how the DocumentDB pricing works. After lots of emails and support requests, I found that the pricing is based on per collection basis. So, if I have a DB with say two collections in it and each with a performance tier of...
RequestRateTooLargeException when trying to insert 200 documents in DocumentDB
node.js,azure,azure-documentdb
This happens when your rate of access exceeds your allowed quota. DocumentDB has a max of 2,000 request units per second per collection. An insert is charged (based on the size of your document, and number of indexed terms). If your insert costs 20 request units, you would be able...
DocumentDB find deep key:value
You must reference the collection from the FROM clause. Note: Since you are issuing queries directly to a collection, you can use any arbitrary value for the name. However, you must re-use whatever value you chose in the other clauses because it serves as a reference point for projections and...
Azure DocumentDB - The MAC signature found in the HTTP request is not the same as the computed signature
c#,.net,azure,authorization,azure-documentdb
We have confirmed this to be an issue isolated to the North Europe region. We are applying a hotfix to known accounts effected and will be deploying a fix shortly. If you are NOT in North Europe and are experiencing this issue, or if you continue to see if within...
Querying DocumentDB using a propery by its name
SQL is the best option since LINQ requires type-binding. You can alternatively query using the type of Dictionary, then convert to Document (either by reading as an ExpandoObject, or by converting using JsonConvert). But SQL is going to be relatively cleaner: var query = client.CreateDocumentQuery<Dictionary<string, object>>(collectionLink) .Where(c => (string)c[propertyName] ==...
DocumentDb - get Json representation of Document
c#,azure,nosql,azure-documentdb
You can use JavaScript's delete operator to strip out DocumentDB's native properties. Take a look at this thread: How to remove a property from a JavaScript object Something like this should work: delete doc._rid; delete doc._ts; delete doc._etag; getContext().getResponse().setBody(doc); ...
Is there a way of using JavaScript libraries in Azure DocumentDB server code?
javascript,azure,stored-procedures,nosql,azure-documentdb
Yes, you can embed common functions within stored procedure code - granted it fits inside the 256kb limit for a stored procedure. For your example, underscore is only 6kb minified and can be embedded inside a sproc. Note: keep in mind server-side scripts are resource governed (you may hit request...
DocumentClient.CreateDatabaseQuery - No definition exists
I believe you are missing the using directive for the DocumentDB Linq provider (Microsoft.Azure.Documents.Linq). Please make sure you have the following directives to the top of your .cs file: using Microsoft.Azure.Documents; using Microsoft.Azure.Documents.Client; using Microsoft.Azure.Documents.Linq; ...
Converting created document result to POCO
(Copying over Andrew Davis's answer, from the DocumentDB MSDN forums, for the stackoverflow community) The simplest way would be to have your Employee class inherit from Document, and then cast result.Resource to Employee. If you don't want to inherit from Document, you could also define an explicit cast between Document...
Associate DocumentDB's document id to class property
if you decorate your unique id using JsonProperty attributes then you can use your fields as the unique id, instead of having another id field on every document. and example is; [JsonProperty(PropertyName="id")] public string ProductId { get; set; } ...
WHERE clause on an array in Azure DocumentDb
You should take advantage of DocumentDB's JOIN clause, which operates a bit differently than JOIN in a RDBMs (since DocumentDB deals w/ denormlaized data model of schema-free documents). To put it simply, you can think of DocumentDB's JOIN as self-joins which can be used to form cross-products between nested JSON...
Cross-links/updates between documents in NoSQL database
mongodb,couchdb,azure-documentdb
Most NoSQL databases do not support JOINs between multiple records - so you are left with a few different ways to model relationships. Normalizing As you mentioned, you could leave a soft-link to another document... and then resolve the reference with a follow-up query. Generally, you'd normalize data that you'd...
DocumentDb query with both ordering and paging on the server side, possible?
c#,azure,asp.net-web-api,azure-documentdb
It's not supported out of the box, but you can implement a stored procedure that does this. The msft product group has supplied some code samples here: https://code.msdn.microsoft.com/windowsazure/Azure-DocumentDB-NET-Code-6b3da8af/sourcecode?fileId=132409&pathId=34503205 Look under server side script, JS folder, you'll see an "orderby" script that does this. Adjust to your needs and try it....
Escape symbol in documentdb query
DocumentDB uses JSON notation for escaping property names. Try: SELECT * FROM c.inside["abs-humid"]...
Connect to Azure DocumentDB from Android
android,azure,azure-documentdb
Okay, this was a nightmare to solve. Nine hours later... -_-'. Long story short, here is some code that will actually work successfully. This code isn't perfect, and is dependent on Retrofit First, this is an example "Service" interface in Retrofit: import retrofit.Callback; import retrofit.http.Body; import retrofit.http.Header; import retrofit.http.POST; public...