Some thoughts/suggestions below: Please take a look at this idea on how to store and query dates in DocumentDB (as epoch timestamps). http://azure.microsoft.com/blog/2014/11/19/working-with-dates-in-azure-documentdb-4/ To get top N results, set FeedOptions.MaxItemCount and read only one page, i.e., call ExecuteNextAsync() once. See https://msdn.microsoft.com/en-US/library/microsoft.azure.documents.linq.documentqueryable.asdocumentquery.aspx for an example. We're planning to add TOP to...
database,data-structures,nosql,query-performance,azure-documentdb
There isn't a single "correct" answer to this. Choosing whether to represent relationships as a single embedded document (aka de-normalizing) or as references as you would in a RDBMS (aka normalizing) depends highly on your use-case / scenario. Typically you will want to de-normalize for read-heavy scenarios and normalize for...
java,apache-httpclient-4.x,azure-documentdb
You are seeing ConnectionPoolTimeoutException because the DocumentClient does not automatically close the stream when createXXXXXXXX() is called. This behavior was intended to support streaming blob attachments for createAttachment(). To close the stream, you can call close() to close the stream or .getResource() to return the resource and then close the...
Yes, a store procedure would work great for upsert. There are even code samples available on DocumentDB's Github: Upsert (Optimized for Insert): https://github.com/Azure/azure-documentdb-js/blob/master/server-side/samples/stored-procedures/upsert.js Upsert (Optimized for Replace): https://github.com/Azure/azure-documentdb-js/blob/master/server-side/samples/stored-procedures/upsertOptimizedForReplace.js Bulk Import / Upsert: https://github.com/Azure/azure-documentdb-hadoop/blob/master/src/BulkImportScript.js ...
node.js,azure,azure-documentdb
Here are a few tips: Looking at the exception you are getting: client.createCollection(database._self,collectionDefinition, function(err,coll ^ TypeError: Cannot read property '_self' of undefined database is undefined because you are getting an error passed in to the callback. Looks like you the error message you are receiving is: { code: 401, body:...
c#,auto-increment,azure-documentdb
DocumentDB doesn't include auto-increment functionality out of the box. As Gaurav mentioned in Do we have Identity Column in DocumentDB, the id field is special in that if a value isn't provided by the application - DocumentDB will assign a GUID automatically. If you need auto-increment functionality - one potential...
I too was experiencing this so I came up with a workaround. Whenever I get this out of resources response, I actually remove and recreate the stored procedure. I created a library that takes care of this automatically based upon the response. It also takes care of automatic delay from...
Here's the gist... Today, all JSON properties in DocumentDB get automatically indexed by a Hash index; which means queries with equality operators (e.g. WHERE d.name= "george") are extremely fast. On the other hand, range queries (e.g. WHERE d.lastUpdateTime > 14) require a range index to operate efficiently. Without a range...
There is not current way of pausing the service. Anyway, as almost all things in Azure, even when paused you would still be paying for it.
Queries using != will require a scan, since all index entries have to be looked up to eliminate the values that are not equal to 5. This will lead to a high RUs. Whenever possible, please try rewriting the query as an equality or a range query. If the number...
There will be no perceptible performance difference between SelfLinks and IDs. You can use IDs directly within your application in place of SelfLinks.
You'll need a geospatial function like ST_NEAR or ST_DISTANCE. This is not currently available - you can check the status of the feature here. Short-term, especially if have < 1000 documents, you might be able to use an existing geospatial library e.g., System.Spatial and perform processing client-side. Alternatively, you can...
The stored procedure above queues document creates in a while loop until the API returns false. Keep in mind that createDocument() is an asynchronous method. The boolean returned represents whether it is time to wrap up execution right there and then. The return value isn't "smart" enough to estimate and...
security,azure,azure-documentdb
You are doing the right thing by storing your connection strings in your Azure Website environment so they are not in your web.config and therefore not in your source control system. And your local development process works well for you only because you are using the emulator which doesn't require...
c#,async-await,asp.net-web-api2,azure-documentdb
you can only use await inside a method if that method is async and async methods need to return Task, Task<T> or void although void returning async methods are reserved for event handlers because the exceptions thrown within them are swallowed and you cannot await their completion or chain subsequent...
The short answer is: you can't retrieve a list of collections inside a document. You can, however, retrieve a list of collection inside a database. Here's a look at the DocumentDB resource model: Looking at the DocumentDB resource model - databases contain collections, which in turn contain stored procedures, triggers,...
stored-procedures,triggers,azure-documentdb
Triggers can not be called from the server-side SDK (e.g. from inside another trigger or sproc).
mysql,mongodb,azure-documentdb
Based on the provided information you have the choice of several schema designs (with JSON as examples). I've made some assumptions, such as that more than one tank can be on one map and map data is only linked to a single map. You have to tweak it for your...
Yes - you can calculate the size of the document query response, so that all the system properties (e.g. _rid, _ts) are included. You will want to use UTF-8 encoding to get the correct size. You will also want to factor in an additional ~10% for indexing storage costs....
c#,search,like-operator,azure-documentdb
Wildcards like SQL's LIKE '% %' has not been implemented in DocumentDB yet. Please voice your opinion and vote for this feature on DocumentDB's feedback forum....
You can find sample code using the TransientFaultHandling Nuget package in the Github repo for the DocumentDB Data Migration Tool: https://github.com/Azure/azure-documentdb-datamigrationtool/blob/master/DocumentDb/Microsoft.DataTransfer.DocumentDb.FunctionalTests/DocumentDbHelper.cs Which looks something like this: private static IReliableReadWriteDocumentClient CreateClient(IDocumentDbConnectionSettings connectionSettings) { return new DocumentClient(new Uri(connectionSettings.AccountEndpoint), connectionSettings.AccountKey)...
At the moment, you'll need to sort the collection on the client or stored procedures (although I'd I'd recommend doing so on the client since stored procedures have bounded execution). To answer your questions: 1) I don't believe there is a guaranteed default ordering 2) The Stored Procedure would result...
You can use client.ReadDocumentCollectionAsync to get this information, e.g., ResourceResponse<DocumentCollection> response = await client.ReadDocumentCollectionAsync(collectionLink); Console.WriteLine(response.CollectionSizeQuota); Console.WriteLine(response.CollectionSizeUsage); See https://msdn.microsoft.com/en-us/library/microsoft.azure.documents.client.documentclient.readdocumentcollectionasync.aspx and https://msdn.microsoft.com/en-us/library/dn799209.aspx for more details. The equivalent in the REST API is the...
You can check for existing UDFs by running query using queryUserDefinedFunctions. Example: List<UserDefinedFunction> udfs = client.queryUserDefinedFunctions( myCollection.getSelfLink(), new SqlQuerySpec("SELECT * FROM root r WHERE [email protected]", new SqlParameterCollection(new SqlParameter("@id", myUdfId))), null).getQueryIterable().toList(); if (udfs.size() > 0) { // Found UDF. } ...
asp.net,asp.net-mvc,azure-documentdb
Set organization.Videos to a non-null value. Document db simply preserves what you stored. Apparently, you previously stored null.
azure-table-storage,azure-documentdb
Both are NOSql technologies, but they are massively different. Azure Tables is a simple Key/Value store and does not support complex functionality like complex queries (most of them will require a full partition/table scan anyway, which will kill your performance and your cost savings), custom indexing (indexing is based on...
No, index is not reserved. You should be able to use this.
You can combine the JOIN operator , which is used to form cross products with nested array elements, with the IN operator. SELECT docs FROM docs JOIN tags IN docs.tags WHERE tags IN ("B", "C") Note that because you are creating a cross product, that you will get a result...
Until the implementation of the "count" keyword, you should do your query in a store procedure on the server. Take care to not get all columns/properties in your query if you want only a count. Select only the id like dc.CreateDocumentQuery(update.SelfLink, "SELECT c.id FROM c")
Though DocumentDB allows you to store files (they are stored as attachments), I would not recommend using it. Here are my reasons: In the current version, the maximum size of an attachment is 2 MB. You can't really stream attachments. You would need to first read the attachment contents in...
c#,azure,async-await,task-parallel-library,azure-documentdb
That's because you're not awaiting an asynchronous method: This: var doc = _applicationResource.Save(data); Needs to be: var doc = await _applicationResource.Save(data); Your method should look as follows: [HttpPost] [Route("")] public async Task<IHttpActionResult> CreateNewApplication(dynamic data) { if (data == null) { return BadRequest("data was empty"); } try { var doc =...
You could define a sequential naming convention and create a range index on the collection indexing policy. In this way, if you need to retrieve a range of documents, you can do it in this way, which will leverage the indexing capabilities of docdb efficiently. As a recommendation, you can...
azure,multi-tenant,multi-tenancy,azure-documentdb
There's no such thing as a one-size-fits-all answer when it comes to partitioning / sharding tenant data. Generally, how you partition data depends on your application's query patterns as well as the resource requirements per tenant (in terms of both storage and throughput). Just keep in mind that collections are...
The issue was caused by not getting async calls right. I was calling an async method i.e. CreateDocumentAsync() incorrectly. I posted a separate question on async programming and the answers I got helped me troubleshoot the issue. Here's that post: Want to understand async...
azure,azure-worker-roles,azure-documentdb
The solution for me to get rid of the security exception and null reference exception was to disable intellitrace. Once I did that, I was able to deploy and attach debugger and see everything working. Not sure what is between the null in intellitrace and the DocumentClient, but hopefully it's...
You have to include "/" as an additional IncludedPath like: var collection = new DocumentCollection { id = "myCollectionID" }; collection.IndexingPolicy.IncludedPaths.Add(new IndexingPath { IndexType = IndexType.Range, Path = "/\"TimeStamp\"/\"Epoch\"/?", NumericPrecision = 7 }); collection.IndexingPolicy.IncludedPaths.Add(new IndexingPath { IndexType = IndexType.Hash, Path = "/" }); this._client.CreateDocumentCollectionAsync(database.SelfLink, collection); Alternatively, if you want to...
So with a slight modification to your schema my document now looks like this - {id : "123", name : "sai", marks : [ {Subject : "English", Grade : 50}, {Subject : "Maths", Grade : 60}, {Subject : "Science", Grade : 90} ] } Then a custom IndexingPolicy IncludePath has...
c#,database,excel,bulkinsert,azure-documentdb
Update: As of 4/8/15, DocumentDB has released a data import tool, which supports JSON files, MongoDB, SQL Server, and CSV files. You can find it here: http://www.microsoft.com/en-us/download/details.aspx?id=46436 In this case, you can save your Excel file as a CSV and then bulk-import records using the data import tool. Original Answer:...
The client is constructed with the authorization key and has no more information about it than you at that point. It's just a string. You may be able to leverage the DocumentClient.ReadPermissionAsync method to get permissions of a resource. I have not tried this first hand, but may be good...
azure,azure-documentdb,azure-search
Interesting that you're using Azure Search as your primary storage, as it's not built to be a database engine. The storage is specifically for search content (type typical pattern is to use Azure Search in conjunction with a database engine, such as SQL Database or DocumentDB, for example), using results...
node.js,azure-documentdb,docooment
I'd recommend going with one of the following options: DocumentDB's Data Migration Tool The easiest way to bulk import data from a CSV in to DocumentDB is to use DocumentDB's migration tool. You can find details on how to use it here. Programatically via Stored Procedures (Database-Side Scripting) If you'd...
c#,excel,insert,bulkinsert,azure-documentdb
Update (4/8/15): DocumentDB just released a data import tool, which supports JSON files, MongoDB, SQL Server, and CSV files. You can find it here: http://www.microsoft.com/en-us/download/details.aspx?id=46436 Original Answer: DocumentDB doesn't support bulk importing from excel files just yet... However, you can leverage DocumentDB's store procedures to make bulk import a bit...
You can find the DocDbWrapperScript.js on the DocumentDB JS Github here: https://raw.githubusercontent.com/Azure/azure-documentdb-js/master/server-side/DocDbWrapperScript.js...
mongodb,azure,migration,azure-documentdb
Are you downloading from http://www.microsoft.com/en-us/download/details.aspx?id=46436? You will need to temporarily enable popups for the download to initiate.
azure,primary-key,azure-documentdb
The account can take several minutes to provision, during which time the keys will not be available (since the account has finished creating). If, in the portal, you choose to Browse | DocumentDB accounts, you will see the status (creating, updating, online). Once your account is in an online state,...
This behavior is attributed to how JSON.NET deals with properties on dynamic objects. It effectively ignores them unless you decorate them with the JsonProperty attribute. You can either work with plain POCO or you can extend from Resource (shown below), which is a static object that Document itself extends. public...
Following pull request merge, if you want to use a proxy you need to get a release version greater than 1.0.1. As the release is not available at this moment, you could build github source and use the sdk snapshot version. Then you just need to setup your proxy through...
There is no built-in export option in the database service itself. However, the DocumentDB Data Migration Tool, published by the DocumentDB team, is available and provides both import and export capabilities (export is to JSON). Everything is documented on the DocumentDB area of the Azure site, specifically here. This isn't...
You can use DocumentDB's JOIN to create a cross product on documents with array elements. For example, the following query creates a cross-product on documents with it's students property to query on the students.name: select doc.id from doc join students in doc.students where students.name = 'sunny' returns the following dataset:...
web-api,asp.net-web-api2,azure-documentdb
First, when you await for a function that returns a Task<T>, you get a value of type T. That's why you put the await word in front of the function call. Second, what makes Web API very easy to use is that you don't need to work on the HTTP...
azure,azure-documentdb,azure-search
I can not comment below the current thread, so I will add a new reply. I am a Program Manager with Azure Search and I can confirm Daron's comments about this being a top request. There is also a fair amount of voting for it from our UserVoice page (http://feedback.azure.com/forums/263029-azure-search/suggestions/6328680-auto-indexing-of-docdb)....
I had a hard time getting information on how the DocumentDB pricing works. After lots of emails and support requests, I found that the pricing is based on per collection basis. So, if I have a DB with say two collections in it and each with a performance tier of...
node.js,azure,azure-documentdb
This happens when your rate of access exceeds your allowed quota. DocumentDB has a max of 2,000 request units per second per collection. An insert is charged (based on the size of your document, and number of indexed terms). If your insert costs 20 request units, you would be able...
You must reference the collection from the FROM clause. Note: Since you are issuing queries directly to a collection, you can use any arbitrary value for the name. However, you must re-use whatever value you chose in the other clauses because it serves as a reference point for projections and...
c#,.net,azure,authorization,azure-documentdb
We have confirmed this to be an issue isolated to the North Europe region. We are applying a hotfix to known accounts effected and will be deploying a fix shortly. If you are NOT in North Europe and are experiencing this issue, or if you continue to see if within...
SQL is the best option since LINQ requires type-binding. You can alternatively query using the type of Dictionary, then convert to Document (either by reading as an ExpandoObject, or by converting using JsonConvert). But SQL is going to be relatively cleaner: var query = client.CreateDocumentQuery<Dictionary<string, object>>(collectionLink) .Where(c => (string)c[propertyName] ==...
c#,azure,nosql,azure-documentdb
You can use JavaScript's delete operator to strip out DocumentDB's native properties. Take a look at this thread: How to remove a property from a JavaScript object Something like this should work: delete doc._rid; delete doc._ts; delete doc._etag; getContext().getResponse().setBody(doc); ...
javascript,azure,stored-procedures,nosql,azure-documentdb
Yes, you can embed common functions within stored procedure code - granted it fits inside the 256kb limit for a stored procedure. For your example, underscore is only 6kb minified and can be embedded inside a sproc. Note: keep in mind server-side scripts are resource governed (you may hit request...
I believe you are missing the using directive for the DocumentDB Linq provider (Microsoft.Azure.Documents.Linq). Please make sure you have the following directives to the top of your .cs file: using Microsoft.Azure.Documents; using Microsoft.Azure.Documents.Client; using Microsoft.Azure.Documents.Linq; ...
(Copying over Andrew Davis's answer, from the DocumentDB MSDN forums, for the stackoverflow community) The simplest way would be to have your Employee class inherit from Document, and then cast result.Resource to Employee. If you don't want to inherit from Document, you could also define an explicit cast between Document...
if you decorate your unique id using JsonProperty attributes then you can use your fields as the unique id, instead of having another id field on every document. and example is; [JsonProperty(PropertyName="id")] public string ProductId { get; set; } ...
You should take advantage of DocumentDB's JOIN clause, which operates a bit differently than JOIN in a RDBMs (since DocumentDB deals w/ denormlaized data model of schema-free documents). To put it simply, you can think of DocumentDB's JOIN as self-joins which can be used to form cross-products between nested JSON...
mongodb,couchdb,azure-documentdb
Most NoSQL databases do not support JOINs between multiple records - so you are left with a few different ways to model relationships. Normalizing As you mentioned, you could leave a soft-link to another document... and then resolve the reference with a follow-up query. Generally, you'd normalize data that you'd...
c#,azure,asp.net-web-api,azure-documentdb
It's not supported out of the box, but you can implement a stored procedure that does this. The msft product group has supplied some code samples here: https://code.msdn.microsoft.com/windowsazure/Azure-DocumentDB-NET-Code-6b3da8af/sourcecode?fileId=132409&pathId=34503205 Look under server side script, JS folder, you'll see an "orderby" script that does this. Adjust to your needs and try it....
DocumentDB uses JSON notation for escaping property names. Try: SELECT * FROM c.inside["abs-humid"]...
android,azure,azure-documentdb
Okay, this was a nightmare to solve. Nine hours later... -_-'. Long story short, here is some code that will actually work successfully. This code isn't perfect, and is dependent on Retrofit First, this is an example "Service" interface in Retrofit: import retrofit.Callback; import retrofit.http.Body; import retrofit.http.Header; import retrofit.http.POST; public...