Create a Data Storage using Azure Cosmos DB – DotNet Core Migration Strategy (Part 1)

cosmos

In accordance to my series on .NET Core migration, this one is the initial phase where I am going to demonstrate how Azure Cosmos DB works. However, this post do cover an introduction of Azure Cosmos DB even if you not intending to perform any migration and would like to use Cosmos DB as part of your solution design.

 

Why Azure Cosmos DB

Apart from various benefits that Azure Cosmos DB offers which you can find here, my intention to go for Cosmos DB were for the following reasons.

  • It is a NoSQL database which support query language like SQL and ACID transactions.
  • Following a PaaS model, it support global distribution of database in much more scalable way.
  • It is faster having minimum latency and 99.9% availability.
  • With automatic indexing of data, it provide an accessibility using any API for SQL, Azure table storage, Mongo DB, etc.
  • It provide the database model in various formats like Key-Value store, Graph DBMS, Document Store, etc.
  • It is serverless and geo-distributed which makes it an ideal candidate and an alternative of Redis Cache

There are two ways you can go for development stage with Cosmos DB.

  • Create a Cosmos DB resource through Azure portal and use it throughout your development lifecycle.
  • Use Cosmos DB emulator on-premise or locally throughout your development lifecycle and once you are good with the usage, initiate the process to create the Cosmos DB resource in Azure.

I would prefer to go for on-premise Cosmos DB emulator usage as for development purpose, I will heavily use the Cosmos DB as my data storage. The consumption might incur some cost which I don’t think is necessary for the development purpose. You can check the pricing details here. However, I am going to demonstrate how we can create a Cosmos DB resource through Azure portal and also how we can use emulator for development purpose.

 

Creating an Azure Cosmos DB in Portal

Search for Azure Cosmos DB in Azure Marketplace and select it to start the creation process.

image

In the DB Account creation page, create a new or select an existing Resource Group where the resource will be available. Create an Account Name which will be interpreted as {Account name}documents.azure.com.

Select the type of API you want to use. Currently, it supports five types of API – SQL, MongoDB, Gremlin, Cassandra and Azure table storage. Based on your requirement, you can select the preferred one. For this post and continuation of my DotNet Core Migration Strategy series, I am going to use SQL API.

By default, the Geo-Redundancy and Multi-region Writes will be disabled. I will keep it like that for now since I don’t need it here.

 

image

 

Next we need to setup the Network for our Cosmos DB account. In this section, create a new Vnet if you want or can use an existing Vnet. I am going to create a new Vnet here.

 

image

 

image

 

image

 

This section will ask you to provide some Tags that will help you to view consolidated billing. I am going to keep it empty for now.

image

 

Once everything is complete, the Cosmos DB Account will be created successfully.

image

 

You can review the overview of the deployment that has completed.

image

 

Now if you select the Cosmos DB Account, you might see an error message which says that it has failed to get the collection list. You can compare collection to a model like Order, Product, etc. The same issue is reflected when you open the Data Explorer which says that retrieval of data is blocked due to firewall configuration.

image

 

image

 

All you need to do is click on the arrow in the Data Explorer which asks for review. This you open the Firewall configuration of the Data Explorer. Check the checkbox which says Allow access from Azure Portal. That will fix both the issues.

image

All good till now. Our Azure Cosmos DB in portal is ready to use. Let’s have a look on how to use Azure Cosmos DB emulator.

 

Using Azure Cosmos DB Emulator

As I was sharing earlier, the emulator provide a local or on-premise environment that emulates Azure Cosmos DB for development purpose. I would prefer to use the emulator during development stage. For staging or production, I will switch to Azure Cosmos DB account from cloud. This will make the development lifecycle cost effective.

You can download the emulator from the following link. Once the MSI installer is download, proceed with the installation steps.

Upon successful installation, run the emulator which will open the local instance of the service like https://localhost:8081/_explorer/index.html

image

In the Quickstart window, the fields URI and Primary Key are the crucial ones. The URI will be used by DocumentDB Client to connect to Azure Cosmos DB and Primary Key will be used to authenticate the request and establish a connection to Cosmos DB. These two fields are going to be used as key-value pair in AppSettings section of application configuration or web.config file.

When you select the Explorer windows, you will find options to create a new database and add a collection to it. Every time you want to add a collection to an existing database, you just need to select the option Use existing Database id and add a collection to it.

image

image

 

Just for demonstration purpose, I would like to show how it looks like when you add a collection. After adding a new database and collections, you will find the database and collection getting created. When you select Documents from  a specific collection, you can click New Document to insert a new record. The request here is taken in a JSON format. Remember, the “id” field in the request always takes a string and represent the record. You can also associate unique key while creating the collection.

image

Clicking the record id, you can view individual record. The SQL query associated to display the records can be changed with additional filters based on your requirement.

image

However, we are going write utility to generate databases and collections dynamically on need basis. Let’s go ahead with the next step of programming.

 

Programmatically access and perform CRUD operation to Cosmos DB

In order to programmatically create a database and collections to Cosmos DB using C#, we need to do the following steps.

  • Add Microsoft.Azure.DocumentDB Nuget package to the project.
  • Modify your Web.Config file to add Cosmos DB Authentication Key, CosmosDB endpoint and the Database Name

image

  • Create the DocumentClient which will take the endpoint and authentication key

image

  • Create the database

image

  • Create the Collection

image

You need to create a helper class which will take input from various GET, PUT, POST, DELETE operations from API and perform the same using the Document Client. I have included the source code library in this post which will help you to see how this works. The DocumentDBRespository.cs file is the helper class which is called from various Action methods of API Controller to get, insert, update or delete records from or to a collection. The helper class identifies whether the database exists and if not it will create it. Same goes for collection.

The Endpoint and Authentication Key in Web.Config file are pointing to the emulator. In case of staging/testing or production deployment, we need to set the values of these keys to Azure Cosmos DB service from the portal.

To learn more about Cosmos DB and how programmatically you can achieve it, follow the link here.

Hope this post helps you.

 

Source Code:

sourcecode

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.