Provision CosmosDB in Azure using Terraform

Recently we began exploring option to fulfil a requirement from project teams, to provision NoSql instances on demand. I will go ahead and describe one of those options below.

Since, we are working with Azure; thought to start our investigation using CosmosDB. To be independent and later to migrate to another provider, one of the best options available out there, in the market is “Terraform”. I used Terraform before to provision multiple resources both in AWS and GCP, so, wanted to utilize this opportunity to explore Azure.

CosmosDb information — copy/paste from Azure docs. Since “picture speaks more than a thousand words”, please look at image after reading below definition.

Azure Cosmos DB is a fully managed database service with turnkey global distribution and transparent multi-master replication. Get single-digit millisecond read and write latencies at the 99th percentile, automatic and elastic scaling of throughput and storage worldwide, 99.999-percent high availability, and five well-defined consistency choices — all backed by industry-leading comprehensive SLAs.

Azure Cosmos DB & its offerings

Let’s talk a bit more in depth about CosmosDB , the highest abstraction in CosmosDB resource is account, this is where we manage our data by creating multiple databases and containers. Basically under one account we can create multiple databases and then multiple containers under those. Order is as below,

CosmosDB Account → Databases → Containers → Items

Containers can be collections, tables, graphs etc., based on the API you chose while creating the database. Items in turn can be Document, Row, Node accordingly. In this blog I will mention about how to create an account.

All the information about CosmosDB can be further found here → https://docs.microsoft.com/en-us/azure/cosmos-db/introduction

To begin, we need few parameters to start our journey.

1. A free Azure trial account, it provides us with 260 CAD for 30 days, should be sufficient for this task. Also, recently Azure made CosmosDB, one of the “always free” products, of course, with some limitations though.
2. Install Azure cli on your local machine, for Mac, you can use “brew install azure-cli” or windows follow this guide.(https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)
3. Once installed, run “az login”, this will open a link in the browser. Once logged in, you should see something like below with “id” & “homeTenantId” details. Keep a note of these IDs. These basically tells the azure provider to create the resources for this subscription & tenant, that means, basically your account.

Let’s start with our config for Terraform, please clone this repo from Github. https://github.com/kprasant/cosmosDb-tf-demo

The repo contains 4 files, I always try to separate out provider info from the actual resource. That is why you see 4 files, 2 for provider (azure-variables.tf and provider.tf) and 2 for the actual resource. Let’s get to the details file by file.

Azure-variables.tf

variable "subscription_id" {
  default = "<ADD YOUR ID>"
}
variable "tenant_id" {
  default = "<ADD YOUR HOMETENANTID>"
}

Earlier above in step 3, we noted down “id” and “homeTenantId”, please paste them in the above as default values. “id” goes into “subscription_id” and “homeTenantId” into “tenant_id”. This basically tells terraform to tell Azure provider to create the resources under this “subscription_id” using provider.tf file below.

provider.tf

provider "azurerm" {
  version = "~> 1.34.0"
  subscription_id = "${var.subscription_id}"
  tenant_id = "${var.tenant_id}"
} 
resource "random_integer" "ri" {
  min = 10000
  max = 99999
}

Here we are creating a random integer “ri” resource, it’s an integer between 10000 to 99999, we will use this to append to our CosmosDb account name. Well, technically this integer is not needed if everything goes smoothly and we are going to setup only 1 account. This random integer will help to create multiple accounts with out a need to change any of our code. Also, I observed some issues during provisioning, where Azure complained about unavailability in certain regions and terraform was unable to neither delete or recreate these accounts, so this random integer came to rescue there. But, technically this random integer resource is optional.

Next, let’s look at “variables.tf” file.

variables.tf

variable "resource_group_name" {
  default = "rg-cosmosdb-tf"
}
variable "resource_group_location" {
  default = "eastus"
}
variable "cosmos_db_account_name" {
  default = "db-cosmos-account-demo"
}
variable "failover_location" {
  default = "eastus2"
}
variable "consistency_level" {
  default = "session"
}
variable "kind" {
  default = "GlobalDocumentDB"
}

In Azure, “Resource Groups” are used to bundle all related resources in one place, to efficiently control access and to audit. Also, there are certain guidelines when coming to naming our resources, please follow this link. https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/naming-and-tagging

I tried to stay with in the guidelines while naming the resources. So we create a resource group “rg-cosmosdb-tf”. Next we set the location (region) where this resource group needs to be created. We then assign a name to our account and failover location, consistency_level and kind of the account we want to create. All details regarding these variables can be found in terraform page → https://www.terraform.io/docs/providers/azurerm/r/cosmosdb_account.html

Offer_type is standard and that’s the only choice, ‘kind’ defaulted to “GlobalDocumentDB” and the other kind is “MongoDB. ‘Consistency_policy’ basically defines that data consistency requirements for this account, it has the following options,

• Eventual doesn’t guarantee any ordering and only ensures that replicas will eventually converge
• Consistent prefix adds ordering guarantees on top of eventual
• Session is scoped to a single client connection and basically ensures a read-your-own-writes consistency for each client; it is the default consistency level
• Bounded staleness augments consistent prefix by ensuring that reads won’t lag beyond x versions of an item or some specified time window
• Strong consistency (or linearizable) ensures that clients always read the latest globally committed write

‘failover_priority’ of ‘0’ indicates a write region and rest of regions take unique values with max of (n-1), n being number of regions. Since we have only 2 regions specified, we have 0,1.

Now, the resource provisioning tf file “cosmosDb-account.tf”.

cosmosDb-account.tf

resource "azurerm_resource_group" "rg" {
  name = "${var.resource_group_name}"
  location = "${var.resource_group_location}"
}
 
resource "azurerm_cosmosdb_account" "acc" {
  name = "${var.cosmos_db_account_name}-${random_integer.ri.result}"
  location = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  offer_type = "Standard"
  kind = "${var.kind}"
  enable_automatic_failover = true
  consistency_policy {
    consistency_level       = "${var.consistency_level}"
  }
  geo_location {
    location = "${var.failover_location}"
    failover_priority = 1
  }
  geo_location {
    prefix = "${var.cosmos_db_account_name}-${random_integer.ri.result}-customid"
    location = "${var.resource_group_location}"
    failover_priority = 0
  }
}

Now, let’s provision this account,

Since you have already logged in using “az login” before, we can straight away proceed. Start with running, the following in order inside the cloned folder,

terraform init
terraform plan -out cosmos.txt

The ‘terraform init’ command will create a new environment and download and install all binaries that are based on the provider that we have chose.

terraform init

‘terraform plan -out cosmos.txt’ checks the syntax and connect to azure based on the details that we provided and compares the state of the resource if exists with what we have requested. This information is then stored in “cosmos.txt” file. We can view the file and confirm what we want to created is what that is represented in that file. ‘-out cosmos.txt’ is optional.

Run 'terraform apply "cosmos.txt"

This will then create the cosmosDb account after 10–15 mins. You will see the following output:

terraform apply “cosmos.txt”

You can then run ‘terraform show’ to view the state and the endpoint details of the cosmosDb account that was created. Or you can simple cat the ‘terraform.tfstate’ file, which I highly recommend not to edit. You can also login into Azure portal and confirm all the details.

Once you are satisfied with the results and decided to de-provision, simple run ‘terraform destroy’. This would then delete all the resources that are created before. You should see similar output like below.

terraform destroy

This is a very basic blog describing how to simply create a cosmosDb account in Azure. Next steps are to create databases with in this account and then containers with in those databases, which is out of scope for my exploration into the requirement stated early in this page :).

Anyway, I hope this page helps you with your journey/research with cosmosDb in Azure. Let me know if you have any questions.

Thank you.