Terraform for the Azure ARM Developers

HashiCorp has launched 1.10 of the Terraform launching many more azure services than they used to for the Azure Provider. More details can be read here on their github page. Up until now, Azure ARM has been the choice of Azure Automation for DevOps practitioners. Since terraform can be used to automatically provision resources across major cloud providers, it is better to learn terraform sooner rather than later. In this blog post, we’ll see how the Terraform translates with respect to Azure ARM and understand its way of doing things.

Providers

Terraform allows to create, configure and manage almost all types of resources from on-premise physical machines to cloud based resources.
This is done by making use of different providers that abstract the underlying APIs for doing the work. So we have a provider for AWS, a provider for Azure, etc. The list of all terraform providers can be found here. We can also found documentation for Microsoft Azure here which we’ll be making use of most of the time.

In ARM Templates, resources are referred to by type and the API version that they exist in:

{
  "name": "MyVnet",
  "type": "Microsoft.Network/networkInterfaces",
  "apiVersion": "2016-09-01",
  ... 
}

With Terraform, you specify the resource as it is named in the provider and then give it a local name you can refer to it by in the Terraform (in the example below, the resource type is azurerm_virtual_network – a vnet from the azurerm provider – and the local name for this vnet – used only in the Terraform file – is vnet1):

resource "azurerm_virtual_network" "vnet1" {
  name                = "MyVnet"
  ... 
}

This local name ‘vnet1’ can be then used anywhere else in the Terraform file to refer this resource.

The provider must be downloaded and be available locally. This is done by running the ‘terraform init’ command which downloads all the providers referred to by the template files:

Install providers using terraform init
Install providers using terraform init

Commands and Files

When running ARM templates with PowerShell, most of the time one would end up using one of the two cmdlets:
1) Test-AzureRmResourceGroupDeployment is to test a deployment (looking for errors)
2) New-AzureRmResourceGroupDeployment which does the actual deployment.

Each command accepts a single template file and parameter file to be run in that resource group. For example:

New-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName -TemplateFile $templateFilePath -TemplateParameterFile $parametersFilePath

With Terraform we have four main actions – init, plan, apply and destroy.
1) terraform init will prepare the directory, download any listed providers, etc. You run it once in a new Terraform project and then again if anything changes
2) terraform plan will show all the changes that potentially will be made if the Terraform is run. It is like doing a test run
3) terraform apply does the actual creation or change of the resources
4) terraform destroy will delete the resources described in the template files

With the terraform commands, there is no need to specify the input files. Instead, Terraform will load and combine the all the relevant files in the current directory and its sub-directories. Template files end with .tf and variable files are named either terraform.tfvars or *.auto.tfvars – thus, you can separate and name your files whatever you like (e.g. all the storage resources in a file called storage.tf, all the storage variables in a file called storage.auto.tfvars, etc.)

Resource Groups

The first difference between ARM and Terraform to point out is of resource groups. By default, ARM templates runs within the context of a resource group and it must already exist when running the ARM templates. If one has resources that span resource groups, then he/she needs to deploy them separately (one per resource group).

Terraform allows resource group creation so you can have multiple resource groups (and their associated resources) within a single Terraform definition. So this allows us more simpler definitions.

Parameters and Variables

ARM templates have parameters (that may change between deployments), variables (that change less often) and the actual resource definitions (which don’t generally change). With ARM, parameter values can be placed into a separate file (e.g. parameters.json) or entered at runtime.

Terraform parameters are called variables and their values can be stored in a file (for example, terraform.tfvars), entered on the command line (for example, var ‘access_key=foo’), or via environment variables (for example, set TF_VAR_access_key=foo).

For comparison here is a parameter definition in ARM (contained in the .json template file):

"parameters": {
    "location": {
      "type": "string",
      "defaultValue": "centralindia",
      "metadata": {
        "description": "Which region to use."
      }
    }
}

Here is the same parameter (called a variable) in Terraform (contained in a .tf template file):

variable "location" {
  description = "Which region to use."
  default = "centralindia"
}

Setting a value for a parameter in an ARM parameters.json file looks like:

"parameters": {
        "location": {
            "value": "southindia"
        }
}

Setting a value for a variable in a Terraform terraform.tfvars file looks like:

location = "southindia"

In ARM templates, you use variables to store values that do not change as often as parameters but that you don’t want to code in-line with your resource descriptions. An example might be the name of the VNET. In Terraform, these are called locals. For example, for the virtual network name, we can use the name of the resource group with “-vnet” as suffix.

With ARM, this looks like this:

"variables": {
    "virtualNetworkName": "[concat(parameters('resourceGroupName'), '-vnet')]"
}

With Terraform, this looks like this:

locals {
  virtualNetworkName = "${var.resourceGroupName}-vnet"
}

When making use of parameters and variables in ARM templates you refer to them like:

{
      "name": "[variables('publicIpAddressName')]",
      "type": "Microsoft.Network/publicIpAddresses",
      "apiVersion": "2016-09-01",
      "location": "[parameters('location')]",
      "properties": {
        "publicIpAllocationMethod": "[variables('publicIpAddressType')]"
      }
}

With Terraform, they are referred to as:

resource "azurerm_public_ip" "basicvm" {
  name                         = "${local.publicIpAddressName}"
  location                     = "${var.resource_group_location}"
  ... 
  public_ip_address_allocation = "${local.publicIpAddressType}"
}

Resources and Dependencies

One of the other differences is how resource dependencies are determined when defining resources. With ARM you must explicitly specify that a resource depends on another resource by either nesting the resources or listing them in the dependsOn property. With Terraform, dependencies are implied by referring to the dependent objects, so no explicit dependency declaration is needed (although it can be done if you want to control the order of creation).
For example, here is the code to create a NIC with ARM:

{
      "name": "[variables('networkInterfaceName')]",
      "type": "Microsoft.Network/networkInterfaces",
      "apiVersion": "2016-09-01",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
        "[concat('Microsoft.Network/publicIpAddresses/', variables('publicIpAddressName'))]",
        "[concat('Microsoft.Network/networkSecurityGroups/', variables('networkSecurityGroupName'))]"
      ],
     "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "subnet": {
                "id": "[variables('subnetRef')]"
              },
              "privateIPAllocationMethod": "Dynamic",
              "publicIpAddress": {
                "id": "[resourceId(resourceGroup().name,'Microsoft.Network/publicIpAddresses', variables('publicIpAddressName'))]"
              }
            }
          }
        ],
        "networkSecurityGroup": {
          "id": "[resourceId(resourceGroup().name, 'Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"
        }
}

In the above ARM example, its specified that the VNET, Public IP, and network security group must be created before the NIC can be created by listing these resources in the dependsOn section.

The Terraform version looks like:

resource "azurerm_network_interface" "basicvm" {
  name                = "${local.networkInterfaceName}"
  location            = "${var.resource_group_location}"
  resource_group_name = "${azurerm_resource_group.basicvm.name}"

  ip_configuration {
    name                          = "ipConfig"
    subnet_id                     = "${azurerm_subnet.basicvm.id}"
    private_ip_address_allocation = "dynamic"
    public_ip_address_id          = "${azurerm_public_ip.basicvm.id}"
  }
}

In the example above, because we refer to the subnet and public IP via the ID of the object, this means Terraform will create these resources first (and then obtain the IDs of the created objects) before creating the NIC.

Data / Output

Sometimes one needs to retrieve a value from an existing resource to use. In ARM, you have helper functions such as resourceId that will look up the ID of a resource previously created based on the name. For example:

"variables": {
          "nsg_id": "[resourceId('MyResourceGroup', 'Microsoft.Network/networkSecurityGroups', 'MyNsg')]"
}

This will create a variable ‘nsg_id’ by looking up the ID of an NSG contained in the resource group ‘MyResourceGroup’ with the name ‘MyNsg’ and obtaining the resource ID for it.

To do the same thing in Terraform, we use the ‘data’ resource type. For example, to get the ID of an existing NSG and output it at the end of the Terraform we would do:

data "azurerm_network_security_group" "test" {
  name                = "MyNsg"
  resource_group_name = "MyResourceGroup"
}
output "nsg_id" {
  value = "${data.azurerm_network_security_group.test.id}"
}

State

ARM Templates are stateless in that they describe the end state of the resources. When run, the ARM mechanism will determine what needs to be done to make the changes to the resource group to match the state described in the template. This may mean creating or modifying resources as needed. Unless run in incremental mode, running an ARM template will not remove resources not listed in the template.
Terraform, however, requires that the state of the deployment be persisted. This includes all the changes that Terraform has made. This state is used to determine what to change or delete when the template file is changed. If the state file is lost or corrupted Terraform does not know what has been created and therefore will simply try to recreate everything. For groups working together, there are ways to share state across users.

We have already discussed in one of our previous blog posts, on how to authenticate to Azure when using Terraform. Once we are authenticated, we can run the Terraform scripts that we created. Or we can also incorporate authentication part and resource provisioning part into single Terraform script.

4 thoughts on “Terraform for the Azure ARM Developers

    1. Terraform seems to be the future choice by Microsoft too as it is now part of Azure Certification exams besides its other advantages. So its good to learn anyways. However keeping and maintaining legacy code mandates one to learn azure arm as well.

      Like

      1. State is a very powerful concept but also one of the weakness in the Terraform. Since we are focusing on disadvantages here, here’s some wrt states 1) It stores sensitive configuration data about deployments in plain text 2) Each state is tied to an environment. So if you need multiple environments, you need to use workspaces, another complication. 3) You need to have backup provisioning for state files.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s