Tag Archives: Terraform

Setting up Azure Virtual Machines with Terraform

In my previous article about terraform, I explain what is terraform and what it can do. Also, I explain how to set it up and how we can use it with Azure to simplify infrastructure configuration. If you didn’t read it before you can view it using this link  

In this post, we are going to look further in to Azure infrastructure setup using terraform.

Before that lets look in to sample configuration of an Azure resource and see how syntax been used.

resource "azurerm_resource_group" "test" {

  name     = "acctestrg"

  location = "West US"

}

 resource "azurerm_virtual_network" "test" {

  name                = "acctvn"

  address_space       = ["10.0.0.0/16"]

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.test.name}"

}

Above code is to create an Azure resource group and Azure virtual network. In the code azurerm_resource_group and azurerm_virtual_network defines the azure resource type. The text test defines the name for that resource instance. This is not the azure resource group or azure virtual network name. This is the instance name. so, if you have another resource group it can be test2. Actual resource names are defined using name attribute. So, in above code the actual resource name for resource group is acctestrg and for virtual network its acctvn.

In above example, new virtual network is need placed under the acctestrg resource group. in the code it is defined using,

resource_group_name = "${azurerm_resource_group.test.name}"

in there, by azurerm_resource_group.test it defines the related resource group instance. In our example, it is test. Then using .name it calls for the attribute value of name under that particular resource group.

In the plan stage terraform creates the execution plan. It does not process the code top to bottom. It evaluates the code and then build the plan logically. There for it no longer consider the resource order. Let’s try it with an example, 

resource "azurerm_virtual_network" "test" {

  name                = "acctvn"

  address_space       = ["10.0.0.0/16"]

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.test.name}"

}

 resource "azurerm_resource_group" "test2" {

  name     = "acctestrg2"

  location = "West US"

}

 resource "azurerm_virtual_network" "test2" {

  name                = "acctvn2"

  address_space       = ["11.0.0.0/16"]

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.test2.name}"

}

 resource "azurerm_resource_group" "test" {

  name     = "acctestrg"

  location = "West US"

}

In above example, I am creating two resources group and two virtual networks. If you look in to highlighted sections, I placed the code related to virtual network before creating resources group. But when I run terraform plan it creates the execution plan in correct order.

tf1 

And once it is executed, it creates the expected resources.

tf2

As next step on demo, let’s see how we can create virtual machines in Azure using terraform.

resource "azurerm_virtual_machine" "testvm" {

  name                  = "acctvm"

  location              = "West US"

  resource_group_name   = "${azurerm_resource_group.test.name}"

  network_interface_ids = ["${azurerm_network_interface.test.id}"]

  vm_size               = "Standard_A0"

above code is an example to create a VM in azure. In code sample, azurerm_virtual_machine defines the resource type. testvm is the resource instance name. acctvm is the name of the virtual machine. According to code the resource will deploy under West US region. resource_group_name defines the resource group it belongs to. network_interface_ids defines the network interface id for the VM. vm_size defines the Azure VM template. The template list for the region can list down using following Azure CLI command.

az vm list-sizes --location west-us

This will list down the all available VM sizes in West US region.

tf3

Azure VM also need other components such as virtual network, storages, operating system so on. Let’s see how we can add these to the configuration.

In earlier on the post, I share samples for creating a resources group and virtual network. The next step of it will be to add a subnet under the virtual network.

resource "azurerm_subnet" "sub1" {

  name                 = "acctsub1"

  resource_group_name  = "${azurerm_resource_group.test.name}"

  virtual_network_name = "${azurerm_virtual_network.test.name}"

  address_prefix       = "10.0.2.0/24"

}

In above I am creating a subnet 10.0.2.0/24 under virtual network and resources group I already have. In code, azurerm_subnet defines the resource type. sub1 is the instance name and acctsub1 is the subnet name. resource_group_name defines on which resources group it belongs to. virtual_network_name defines which azure virtual network it associated with. address_prefix specifies the subnet value.

Now we have subnet also associated with network. We also need public IP address in order to connect to VM from internet. 

resource "azurerm_public_ip" "pub1" {

  name                         = "pub1"

  location                     = "West US"

  resource_group_name          = "${azurerm_resource_group.test.name}"

  public_ip_address_allocation = "dynamic"

}

According to above, I am creating public IP instance called pub1 under same resource group. it’s IP allocation is set to Dynamic. If need it can be static as well.

Next step is to create network interface for the VM.

resource "azurerm_network_interface" "ni1" {

  name                = "acctni1"

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.test.name}"

ip_configuration {

    name                          = "lan1"

    subnet_id                     = "${azurerm_subnet.test.id}"

   private_ip_address_allocation = "dynamic"

   public_ip_address_id  = "${azurerm_public_ip.pub1.id}"

  }

In above azurerm_network_interface is the resource type for the network interface. the interface name we are creating is acctni1. the second part of code which starts with ip_configuration defines the IP configuration for the network interface. subnet_id defines the subnet it belongs to. private_ip_address_allocation defines the ip allocation method. It can be Dynamic or Static. public_ip_address_id associates with the public ip created in the previous step. If this is not done you will not be able to connect to VM remotely once it is deployed.    

Next thing we need for the VM is storage. Let’s start with creating a Storage Account 

resource "azurerm_storage_account" "asa1" {

  name                = "accsa"

  resource_group_name = "${azurerm_resource_group.test.name}"

  location            = "westus"

  account_type        = "Standard_LRS"

 }

azurerm_storage_account is the resource type and accsa is the name for the account. account_type defines the storage account type. it can be Standard_LRS, Standard_GRS, Standard_RAGRS, Standard_ZRS, or Premium_LRS. More info about these account types can find from https://docs.microsoft.com/en-us/azure/storage/storage-introduction .

as next step, we can create a new storage container under the storage account.

resource "azurerm_storage_container" "con1" {

  name                  = "vhds"

  resource_group_name   = "${azurerm_resource_group.test.name}"

  storage_account_name  = "${azurerm_storage_account.test.name}"

  container_access_type = "private"

}

In above azurerm_storage_container is the resource type and it name is vhds. resource_group_name defines the resource group it belongs to and storage_account_name defines storage account it belongs to. container_access_type can be private, blob or container. More info about these container types can find from https://docs.microsoft.com/en-us/azure/storage/storage-introduction

Following image shows what it looks like when using GUI option. 

tf4

By now we have most of the resources ready for the VM. Next step is to define image for the VM.

  storage_image_reference {

    publisher = " MicrosoftWindowsServer"

    offer     = " WindowsServer"

    sku       = " 2016-Datacenter"

    version   = "latest"

  }

In above I am using windows server 2016 datacenter as image for the VM. Publisher, offer, sku and version info need to provide in order to select correct image. For windows servers, you can find these info in https://docs.microsoft.com/en-us/azure/virtual-machines/windows/cli-ps-findimage. For Linux, this info available at https://docs.microsoft.com/en-us/azure/virtual-machines/linux/cli-ps-findimage

Next step is to add a hard disk,

storage_os_disk {

    name          = "myosdisk1"

    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/myosdisk1.vhd"

    caching       = "ReadWrite"

    create_option = "FromImage"

  }

  storage_data_disk {

    name          = "datadisk0"

    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/datadisk0.vhd"

    disk_size_gb  = "60"

    create_option = "Empty"

    lun           = 0

  }

Above create two disks. one is for OS and one is for data. vhd_uri defines the path for the VHD which is saved under the storage account created.

Last but not least we need to define the OS configuration data such as hostname and administrator account details.

  os_profile {

    computer_name  = "rebelpro1"

    admin_username = "rebeladmin"

    admin_password = "Password1234!"

  }

In above, computer_name specify the hostname of the VM. admin_username specify the local administrator name and admin_password specify the local administrator password.

Now we have all the components ready to deploy a new VM. Some of the components we just need to create one time. as example virtual networks, subnets, storage accounts not need to create for each VM unless there is valid requirement. Let’s put all these together in to a one script so it will make more sense. 

# Configure the Microsoft Azure Provider

provider "azurerm" {

  subscription_id = "d7xxxxxxxxxxxxxxxxxxxxxx"

  client_id       = "d9xxxxxxxxxxxxxxxxxxxxxx"

  client_secret   = "f1xxxxxxxxxxxxxxxxxxxxxx "

  tenant_id       = "05xxxxxxxxxxxxxxxxxxxxxx "

}

resource "azurerm_resource_group" "rg1" {

  name     = "acctestrg"

  location = "West US"

}

resource "azurerm_virtual_network" "vn1" {

  name                = "vn1"

  address_space       = ["10.0.0.0/16"]

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.rg1.name}"

}

resource "azurerm_public_ip" "pub1" {

  name                         = "pub1"

  location                     = "West US"

  resource_group_name          = "${azurerm_resource_group.rg1.name}"

  public_ip_address_allocation = "dynamic"

}

resource "azurerm_subnet" "sub1" {

  name                 = "sub1"

  resource_group_name  = "${azurerm_resource_group.rg1.name}"

  virtual_network_name = "${azurerm_virtual_network.vn1.name}"

  address_prefix       = "10.0.2.0/24"

}

resource "azurerm_network_interface" "ni1" {

  name                = "ni1"

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.rg1.name}"

 

  ip_configuration {

    name                          = "config1"

    subnet_id                     = "${azurerm_subnet.sub1.id}"

    private_ip_address_allocation = "dynamic"

    public_ip_address_id  = "${azurerm_public_ip.pub1.id}"

  }

}

 resource "azurerm_storage_account" "storevm123" {

  name                = "storevm123"

  resource_group_name = "${azurerm_resource_group.rg1.name}"

  location            = "westus"

  account_type        = "Standard_LRS"

 

  tags {

    environment = "demo"

  }

}

 resource "azurerm_storage_container" "cont1" {

  name                  = "vhds"

  resource_group_name   = "${azurerm_resource_group.rg1.name}"

  storage_account_name  = "${azurerm_storage_account.storevm123.name}"

  container_access_type = "private"

}

 resource "azurerm_virtual_machine" "vm1" {

  name                  = "vm1"

  location              = "West US"

  resource_group_name   = "${azurerm_resource_group.rg1.name}"

  network_interface_ids = ["${azurerm_network_interface.ni1.id}"]

  vm_size               = "Standard_DS2_v2"

 

   storage_image_reference {

    publisher = "MicrosoftWindowsServer"

    offer     = "WindowsServer"

    sku       = "2016-Datacenter"

    version   = "latest"

  }

   storage_os_disk {

    name          = "osdisk1"

    vhd_uri       = "${azurerm_storage_account.storevm123.primary_blob_endpoint}${azurerm_storage_container.cont1.name}/osdisk1.vhd"

    caching       = "ReadWrite"

    create_option = "FromImage"

  }

   storage_data_disk {

    name          = "datadisk1"

    vhd_uri       = "${azurerm_storage_account.storevm123.primary_blob_endpoint}${azurerm_storage_container.cont1.name}/datadisk1.vhd"

    disk_size_gb  = "60"

    create_option = "Empty"

    lun           = 0

  }

     os_profile {

    computer_name  = "rebelpro1"

    admin_username = "rebeladmin"

    admin_password = "Password1234!"

  }

   tags {

    environment = "demo"

  }

}

Let’s verify the resources using Azure portal.

As we can see it is created all the expected resource under the resource group acctestrg.

tf5

Also, we can see it is created the VM as expected.

tf6

In this post, we went through the process of creating Azure VM and related components using terraform. Hope this was useful and if you have any questions feel free to contact me on rebeladm@live.com

Azure resource setup simplified with terraform

This week I was testing Terraform , a simple tool which can use to automate Azure resource deployment.

It will be easier to explain terraform with a real-world example. I am developing a web application and as my resource provider I am using Azure. my first requirement is to setup a development environment. For that I need at least one web server, one database server and connectivity between these two servers. to setup the environment, I log in to the portal and then setup resource group, storage account and virtual network. After that I start to build servers. after I complete it, I setup web server application and database server. so even its looks like straight forward, its takes time. later on, in the development process I also required a test platform where I can try my application with different operating systems. Also, I like to test application by adding more components such as load balancers, web servers to the environment. These testing environments are temporally. So, each and every time I need to setup environment with all the different components and once testing process completes, needs to destroy it. when I need to sell it as a solution to people, I face another challenge as not everyone wants to run it on Azure. Even if it’s another service provider or on-premises environment, application should test in similar environment prior to sell as a solution. Each provider has their own way of setting up things.

On this given scenario I faced few challenges,

To setup required resources for application takes time as each component need to configure in certain way. 

To setup integration between components, I need to log in to different systems and adjust the settings. A single mistake can cause hours of disruptions to the project. 

Due to the complexity of setting up environments, I may end up keep running test environments longer which probably increase my development cost. 

How Terraform can help?

Using Terraform I can deploy the whole environment by executing a single script file. This script is basically a set of instructions explaining how to setup each and every component. If I need to setup a new server in azure from scratch there is a procedure for it. we need to setup relevant resource groups, network components, storage accounts before we start to build the server. terraform itself understand these dependencies and build the environment according to that. This also helps to standardize the resource setup process.

Terraform also can use to configure application settings as part of the environment setup. That means we do not need to log in to systems to make initial software configurations. This will prevent the human errors.

Once we setup an environment using terraform we can change it, destroy it using a single command. As an example, let’s assume we setup a test environment with two web servers (using terraform). I have a new requirement to add new web server to the same environment. To do that all I need to do is modify the same script and add new entry for new webserver. once I execute it, it will automatically detect the current environment and only add the missing components. When destroy, it is again a single command and it will remove each component in the proper order. As example, it will understand that before remove resource group, it need to remove all other components under it.

As setup and destroy process of resources is easy with terraform, we do not need to keep running non-critical resources. As an example, if I need to give a POC or show a demo to a customer, all I need to do is to execute the pre-created terraform script when needed and destroy it afterwards.

Terraform support different service providers. It is not only for cloud based solutions. It also supports on-premises solutions. As an example, terraform can use with Azure Pack and Azure stack to do the same thing in on-premises Hyper-V environment. It also supports to SaaS application configurations. The supported providers list can be found in here https://www.terraform.io/docs/providers/index.html

Terraform mainly have three functions.

Plan – Before execute the configuration, it should go for the planning stage. In here terraform will build the execution plan based on the configuration provided by the engineer. It will explain what will be created when configuration is executed.

Apply – In this phase it will apply execute the execution plan created on the “planning” stage. It will also report back once its completed the resource setup. If there were errors, it will also explain it in details.

Destroy – This is basically to undo the execution plan. By calling this, we can destroy all the resource created by a particular terraform configuration file.

I think it’s enough with the theory, let’s see why it’s so cool.

In my demo, I am going to show how to setup terraform and how to use it to create resource in azure. 

Setup Terraform

In my demo, I am going to use windows 10 as the system. Terraform also supported on Linux, Mac and solaris systems.

1) Go to this link and download the file relevant to windows architecture. 

2) Then create a folder and move the downloaded terraform.exe file. 

3) Next step is to setup the Binary path for the terraform so system knows when we use the terraform commands. To do that, run the PowerShell console as Administrator and then type

$env:Path += ";C:\terraform"

In here C:\terraform is the folder where I saved the terraform.exe

terra1

4) As next step, we can confirm terraform setup by running terraform in the PowerShell console. 

terra2
 
This confirms the Terraform setup and next step to configure Azure side to support terraform. 
 
Retrieve Required info from Azure

Terraform uses Azure ARM API to connect and manage azure resources. To connect to Azure, terraform need to provide following Azure ARM environment variables using configuration file.

ARM_SUBSCRIPTION_ID

ARM_CLIENT_ID

ARM_CLIENT_SECRET

ARM_TENANT_ID

To get ARM_CLIENT_ID, ARM_CLIENT_SECRET and ARM_TENANT_ID we need to create a Service Principal in Azure.

To do the we can use Azure Cloud Shell.  

1) Log in to Azure Portal ( https://portal.azure.com ) as a Global Administrator

2) Click on Cloud Shell Button. 

terra3

3) Then it will open the shell in the same window. If it’s your first time using this feature, it will ask to create a storage account. 

terra4

4) Next step is to fine the Subscription Id. To do that type following and press enter. 

az account list

Then it will provide an output like following. In there “id” represent the Subscription ID we required. 

terra5

5) Next step is to create the Service Principal. In order to do that use,

az ad sp create-for-rbac –role="Contributor" –scopes="/subscriptions/xxxxxxxxxxxx"

in above command xxxxxxxxxxxx should replace with the Subscription ID we found in the previous step. 

Then it gives an output similar to following

terra6

In above image 

appId is equal to Client ID.

Password is equal to Client Secret

Tenant is equal to Tenant ID

Now we have all the information we need in order to connect to Azure trough terraform. 

Create first configuration

Next step is to create first terraform configuration file. The file is using the extension of .tf. You can use your favorite text editor to create the file. I am using Visual Studio code and it can be download from https://code.visualstudio.com/

The file no need to save on the same folder where your terraform.exe file. However, you need to navigate to that folder before execute the terraform commands. In my demo, it is C:\terraform

My first configuration is following

 

# Configure the Microsoft Azure Provider

provider "azurerm" {

  subscription_id = "xxxxxxx"

  client_id       = " xxxxxxx "

  client_secret   = " xxxxxxx "

  tenant_id       = " xxxxxxx "

}

resource "azurerm_resource_group" "myterrapro1" {

  name     = "myterrapro1"

  location = "West US"

}

resource "azurerm_virtual_network" "myterrapro1network" {

  name                = "myterrapro1vn"

  address_space       = ["10.11.12.0/24"]

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.myterrapro1.name}"

}

In above code,

provider "azurerm"

define the service provider as Azure ARM.

 subscription_id = "xxxxxxx"

                client_id       = " xxxxxxx "

                client_secret   = " xxxxxxx "

                tenant_id       = " xxxxxxx "

the above values should replace by the values we found through the Azure.

resource "azurerm_resource_group" "myterrapro1" {

  name     = "myterrapro1"

  location = "West US"

}

The above saying to create new azure resource group called myterrapro1 in West US region.

resource "azurerm_virtual_network" "myterrapro1network" {

  name                = "myterrapro1vn"

  address_space       = ["10.11.12.0/24"]

  location            = "West US"

  resource_group_name = "${azurerm_resource_group.myterrapro1.name}"

in the next section, it is creating an Azure Virtual Network called myterrapro1vn. It got address space allocated as 10.11.12.0/24. This will create under resource group called myterrapro1. This virtual network also will be created under West US region.

Once script is ready, save it with .tf extension.

Then launch PowerShell as Administrator. Then change the folder path to where script file is saved. In my demo, its C:\terraform. After that type following,

terraform plan

This is step where terraform build the execution plan.

terra7

This output shows what will happen when apply the execution plan.

To apply the plan type following and press enter.

terraform apply

once process is started, it will also show the progress of setting up resources.

terra8

according to above image we can see its successfully created the resources. we can confirm it using the Azure portal.

terra9

terra10

Now we know how to create resource. Let’s see how to destroy the resources we created.

In order to do that you do not need to change anything in the script. All you need to do is to issue the command,

terraform destroy

once we run the command it will ask to confirm. Type yes and press enter to proceed with the destroy process.

terra11

As we can see it remove all the resource in the configuration file. Once it done we can also log in to azure portal and confirm.

Isn’t this cool?

In this blog post I explained what is terraform and how we can use it to simplify resource setup in azure. in next blog post I will share more examples to show its capabilities.

Hope this was useful and if you have any questions feel free to contact me on rebeladm@live.com