Terraform for vSphere – OVA/OVF support

As I am gearing up for a VMworld session with Gilles Chekroun on Terraform and an upcoming webinar with HashiCorp’s Grant Orchard, I thought I’d publish a series of short posts on Terraform around some of the features and gotchas I have myself discovered over the past few months. First one: remote OVA/OVF support with Terraform!

Terrafom support for remote OVF

This feature is something that regular Terraform for vSphere users have been asking for.

If you look at my previous posts around deploying VMs in Terraform, you will see I tend to clone VMs from an existing VM template. What I also wanted to do was to deploy an OVA/OVF, ideally from a remote location, such as an AWS S3 bucket.

And of course, I would want to deploy the VM in VMware Cloud on AWS SDDC !

William Lam published a great post around this Terraform feature here but there are a few additional aspects I would want to mention.

ESXi host access

First thing, to deploy a VM from a remote OVA/OVF, you need access to the ESXi hosts from the client running Terraform. Given the way VMware Cloud on AWS is designed, vCenter can be accessed over the Internet (via a public IP address) or over VPN/Direct Connect but ESXi access is only possible over its private IP.

This means that the client executing Terraform to deploy a remote OVA/OVF to a vCenter needs to be either:

  1. Deployed within VMware Cloud on AWS
  2. In a remote site (VPN/Direct Connect) where routing and security have been configured to allow traffic from the client to the ESXi hosts network
  3. In the VPC directly attached to VMware Cloud on AWS

This third option is what I’ve done in my lab. I deployed an AWS Linux machine on EC2 in the VPC connected to my VMware Cloud on AWS SDDC:

[ec2-user@ip-172-201-10-124 ~]$ uname -srm
Linux 4.14.186-110.268.amzn1.x86_64 x86_64
Deploy VM from remote OVA/OVF with Terraform with VMC

Preparation – Installation of Terraform on EC2 Linux

In an upcoming blog, I will show how to install Terraform on my Linux VM using Terraform (let me repeat that – I used the Terraform AWS provider to deploy an EC2 instance and install Terraform at boot time) but here is how to do it manually:

  1. Download Terraform binary from the Terraform website – for example, the file might be terraform_0.12.29_linux_amd64.zip
  2. Securely transfer Terraform via SCP to the EC2 instance from my Mac
  3. Unzip and move it to /usr/bin

On my Mac Terminal, here is how to upload over SCP to an EC2 instance:

scp -i "MyKeyPair.pem" terraform_0.12.29_linux_amd64.zip ec2-user@ec2-A-B-C-D.eu-west-2.compute.amazonaws.com:

On EC2, just unzip the Terraform provider and move it to your executable folder:

unzip terraform_0.12.29_linux_amd64.zip 
sudo mv terraform /usr/local/bin/

Here are my Terraform files below. You can also find them on this GitHub repo.

What I am creating is some vSphere tags and two VMs from remote OVFs:

  1. The Nested ESXi template William used in his blog
  2. A VMware Photon OS OVA
provider "vsphere" {
  user                 = var.vsphere_user
  password             = var.vsphere_password
  vsphere_server       = var.vsphere_server
  allow_unverified_ssl = true
}

data "vsphere_datacenter" "dc" {
  name = var.data_center
}
data "vsphere_compute_cluster" "cluster" {
  name          = var.cluster
  datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_datastore" "datastore" {
  name          = var.workload_datastore
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_resource_pool" "pool" {
  name          = var.compute_pool
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_host" "host" {
  name          = "10.10.10.68"
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_network" "network" {
  name          = "sddc-cgw-network-1"
  datacenter_id = data.vsphere_datacenter.dc.id
}

resource "vsphere_tag_category" "environment" {
    name        = "environment"
    cardinality = "SINGLE"
 
    associable_types = [
        "VirtualMachine"
    ]
}
 
resource "vsphere_tag_category" "region" {
    name        = "region"
    cardinality = "SINGLE"
 
    associable_types = [
        "VirtualMachine"
    ]
}
 
resource "vsphere_tag" "environment" {
    name        = "test-dev"
    category_id = vsphere_tag_category.environment.id
}
 
resource "vsphere_tag" "region" {
    name         = "UK"
    category_id = vsphere_tag_category.region.id
}


resource "vsphere_virtual_machine" "Red" {
  name   = "Nico-VM"
  resource_pool_id = data.vsphere_resource_pool.pool.id
  datastore_id = data.vsphere_datastore.datastore.id
  datacenter_id = data.vsphere_datacenter.dc.id
  host_system_id = data.vsphere_host.host.id
  folder           = "Workloads"
  wait_for_guest_net_timeout = 0
  wait_for_guest_ip_timeout = 0

  ovf_deploy {
    remote_ovf_url = "https://xxxxxx.s3-us-west-2.amazonaws.com/yyyy.ova"
    disk_provisioning = "thin"
    ovf_network_map = {
      "sddc-cgw-network-1" = data.vsphere_network.network.id
    }
  }
}


resource "vsphere_virtual_machine" "vmFromRemoteOvf" {
  name             = "vm-deployed-from-ova"
  resource_pool_id = data.vsphere_resource_pool.pool.id
  datastore_id     = data.vsphere_datastore.datastore.id
  datacenter_id    = data.vsphere_datacenter.dc.id
  host_system_id   = data.vsphere_host.host.id
  tags = [
        vsphere_tag.environment.id,
        vsphere_tag.region.id,
   ]
  wait_for_guest_net_timeout = 0
  wait_for_guest_ip_timeout  = 0

  ovf_deploy {
    // Url to remote ovf/ova file
    remote_ovf_url    = "https://download3.vmware.com/software/vmw-tools/nested-esxi/Nested_ESXi7.0_Appliance_Template_v1.ova"
    disk_provisioning = "thin"
    ovf_network_map = {
      "VM Network" = data.vsphere_network.network.id
    }
  }

  vapp {
    properties = {
      "guestinfo.hostname"  = "tf-nested-esxi-1.primp-industries.com",
      "guestinfo.ipaddress" = "192.168.30.180",
      "guestinfo.netmask"   = "255.255.255.0",
      "guestinfo.gateway"   = "192.168.30.1",
      "guestinfo.dns"       = "192.168.30.1",
      "guestinfo.domain"    = "primp-industries.com",
      "guestinfo.ntp"       = "pool.ntp.org",
      "guestinfo.password"  = "VMware1!23",
      "guestinfo.ssh"       = "True"
    }
  }
}

Alongside this file above (main.tf), I also have a variables.tf file:

variable "data_center" { default = "SDDC-Datacenter" }
variable "cluster" { default = "Cluster-1" }
variable "workload_datastore" { default = "WorkloadDatastore" }
variable "compute_pool" { default = "Compute-ResourcePool" }

variable "vsphere_user" {}
variable "vsphere_password" {}
variable "vsphere_server" {}

And a “terraform.tfvars” file, with my security credentials:

vsphere_user     = "cloudadmin@vmc.local"
vsphere_password = "XXXXXXXXXXXXXXXXX"
vsphere_server   = "vcenter.sddc-A-B-C-D.vmwarevmc.com"

I’ve explained this in previous Terraform posts what each file represents so let’s focus on one specific painful aspect of remote OVA/OVF provisioning:

Host_system_id

The way the provider is currently built is less than optimal: users have to specify the host_system_id when deploying a VM from a remote OVA:

data "vsphere_host" "host" {
  name          = "10.10.10.68"
  datacenter_id = data.vsphere_datacenter.dc.id
}

resource "vsphere_virtual_machine" "vmFromRemoteOvf" {
  name             = "vm-deployed-from-ova"
  //
  host_system_id   = data.vsphere_host.host.id

What we were doing here is looking up the host whose name is “10.10.10.68”, using the data block:

This host is known to Terraform as “host”. As we have to specify – with host_system_id – the id of the host in which we will deploy the VM, we will then use the data.vsphere_host.host.id.

It’s a frustrating limitation right now to have to specify it – in theory, we should only need to specify the resource pool. My understanding is that it’s a limitation with govnomi (the Go library for vSphere).

Anyway, while these couple of limitations exist, it’s still a nice new feature to have and both my VMs are successfully deployed:

Nested ESXi
Photon OS deployed from Terraform

As always, I like to finish with a short video to show it being deployed live:

Thanks for reading!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s