One of the advantages of working with Open Source software is that, when you are confused by some of the products or when the documentation hasn’t been updated, you can literally go and check the code.
I realized recently that support for vSphere Content Library had been added to the Terraform provider for vSphere but the documentation was scarce and I wasn’t sure how it was implemented and what was possible.
A quick look at the Terraform vSphere provider changelogs (nice tip I heard on a Ned daily check-in) led me to this GitHub page where you could look at the code changes. While I can’t code in Go, I worked out that you now can:
- Create a Content Library
- Subscribe to an existing one or publish one
- Specify whether you want to synchronize the content library and/or whether you want to download all the content or only want it when needed (“on-demand” setting)
- Add an item to the content library you are creating through Terraform or, using a data block, add an item to an existing content library
- Clone a VM from a Content Library item
This is a huge improvement for any Terraform for vSphere user and something I would in theory probably much rather use than the OVF/OVA remote deploy option I described in a previous blog. As you will see later in the blog though, there are still some imperfections.
Let’s go through how we do it.
Tutorial
I’d rather over-explain than under-explain so let’s review the configuration requirements for Terraform:
- main.tf describing your configuration
- variables.tf describing your variables
- terraform.tfvars describing your secrets and passwords.
Let’s start with variables.tf:
variable "data_center" { default = "SDDC-Datacenter" }
variable "cluster" { default = "Cluster-1" }
variable "workload_datastore" { default = "WorkloadDatastore" }
variable "compute_pool" { default = "Compute-ResourcePool" }
variable "vsphere_user" {}
variable "vsphere_password" {}
variable "vsphere_server" {}
variable "Subnet13_name" { default = "segment13" }
variable "subnet13" { default = "13.13.13.0/24"}
As you will see later, we will create a VM on the “subnet13” network (previously created by Terraform for NSX-T but could have been created by another means) and we will deploy it in the datastore specified above.
The passwords and details are stored in a separate terraform.tfvars file. For example:
vsphere_user = "cloudadmin@vmc.local"
vsphere_password = "s3cr3tpassw0rd"
vsphere_server = "vcenter.sddc-A-B-C-D.vmwarevmc.com"
Now we will describe our configuration in the main.tf file (feel free to read through the many Terraform posts in my blog for more details):
provider "vsphere" {
user = var.vsphere_user
password = var.vsphere_password
vsphere_server = var.vsphere_server
allow_unverified_ssl = true
}
data "vsphere_datacenter" "dc" {
name = var.data_center
}
data "vsphere_compute_cluster" "cluster" {
name = var.cluster
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_datastore" "datastore" {
name = var.workload_datastore
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_resource_pool" "pool" {
name = var.compute_pool
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_network" "network" {
name = "sddc-cgw-network-1"
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_network" "network13" {
name = var.Subnet13_name
datacenter_id = data.vsphere_datacenter.dc.id
}
resource "vsphere_content_library" "library" {
name = "Content Library Test"
storage_backing = [data.vsphere_datastore.datastore.id]
description = "A new source of content"
}
resource "vsphere_content_library" "subscribedlibrary_terraform" {
name = "New Subscribed Library by Terraform"
storage_backing = [data.vsphere_datastore.datastore.id]
description = "Another new source of content"
subscription {
subscription_url = "https://s3-us-west-2.amazonaws.com/s3-vmc-iso/lib.json"
authentication_method = "NONE"
automatic_sync = true
on_demand = false
}
}
data "vsphere_content_library_item" "library_item_photon" {
name = "Photon"
library_id = vsphere_content_library.subscribedlibrary_terraform.id
type = "OVA"
}
resource "vsphere_virtual_machine" "vm_terraform_from_cl" {
name = "vm_terraform_from_cl"
resource_pool_id = data.vsphere_resource_pool.pool.id
datastore_id = data.vsphere_datastore.datastore.id
folder = "Workloads"
num_cpus = 2
memory = 1024
guest_id = "other3xLinux64Guest"
network_interface {
network_id = data.vsphere_network.network13.id
}
disk {
label = "disk0"
size = 20
thin_provisioned = true
}
clone {
template_uuid = data.vsphere_content_library_item.library_item_photon.id
customize {
linux_options {
host_name = "Photon"
domain = "vmc.local"
}
network_interface {
ipv4_address = cidrhost(var.subnet13, 200) #fixed IP address .200
ipv4_netmask = 24
}
ipv4_gateway = cidrhost(var.subnet13, 1)
}
}
}
Remember that with Terraform, a data block will look at an already configured resource and a resource block will look at create and manage a resource.
This is what the configuration above does:
- Check the existing vCenter and various components (like datastore, cluster, folder, network)
- Creates an empty local content library called “Content Library Test”
- Creates a content library called “Subscribed Library” and will subscribe to a Content Library hosted on S3 (not officially supported configuration…yet).
- Pulls the details (UUID) of an OVA called “Photon” from the newly configured content library
- Deploy a VM from the Photon OVA
Pretty cool, right? Let’s look at it in action.
Assuming you have already installed Terraform, once you have the three files in a folder, initialize Terraform with terraform init to download and install the latest vSphere provider (Terraform looked at the main.tf file and saw that “vSphere” was the specified provider):
nvibert-a01:Terraform-Content-Library nicolasvibert$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/vsphere...
- Installing hashicorp/vsphere v1.24.1...
- Installed hashicorp/vsphere v1.24.1 (signed by HashiCorp)
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.
* hashicorp/vsphere: version = "~> 1.24.1"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
A “terraform validate” tells me the syntax is correct:
nvibert-a01:Terraform-Content-Library nicolasvibert$ terraform validate
Success! The configuration is valid.
Now, the “terraform plan” command will tell me what it’s going to create and manage:
nvibert-a01:Terraform-Content-Library nicolasvibert$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.vsphere_datacenter.dc: Refreshing state...
data.vsphere_network.network13: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_compute_cluster.cluster: Refreshing state...
data.vsphere_resource_pool.pool: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
<= read (data resources)
Terraform will perform the following actions:
# data.vsphere_content_library_item.library_item_photon will be read during apply
# (config refers to values not yet known)
<= data "vsphere_content_library_item" "library_item_photon" {
+ id = (known after apply)
+ library_id = (known after apply)
+ name = "Phothon"
+ type = "OVA"
}
# vsphere_content_library.library will be created
+ resource "vsphere_content_library" "library" {
+ description = "A new source of content"
+ id = (known after apply)
+ name = "Content Library Test"
+ storage_backing = [
+ "datastore-48",
]
+ publication {
+ authentication_method = (known after apply)
+ password = (known after apply)
+ publish_url = (known after apply)
+ published = (known after apply)
+ username = (known after apply)
}
}
# vsphere_content_library.subscribedlibrary will be created
+ resource "vsphere_content_library" "subscribedlibrary" {
+ description = "Another new source of content"
+ id = (known after apply)
+ name = "Subscribed Library"
+ storage_backing = [
+ "datastore-48",
]
+ publication {
+ authentication_method = (known after apply)
+ password = (known after apply)
+ publish_url = (known after apply)
+ published = (known after apply)
+ username = (known after apply)
}
+ subscription {
+ authentication_method = "NONE"
+ automatic_sync = true
+ on_demand = false
+ password = (known after apply)
+ subscription_url = "https://s3-us-west-2.amazonaws.com/s3-vmc-iso/lib.json"
+ username = (known after apply)
}
}
# vsphere_virtual_machine.vm_terraform_from_cl will be created
+ resource "vsphere_virtual_machine" "vm_terraform_from_cl" {
+ boot_retry_delay = 10000
+ change_version = (known after apply)
+ cpu_limit = -1
+ cpu_share_count = (known after apply)
+ cpu_share_level = "normal"
+ datastore_id = "datastore-48"
+ default_ip_address = (known after apply)
+ ept_rvi_mode = "automatic"
+ firmware = "bios"
+ folder = "Workloads"
+ force_power_off = true
+ guest_id = "ubuntu64Guest"
+ guest_ip_addresses = (known after apply)
+ hardware_version = (known after apply)
+ host_system_id = (known after apply)
+ hv_mode = "hvAuto"
+ id = (known after apply)
+ ide_controller_count = 2
+ imported = (known after apply)
+ latency_sensitivity = "normal"
+ memory = 1024
+ memory_limit = -1
+ memory_share_count = (known after apply)
+ memory_share_level = "normal"
+ migrate_wait_timeout = 30
+ moid = (known after apply)
+ name = "vm_terraform_from_cl"
+ num_cores_per_socket = 1
+ num_cpus = 2
+ poweron_timeout = 300
+ reboot_required = (known after apply)
+ resource_pool_id = "resgroup-47"
+ run_tools_scripts_after_power_on = true
+ run_tools_scripts_after_resume = true
+ run_tools_scripts_before_guest_shutdown = true
+ run_tools_scripts_before_guest_standby = true
+ sata_controller_count = 0
+ scsi_bus_sharing = "noSharing"
+ scsi_controller_count = 1
+ scsi_type = "pvscsi"
+ shutdown_wait_timeout = 3
+ storage_policy_id = (known after apply)
+ swap_placement_policy = "inherit"
+ uuid = (known after apply)
+ vapp_transport = (known after apply)
+ vmware_tools_status = (known after apply)
+ vmx_path = (known after apply)
+ wait_for_guest_ip_timeout = 0
+ wait_for_guest_net_routable = true
+ wait_for_guest_net_timeout = 5
+ clone {
+ template_uuid = (known after apply)
+ timeout = 30
+ customize {
+ ipv4_gateway = "13.13.13.1"
+ timeout = 10
+ linux_options {
+ domain = "vmc.local"
+ host_name = "Photon"
+ hw_clock_utc = true
}
+ network_interface {
+ ipv4_address = "13.13.13.200"
+ ipv4_netmask = 24
}
}
}
+ disk {
+ attach = false
+ controller_type = "scsi"
+ datastore_id = "<computed>"
+ device_address = (known after apply)
+ disk_mode = "persistent"
+ disk_sharing = "sharingNone"
+ eagerly_scrub = false
+ io_limit = -1
+ io_reservation = 0
+ io_share_count = 0
+ io_share_level = "normal"
+ keep_on_remove = false
+ key = 0
+ label = "disk0"
+ path = (known after apply)
+ size = 20
+ storage_policy_id = (known after apply)
+ thin_provisioned = true
+ unit_number = 0
+ uuid = (known after apply)
+ write_through = false
}
+ network_interface {
+ adapter_type = "vmxnet3"
+ bandwidth_limit = -1
+ bandwidth_reservation = 0
+ bandwidth_share_count = (known after apply)
+ bandwidth_share_level = "normal"
+ device_address = (known after apply)
+ key = (known after apply)
+ mac_address = (known after apply)
+ network_id = "network-o1004"
}
}
Plan: 3 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
It’s pretty much human-readable so you can see what Terraform is going to create.
As mentioned before, Terraform creates an empty library. Be aware of the little gotcha here: you have to specify the datastore in square brackets (it’s an array of datastores).
data "vsphere_datastore" "datastore" {
name = var.workload_datastore
datacenter_id = data.vsphere_datacenter.dc.id
}
resource "vsphere_content_library" "library" {
name = "Content Library Test"
storage_backing = [data.vsphere_datastore.datastore.id]
description = "A new source of content"
}
If you want to add items to the content library, you can use the resource “vsphere_content_library_item” to specify the VM you want the item to be based on.
resource "vsphere_content_library_item" "cl_item_vm" {
name = "CL VM"
library_id = vsphere_content_library.library.id
source_uuid = data.vsphere_virtual_machine.photo.id
}
/*=================================================================
Get Template data
==================================================================*/
data "vsphere_virtual_machine" "photo" {
name = "PhotoApp"
datacenter_id = data.vsphere_datacenter.dc.id
}
As an aside, you can also use existing Content Libraries created over the UI or another tool than Terraform by using the data block and the following configuration:
data "vsphere_content_library" "existing_library" {
name = "CL-under-control"
}
resource "vsphere_content_library_item" "cl_item_vm2" {
name = "CL_VM"
library_id = data.vsphere_content_library.existing_library.id
source_uuid = data.vsphere_virtual_machine.photo.id
}
Terraform also created a new Terraform Library and subscribed to an existing one.
resource "vsphere_content_library" "subscribedlibrary_terraform" {
name = "New Subscribed Library by Terraform"
storage_backing = [data.vsphere_datastore.datastore.id]
description = "Another new source of content"
subscription {
subscription_url = "https://s3-us-west-2.amazonaws.com/s3-vmc-iso/lib.json"
authentication_method = "NONE"
automatic_sync = true
on_demand = false
}
}
You can see that automatic synchronization is enabled and that the entire library is automatically downloaded as I specified in my Terraform configuration. Pretty neat.
Now what you would expect is that the VM should be deployed, based on the template I specified in the code:
resource "vsphere_virtual_machine" "vm_terraform_from_cl" {
name = "vm_terraform_from_cl"
resource_pool_id = data.vsphere_resource_pool.pool.id
datastore_id = data.vsphere_datastore.datastore.id
folder = "Workloads"
num_cpus = 2
memory = 1024
guest_id = "other3xLinux64Guest"
network_interface {
network_id = data.vsphere_network.network13.id
}
disk {
label = "disk0"
size = 20
thin_provisioned = true
}
clone {
template_uuid = data.vsphere_content_library_item.library_item_photon.id
customize {
linux_options {
host_name = "Photon"
domain = "vmc.local"
}
network_interface {
ipv4_address = cidrhost(var.subnet13, 200) #fixed IP address .200
ipv4_netmask = 24
}
ipv4_gateway = cidrhost(var.subnet13, 1)
}
}
}
Problem is that it worked intermittently – sometimes the VM was created as cleanly as this:
Sometimes, I ran into some issues which I think is due to the fact (I think) that Terraform would try to deploy the VM before the content library sync is completed and I would get an error message such as:
vsphere_virtual_machine.vm_terraform_from_cl: Creating...
Error: 400 Bad Request: {"type":"com.vmware.vapi.std.errors.invalid_argument","value":{"error_type":"INVALID_ARGUMENT","messages":[{"args":[],"default_message":"Specified library item is not an OVF.","id":"com.vmware.ovfs.ovfs-main.ovfs.invalid_library_item"}]}}
on main.tf line 59, in resource "vsphere_virtual_machine" "vm_terraform_from_cl":
59: resource "vsphere_virtual_machine" "vm_terraform_from_cl" {
When the image has been downloaded and I tried to run “terraform apply” again, the image was successfully deployed:
vsphere_virtual_machine.vm_terraform_from_cl: Creating...
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [10s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [20s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [30s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [40s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [50s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [1m0s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [1m10s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Still creating... [1m20s elapsed]
vsphere_virtual_machine.vm_terraform_from_cl: Creation complete after 1m22s [id=422acb3e-ad7a-727c-27bb-27835c9de895]
There seems to be a timing issue. Please let me know if you also ran into the issue and we’ll try to get this improved. Or let me know if you spot any mistakes in my configuration.
Thanks for reading.
nicovibert.com 
Hi Nico,
Have quick question, when the server is deploying from the CL using one of the template disk type is by default coming up as thick for the root disk in the server, for data disk when we specify type as thin, it provisions as thin device only.I am looking for an option to see if we can specify the disk type to be as thin for root disk while deploying, can you please let me know if you have come across this.
Thanks,
-Yash
LikeLiked by 1 person