Consume the HashiCorp Cloud Platform APIs with Go

In my previous post, I had a first look at HashiCorp Cloud Platform (HCP) Packer. At the end of the post, I suggested that the value of HCP Packer will be multiplied when integrated in an API-driven architecture.

The examples I used were:

• vulnerability is detected on an image → API-driven process to check which iterations have the package or OS with the vulnerability → alert users using the vulnerable image
• vulnerability is detected on an image → API-driven process to check which iterations have the package or OS with the vulnerability → automatically update the labels on HCP Packer with a “vulnerable” flag
• new base image release → automatic build of test image → automatic update of AMI used by Terraform in image testing environment

HCP Packer in isolation cannot do all this of course but once you start using its APIs, well, it gets interesting. In this post, I will create a very simple client in Go that interacts with the HCP Packer APIs.

Before I start using the APIs, a good starting point would be reading the API Docs here.

It’s been a few months since I’ve written any Go – last time it was when I was writing new resources for the VMware NSX provider. The HCP Go SDKs are already published here.

There’s of course another Go-based tool that uses these APIs: the Terraform provider for HCP! But as much as I like Terraform, this wasn’t the right tool for some of the use cases I had in mind.

Another good starting point for a Go client that leverages the HCP APIs is the example here. However I am going to write my own example that leverages the HCP Packer APIs.

Set-up

• Go 1.17
• VSCode

I have frankly no idea how anyone would code without tools like VSCode. It does so much of the heavy lifting for me as you will see shortly.

Results

This is what my basic client will do when I run it: it tells me the name, ID and description of the HCP Packer Buckets, details about the latest Packer iteration (authorID, fingerprint) and about the image itself (AMI ID and which region and when it was built):

The full code is this repo.

Code Walkthrough

Let’s go through the code in details.

package main

This is required to turn the Go code into an executable (think “.exe” in Windows). That’s how I executed the short piece of code further above (./hcp-sdk-go-client-nico)

import (
	"fmt"
	"log"
	"os"

	packer "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client/packer_service"

	"github.com/hashicorp/hcp-sdk-go/httpclient"
)

Next, we need to import packages and libraries. Go is by nature an elegant and efficient programming language. For example, Go will not let you import libraries that you will not use later on. Along the same lines, Go will not be happy if you create a variable and don’t use it.

It makes your code much cleaner and faster.

The “fmt”, “log” and “os” packages will enable us to respectively:

• Print output
• Log error messages
• Use the environment variables

The packer "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client/packer_service" line enables us to import the Go SDK for HCP Packer. The packer in front of the URL creates an alias and enables us to refer to these SDKs later on in the code by packer.function.

The HTTP Client SDK will be used to communicate with the HCP platform.

Let’s now look at the main body of the function:

func main() {
    // Initialize SDK http client
	cl, err := httpclient.New(httpclient.Config{})
	if err != nil {
		log.Fatal(err)
	}

	// Import versioned client for each service.
	packerClient := packer.New(cl, nil)

	// These IDs can be obtained from the portal URL
	orgID := os.Getenv("HCP_ORGANIZATION_ID")
	projID := os.Getenv("HCP_PROJECT_ID")
	bucketSlug := os.Getenv("HCP_BUCKET_SLUG")

We call our HTTP Client and create two objects: cl (client itself) and err (the error message it might generate if any exceptions occur). It’s extremely common in Go and we will use the same structure later.

We then initiate a packerClient and we pull some environment variables. It’s expected that the user will have set these on his/her environment before running the command.

The required environment variables are:

• HCP_ORGANIZATION_ID
• HCP_PROJECT_ID
• HCP_BUCKET_SLUG
• HCP_CLIENT_ID
• HCP_CLIENT_SECRET

The user would need to run commands such as these to set the variables:

export HCP_CLIENT_ID="service-principal-key-client-id"
export HCP_CLIENT_SECRET="service-principal-key-client-secret"

Now that we’ve got our parameters & credentials, we are ready to use our clients.

If I want to get information about the HCP Packer Bucket, we will need the PackerServiceGetBucket function from the packerClient. The function requires parameters as an input: it’s an object of the type PackerServiceGetBucketParams:

/*PackerServiceGetBucketParams contains all the parameters to send to the API endpoint
for the packer service get bucket operation typically these are written to a http.Request
*/
type PackerServiceGetBucketParams struct {

	/*BucketID
	  Unique identifier of the bucket; created and set by the HCP Packer
	registry when the bucket is created.

	*/
	BucketID *string
	/*BucketSlug
	  Human-readable name for the bucket.

	*/
	BucketSlug string
	/*LocationOrganizationID
	  organization_id is the id of the organization.

	*/
	LocationOrganizationID string
	/*LocationProjectID
	  project_id is the projects id.

	*/
	LocationProjectID string
	/*LocationRegionProvider
	  provider is the named cloud provider ("aws", "gcp", "azure").

	*/
	LocationRegionProvider *string
	/*LocationRegionRegion
	  region is the cloud region ("us-west1", "us-east1").

	*/
	LocationRegionRegion *string

	timeout    time.Duration
	Context    context.Context
	HTTPClient *http.Client
}

I repeat this in all my Go posts but it’s essential: Go is a strongly-typed language, which means any language element that stores a value has a type associated with it. In this instance, the PackerServiceGetBucketParams is an element that has a very specific structure. It’s mainly, as you see above, a collection of key/value pairs, with the values being strings, such as the HCP Organization ID, Project ID or Bucket Slug.

Next, we create listParamsA as an object of the type PackerServiceGetBucketParams, and then we set the values of listParamsA to the environmental variables.

We then call the client and store the response and its error (if any) as respA and errA.

Finally, we extract the latest iteration from the response:

	listParamsA := packer.NewPackerServiceGetBucketParams()
	listParamsA.LocationOrganizationID = orgID
	listParamsA.LocationProjectID = projID
	listParamsA.BucketSlug = bucketSlug
	respA, errA := packerClient.PackerServiceGetBucket(listParamsA, nil)
	latestIterationID := respA.Payload.Bucket.LatestIteration.ID

By the way, how did I know that the PackerServiceGetBucket was going to ask for an PackerServiceGetBucketParams object? And how I did work out the structure of this object? Well, this is where VSCode is so useful. It almost feels like I’m cheating as it gives us all the answers:

Finally, we want to print the output of the response.

respA is of the type:

type PackerServiceGetBucketOK struct {
	Payload *models.HashicorpCloudPackerGetBucketResponse
}

respA.Payload is based on this model:

type HashicorpCloudPackerGetBucketResponse struct {

	// The requested information about the bucket.
	Bucket *HashicorpCloudPackerBucket `json:"bucket,omitempty"`
}

In turn, respA.Payload.Bucket follows the HashiCorpCloudPackerBucket structure:

type HashicorpCloudPackerBucket struct {

	// When the bucket was created.
	// Format: date-time
	CreatedAt strfmt.DateTime `json:"created_at,omitempty"`

	// A short description of what this bucket's images are for.
	Description string `json:"description,omitempty"`

	// Unique identifier of the bucket; created and set by the HCP Packer
	// registry when the bucket is created.
	ID string `json:"id,omitempty"`

	// The total number of iterations in this bucket.
	IterationCount string `json:"iteration_count,omitempty"`

	// A key:value map for custom, user-settable metadata about your bucket.
	Labels map[string]string `json:"labels,omitempty"`

	// The bucket's most recent iteration -- this iteration may be complete or
	// not
	LatestIteration *HashicorpCloudPackerIteration `json:"latest_iteration,omitempty"`

	// The human-readable version of the most recent completed iteration in
	// this bucket.
	LatestVersion int32 `json:"latest_version,omitempty"`

	// HCP-specific information like project and organization ID
	Location *cloud.HashicorpCloudLocationLocation `json:"location,omitempty"`

	// A list of which cloud providers or other platforms the bucket contains
	// builds for. For example, AWS, GCP, or Azure.
	Platforms []string `json:"platforms"`

	// Human-readable name for the bucket.
	Slug string `json:"slug,omitempty"`

	// When the bucket was last updated.
	// Format: date-time
	UpdatedAt strfmt.DateTime `json:"updated_at,omitempty"`
}

By using respA.Payload.Bucket.x, we extract the Bucket values and display them on the terminal:

	if errA != nil {
		log.Fatal(errA)
	}

	if len(respA.Payload.Bucket.ID) > 0 {
		fmt.Println("### HCP Bucket Slug ###")
		fmt.Printf(respA.Payload.Bucket.Slug)
		fmt.Println()
		fmt.Println("### HCP Bucket ID ###")
		fmt.Printf(respA.Payload.Bucket.ID)
		fmt.Println()
		fmt.Println("### HCP Bucket Description ###")
		fmt.Printf(respA.Payload.Bucket.Description)
		fmt.Println("############################")
		fmt.Println("### HCP Latest Iteration ###")
		fmt.Printf(respA.Payload.Bucket.LatestIteration.ID)
		fmt.Println()

	} else {
		fmt.Printf("Response: %#v\n\n", respA.Payload.Bucket.ID)
	}

The rest of the script is very similar. We make a different API call and fetch the details of different objects but otherwise, it’s the same process.

It’s time to compile the code:

go build

And finally, it’s time to execute it.

First, I need to make sure I add all the required environmental variables. The OrgID and Project ID are below:

The Bucket Slug is the name of the bucket: “learn-packer-ubuntu”.

Let’s set the environment variables:

export HCP_ORGANIZATION_ID=0b1613b0-d488-4225-9313-456d0694b2e2
export HCP_PROJECT_ID=626d56e6-8dc2-4631-a682-900a475110f5
export HCP_BUCKET_SLUG=learn-packer-ubuntu
export HCP_CLIENT_ID=**********
export HCP_CLIENT_SECRET=*******

I can now run my basic client:

%./hcp-sdk-go-client-nico 
### HCP Bucket Slug ###
learn-packer-ubuntu
### HCP Bucket ID ###
01FJHGFRMH75M86SX33G9VN21H
### HCP Bucket Description ###
Our Ubuntu-based Golden Image
############################
### HCP Latest Iteration ###
01FJJCMGFMB4010AE8PNA3FTEX
### HCP Iteration AuthorID ###
packer
### HCP Iteration Ancestor ID ###

### HCP Iteration Fingerprint  ###
5b6fde77367480882bdf69d4a02da9da09ddc03e
### Image ID ###
ami-0fe15aef3fb965734
### Image Creation Date ###
2021-10-21T21:38:11.405Z
### Image Region ###
us-east-2

That’s it ! As you can see, the code and its output are pretty simple but this hopefully gives you an idea of what can be done with basic knowledge of Go, a great tool like VSCode and some-well documented APIs.

Thanks for reading.