In this third post in my HCP Packer series (you can also read Part 1 (Getting Started with HCP Packer) and Part 2 (Building a Go Client for HCP Packer)), I am going to cover some new features that have only just come out, as part of the General Availability of HCP Packer.
I had seen some mock-ups whilst in conversations with product managers and customers and I had been looking forward to this release. Admittedly, there are a few more things I would want from a service like HCP Packer but it’s a very good start.
In previous posts, I walked through what HCP Packer is (an image metadata registry service) and how it integrates nicely with Terraform. The integration has gone further with the update last week. Let’s start with a quick recap:
Packer is a great tool to build “golden” images – templates from which your actual machines are built upon – and is very commonly used by cloud providers and users alike: it’s pretty much the standard tool to build images (and one of my first blog posts here looked at using Packer for VMware).
But what Packer does not do natively is track the created images.
Yes, as Packer builds the images from code, you can track the changes between each golden image from reviewing the code history but you can’t easily work out where these images have been deployed, which have been revoked and why and for which platform the images have been created.
What ends up happening, when a vulnerability on an image is found, are three things:
- we don’t know where these images have been deployed
- some users might still deploy – usually through Terraform – obsolete images.
- we tend to patch the server – in a mutable pattern – instead of building a brand new machine from a clean template (in an immutable pattern).
HCP Packer has been built to address these challenges. I won’t go through in details what I talked about in the first post but essentially when you use Packer to build an image, you can record its status and output in HCP Packer (for example, the AWS AMI ID).
HCP Packer is built around some key concepts: buckets (a logical repo of images), iterations (iterations of when Packer was executed, each identified by the Git commit hash) and channels (referring to where this image might be used – for example, “prod”, “test”, “dev”).
You would assign an iteration (usually the latest, with the right software package and patches) to a channel and Terraform would build applications using the required channel image.
What was just announced last week is the ability to revoke an image.
That can be done immediately or in the future – essentially that’s a way to have a pre-determined lifespan for an image (I have customers who release images on a monthly basis – I am sure some do it more frequently and some less).
Why would you revoke an image? Well, if the image – or the image it is based upon as most customers I speak to have multiple layers to make up their golden image – has a vulnerability, then we need to not only build a new image but also prevent users from deploying the revoked one.
Note in the screenshot below that ami-0f0aee0f3b2867ea6 has now been revoked. We’re going to see what happens later on when I try to build a machine based on this image.
HCP Packer becomes particularly useful with Terraform Cloud through something called Run Tasks. Terraform Cloud introduced the use of ‘Run Tasks‘ last year – it’s a way to insert some logic and analysis done from a 3rd party during the Terraform execution. In our case, the 3rd party is going to be HCP Packer.
Sentinel is probably the first example of Run Tasks but there are other integrations, such as with security vendors Snyk and Bridgecrew.
To set this up, you’re first going to need some details from HCP Packer to authenticate to it:
Input these parameters in Terraform Cloud >> Settings >> Run tasks to create your Run Task:
Enable the Run Task on your Terraform Cloud workspace:
And when you start a Terraform Run, the Run Task will be executed between terraform plan and terraform apply:
Let’s have a look at what happens when I tried to deploy a VM based on an image not tracked by HCP Packer. My Terraform code is as simple as this:
resource "aws_instance" "app_server" {
ami = "ami-01abd5d1c4bc01cbf"
instance_type = "t2.micro"
tags = {
Name = "Image Not Tracked by HCP"
}
}
Terraform Cloud will connect to HCP Packer during the Run Task execution and will realize that the image is not tracked and managed by HCP Packer As it’s not a big deal, TFC just advises us we should ideally track this with HCP Packer:
Now what I found pretty cool is when I try to build an EC2 instance based on an image (ami-0f0aee0f3b2867ea6) I previously revoked:
There are two enforcement levels for the Run Task – Advisory or Mandatory. As I selected “Mandatory”, you can see in the logs above that TFC will block the deployment as we’re using a revoked image. It also informs us that there is a more recent version of the image (from a more recent iteration).
This is cool.
Yes, I am sure there are ways to hack a solution together around Sentinel and some artefact or a spreadsheet but it wouldn’t be particularly elegant.
Yes, I am also certain you could build a pipeline to get the same outcome and that’s fine! There are always many ways and tools to achieve the goal. I just happen to like tools that integrate well together.
To recap, this feature automatically prevents a user from deploying unsecure or obsolete images. If you forgive me the use of this buzz word, but this is a nice example on how to “shift left“.
Again, this has only gone GA last week so there aren’t a ton of features but I think folks will find this pretty handy.
What I would also like to see (and the HashiCorp Product Managers have already heard my feature requests!) :
- Improved user interface on the TFC Run Tasks (the message above is a bit small and hard to read and some pictures would be welcome)
- Automatic scanning of images via an integration with a 3rd party
- Inventory of where the golden image has been deployed and when and by whom: this is probably something Terraform Cloud could show you by doing a research across workspaces states
- Automatically trigger a re-run of Terraform to re-build VM based on the new VMs (in a real “immutable infrastructure” model).
More to come but it’s a great start.
Thanks for reading.
nicovibert.com 