Terraform and Splunk Part 3: Using the Terraform Provider for Splunk

This is the third post in my Terraform and Splunk series. After deploying Splunk with Terraform in Part 1 and setting up the Terraform Cloud integration in Splunk in Part 2, we are going to end this series for now by using Terraform to configure Splunk to monitor Terraform Cloud.

For this, we are going to be the recently-launched Terraform Provider for Splunk.

Setting up the provider

It’s pretty straight-forward to get started with the Splunk provider. I think there are some options to use authentication tokens but I just used admin/password:

terraform {
  required_providers {
    splunk = {
      source = "splunk/splunk"
      //version = "1.0.0"

provider "splunk" {
  url      = "A.B.C.D:8089"
  username = "admin"
  password = "SPLUNK-i-0aa000000000aaa000"

Note that API calls are made over port 8089 by default.

We can then start creating dashboards and searches.

I can refer to the Search I published in Part 2:

resource "splunk_saved_searches" "saved" {
  name                      = "Sentinel_Hard_Failed_Search"
  search                    = <<EOT
  source="terraform_cloud" sourcetype="terraform_cloud" resource.action="hard_failed"
| spath resource.meta.run.id output=Run 
| spath auth.description output=User
| spath resource.meta.sentinel.data.sentinel-policy-networking.policies{} output=policies 
| mvexpand policies 
| spath input=policies trace.rules.main.position.filename output=filename 
| spath input=policies result output=value 
| where value="false" AND match(filename,"^\..hard") 
| spath input=policies trace.print output=error_message 
| rename filename AS Policy, error_message AS Log, timestamp AS Time
| table User Run Time Policy Log


The configuration below will pre-load the Search: this is handy – it means the Splunk administrator can create easy saved searches or dashboards for the consumers of Splunk:

Saved Search

You can then press “Add to Search” and run it and get the results we saw in the previous post.

Saved Search Created with TF

Alternatively, you can also create dashboards. The example in the docs is simple enough:

resource "splunk_data_ui_views" "dashboard" {
  name     = "Terraform_Test_Dashboard"
  eai_data = <<EOF
      Terraform Test Dashboard
  acl {
    owner = "admin"
    app = "search"

If you look at my repo, I found the XML code that describes the dashboard in the official TF Cloud for Splunk app and copy/pasted it into my Terraform code.

Dashboard XML

I recommend you also check out the Splunk Dashboard Examples app – again, you can copy the XML code and use it as you see fit: you mainly need to replace the search query with the one you want to visualize.

That’s all for now – thanks for checking out the Terraform & Splunk series.


2 thoughts on “Terraform and Splunk Part 3: Using the Terraform Provider for Splunk

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s