Hi,
I have some questions on Terraform Cloud integration using Jenkins + Terraform remote backend.
terraform plan using the remote backend?Hello!
There are no sample Jenkinsfiles (yet! hoping to get to generating some) but there is a blog on how to use Jenkins with Terraform Enterprise. You can likely find material for Jenkins + Terraform Enterprise and the same approaches will apply to Terraform Cloud. Generally, when using TFC with CI frameworks, you will want to set the workspace to execute remotely while the CI framework integrates with VCS. As a result, you would call terraform plan and terraform apply within your Jenkinsfile with remote execution on Terraform Cloud (no VCS integration set up).
At this time, we do not have a way to disable workspace runs per commit (something like [ci skip], for example). You can disable workspace runs for all commits by resetting the trigger to a different directory upon changes or removing the VCS integration and setting up the CI Framework as outlined above.
To do this, you’ll need to set up TFC with remote plan, no VCS integration. The backend configuration for the workspace must be a file specified by Jenkins or version control.
There are 2 approaches:
terraform plan, remote plan by Terraform Cloud.input waits for manual approvalterraform apply -auto-approve, remote apply by Terraform Cloud.terraform apply, remote apply by Terraform. This will hang until there is approval in Terraform Cloud. Be sure to set the timeout here to a long timeout for the job.Yes, triggering a speculative plan in PRs would be a terraform plan with the remote backend.
Hope this is somewhat helpful!
Hope this can be of some use… here’s the relevant bits of a Jenkinsfile that uses local apply (not remote apply) that can be used with the TFC remote state backend.
It’s a declarative Jenkinsfile for use with multibranch pipeline jobs. When there’s a new commit to a branch a plan is run and if there are any pending changes it will prompt for approval to run an apply, similar to how it was described above.
Couple things worth noting are the use of the milestone and lockable resources features in the pipeline to resolve concurrency issues if say, multiple devs push stuff within a short amount of time. Any superseded pipeline runs are cancelled and in the case you hit a lock, other waiting (newer) runs will wait for this one to complete.
pipeline {
agent {
...
}
options {
ansiColor('xterm')
timestamps()
}
tools {
type: 'org.jenkinsci.plugins.terraform.TerraformInstallation' 'terraform'
}
environment {
TF_HOME = tool name: 'terraform', type: 'org.jenkinsci.plugins.terraform.TerraformInstallation'
PATH = "$TF_HOME:$PATH"
TF_VAR_some_sensitive_var_from_vault = vault path: "/kv/jenkins/path/to/secret", key: 'key'
... some other vars ...
}
stages {
stage('setup') {
steps {
container('executor') {
milestone 1
...some other setup stuff such as third party providers...
sh 'terraform version'
}
}
}
stage('init') {
when {
// use named env branches, not master
not {
branch 'master'
}
}
environment {
VAULT_ADDR = '...'
VAULT_CACERT = 'path/to/vault-ca.pem'
}
steps {
container('executor') {
milestone 2
lock(resource: env.BRANCH_NAME, inversePrecedence: true) {
sshagent(['jenkins-ssh']) {
sh 'terraform init'
}
// each non-master branch represents a desired environment -- we separate the state
// of these environments using Terraform workspaces. If the workspace doesn't already
// exist in the state, this is a new environment we're bootstrapping, so we create it.
sh 'terraform workspace select ${BRANCH_NAME} || terraform workspace new ${BRANCH_NAME}'
configFileProvider([configFile(fileId: 'vault-ca-cert', targetLocation: env.VAULT_CACERT)]) {
script {
def status = sh(script: '''
... some stuff we have for kubernetes auth ...
terraform plan -var-file "${BRANCH_NAME}.tfvars" -detailed-exitcode
''', returnStatus: true)
env.WANT_APPLY = status
if (status == 1) {
error("Failed to execute Terraform plan")
}
}
}
}
}
}
}
stage('apply') {
when {
allOf {
not {
// use named env branches, not master
branch 'master'
}
environment name: 'WANT_APPLY', value: '2'
}
}
environment {
VAULT_ADDR = '...'
VAULT_CACERT = 'path/to/vault-ca.pem'
}
steps {
container('executor') {
milestone 3
input message: 'Review plan and approve to proceed with apply. Ok to apply?'
lock(resource: env.BRANCH_NAME, inversePrecedence: true) {
configFileProvider([configFile(fileId: 'vault-ca-cert', targetLocation: env.VAULT_CACERT)]) {
// each non-master branch represents a desired environment -- we separate the state
// of these environments using Terraform workspaces. If the workspace doesn't already
// exist in the state, this is a new environment we're bootstrapping, so we create it.
sh 'terraform workspace select ${BRANCH_NAME} || terraform workspace new ${BRANCH_NAME}'
// the detailed exitcode option lets us finish without prompting the user if there are
// no changes to apply.
milestone 4
sh '''
... some stuff we have for kubernetes auth ...
terraform apply -var-file "${BRANCH_NAME}.tfvars" -auto-approve
'''
}
}
}
}
}
}
}
Hello friends, a little late to the party, but here’s a Jenkinsfile and tutorial to use jenkins to run a plan on Terraform Cloud
https://hashiqube.com/jenkins/README
// https://github.com/jenkinsci/hashicorp-vault-plugin
// https://www.jenkins.io/doc/book/pipeline/jenkinsfile/
import hudson.model.Job
import jenkins.scm.api.mixin.ChangeRequestSCMHead
import jenkins.scm.api.mixin.TagSCMHead
import org.jenkinsci.plugins.workflow.multibranch.BranchJobProperty
node {
properties([disableConcurrentBuilds()])
stage('Checkout https://github.com/star3am/terraform-hashicorp-hashiqube') {
git(
url: "https://github.com/star3am/terraform-hashicorp-hashiqube.git",
branch: "master",
changelog: true,
poll: true
)
}
stage('Echo Variables') {
echo "JOB_NAME: ${env.JOB_NAME}"
echo "BUILD_ID: ${env.BUILD_ID}"
echo "BUILD_NUMBER: ${env.BUILD_NUMBER}"
echo "BRANCH_NAME: ${env.BRANCH_NAME}"
echo "PULL_REQUEST: ${env.CHANGE_ID}"
echo "BUILD_NUMBER: ${env.BUILD_NUMBER}"
echo "BUILD_URL: ${env.BUILD_URL}"
echo "NODE_NAME: ${env.NODE_NAME}"
echo "BUILD_TAG: ${env.BUILD_TAG}"
echo "JENKINS_URL: ${env.JENKINS_URL}"
echo "EXECUTOR_NUMBER: ${env.EXECUTOR_NUMBER}"
echo "WORKSPACE: ${env.WORKSPACE}"
echo "GIT_COMMIT: ${env.GIT_COMMIT}"
echo "GIT_URL: ${env.GIT_URL}"
echo "GIT_BRANCH: ${env.GIT_BRANCH}"
LAST_COMMIT_MSG = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%s'")
echo "LAST_COMMIT_MSG: ${LAST_COMMIT_MSG}"
env.ARCH = sh(returnStdout: true, script: "lscpu | grep 'Architecture' | tr -s ' ' | cut -d ' ' -f 2 | tr -d '[:space:]'")
echo "ARCH: ${env.ARCH}"
env.PATH = "${env.PATH}:${env.WORKSPACE}/bin"
env.TF_CLI_ARGS = "-no-color"
echo sh(script: 'env|sort', returnStdout: true)
sh('echo $(hostname)')
}
stage('Create Backend Config for Terraform Cloud') {
withCredentials([string(credentialsId: 'YOUR_CREDENTIALS_ID', variable: 'SECRET')]) {
sh """
cat <<EOF | tee backend.tf
terraform {
cloud {
organization = "YOUR_TF_CLOUD_ORGANIZATION"
workspaces {
name = "YOUR_TF_WORKSPACE"
}
token = "${SECRET}"
}
}
EOF
"""
}
}
stage('Install Dependencies') {
sh """
pwd
mkdir -p bin
"""
if (env.ARCH == "x86_64*") {
script {
env.arch = "amd64"
echo "${env.arch}"
}
}
if (env.ARCH == 'aarch64') {
script {
env.arch = "arm64"
echo "${env.arch}"
}
}
sh """
curl -s "https://releases.hashicorp.com/terraform/1.5.7/terraform_1.5.7_linux_${env.arch}.zip" --output bin/terraform.zip
(cd bin && unzip -o terraform.zip)
curl -Lso bin/tfsec "https://github.com/aquasecurity/tfsec/releases/download/v1.28.4/tfsec-linux-${env.arch}"
chmod +x bin/tfsec
pwd
ls -la
ls -la bin/
terraform -v
echo "${env.arch}"
echo "${env.PATH}"
"""
}
stage('Run Aquasecurity TFSec') {
sh('tfsec ./modules --no-color --soft-fail')
}
stage('Run Terraform init') {
sh('terraform init')
}
stage('Run Terraform plan on Terraform Cloud') {
sh('terraform plan')
}
stage('Get ENV vars from Vault') {
// define the secrets and the env variables
// engine version can be defined on secret, job, folder or global.
// the default is engine version 2 unless otherwise specified globally.
def secrets = [
[path: 'kv2/secret/another_test', engineVersion: 2, secretValues: [
[vaultKey: 'another_test']]],
[path: 'kv1/secret/testing/value_one', engineVersion: 1, secretValues: [
[vaultKey: 'value_one']]],
[path: 'kv1/secret/testing/value_two', engineVersion: 1, secretValues: [
[envVar: 'my_var', vaultKey: 'value_two']]]
]
// optional configuration, if you do not provide this the next higher configuration
// (e.g. folder or global) will be used
def configuration = [vaultUrl: 'http://10.9.99.10:8200',
vaultCredentialId: 'vault-initial-root-token',
engineVersion: 1]
// inside this block your credentials will be available as env variables
withVault([configuration: configuration, vaultSecrets: secrets]) {
sh 'echo $value_one'
sh 'echo $my_var'
sh 'echo $another_test'
}
}
stage('Echo some ENV vars') {
withCredentials([[$class: 'VaultTokenCredentialBinding', credentialsId: 'vault-initial-root-token', vaultAddr: 'http://10.9.99.10:8200']]) {
// values will be masked
sh 'echo TOKEN=$VAULT_TOKEN'
sh 'echo ADDR=$VAULT_ADDR'
}
echo sh(script: 'env|sort', returnStdout: true)
}
}