How to Integrate Github and Terraform with Jenkins

In my last post I discussed the importance & usage of Terraform to create Cloud resources. In a DevOps environment where you have to manage various customer accounts, creation & removal of Cloud environments manually on daily basis seems a cumbersome task.

A proper Automation & CI/CD tool is a de facto when you want to automate daily manual configuration tasks in a seamless way & manage many Cloud environments(as jobs) at the same time. Jenkins is your answer to the above concern. It is an Open source Automation server to execute manual configurations as automated daily jobs.

I will show you how to integrate a Github repo with terraform code to run as a Jenkins pipeline. Lets start with the Integration:

First things first, Have an EC2 instance/vm ready with Jenkins installed & a github repo with terraform code

Configure Terraform Plugin:

1. Login to Jenkins and navigate to Manage Jenkins>Manage Plugins
2. In the filter, search for “Terraform Plugin” and install it
3. Now navigate to Manage Jenkins>Global Tools Configuration & check for Terraform Installations.
4. Click ono Add Terraform and give a name. Do not click “ Install automatically”.
5. SSH to the Jenkins vm and use following set of commands to install Terraform:

1. wget <terraform_linux_64_bit>
wget https://releases.hashicorp.com/terraform/0.14.3/terraform_0.14.3_linux_amd64.zip
2. unzip <xx.zip> 
unzip terraform_0.14.3_linux_amd64.zip
3. Move terraform file to '/usr/bin'
sudo mv terraform /usr/bin/
4. Use 'terraform --version' command to check if terraform is installed:
terraform --version
Terraform v0.14.3

6. Use ‘/usr/bin/’ in the Installation directory:

CloudBees AWS Credentials Plugin:

1. Again navigate to Manage Plugin & install “CloudBees AWS Credentials Plugin”

This plugin helps to bind AWS Access key ID & Secret Key to be used while configuring Terraform via Jenkins

Jenkins Pipeline: Last but lengthy step

1. Navigate to Dashboard> New Item. Enter a name, Select ‘Pipeline’ & click OK
2. Navigate directly to Pipeline section and create the following script step by step:

pipeline {
    agent any
    tools {
        terraform 'terraform'
        }
}

Here, ‘terraform’ is the name that you have provided during Terraform Installations above

3. Add Github stage:

        stage('Git Checkout') {
            steps {
                git branch: 'devops', credentialsId: 'Git', url: 'https://github.com/gauravk29/full_vpc'
            }}

To get the above snippet, Use Pipeline syntax>Snippets Generator and add git creds in the steps:

4. Add Terraform init stage

stage('Terraform Init') {
            steps {
                sh 'terraform init'
            }}

5. Add Terraform plan stage

stage('Terraform Plan') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'aws_credentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) 
                {
                sh 'terraform plan'
                }}}

Again navigate to Snippets Generator & select “withCredentials: Bind credentials to variables”, configure your AWS keys & generate snippet

6. Add Terraform apply stage

stage('Terraform Apply') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'aws_credentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) 
                {
                
                sh 'terraform apply --auto-approve'
                }}}

7. Add Terraform destroy stage: I have added this so that by mistake you do not leave your aws resources as is and get billed unnecessarily over the time. To confirm your created resources you can comment/remove this step temporarily. But do not forget to destroy your resources or to add this step later on :)

stage('Terraform Destroy') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'aws_credentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) 
                {
                
                sh 'terraform destroy --auto-approve'
                }}}

Complete pipeline: after adding all the steps

pipeline {
    agent any
    tools {
        terraform 'terraform'
        }
stages {
        stage('Git Checkout') {
            steps {
                git branch: 'devops', credentialsId: 'Git', url: 'https://github.com/gauravk29/full_vpc'
            }
        }
        stage('Terraform Init') {
            steps {
                sh 'terraform init'
            }
        }
        stage('Terraform Plan') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'aws_credentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) 
                {
                sh 'terraform plan'
                }
            }
            }
        stage('Terraform Apply') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'aws_credentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) 
                {
                
                sh 'terraform apply --auto-approve'
                }
            }
        }
        stage('Terraform Destroy') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'aws_credentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) 
                {
                
                sh 'terraform destroy --auto-approve'
                }
            }
        }
    }
}

Execute Build now & if everything is configured properly. You will get a successful build run:

So, this was complete end to end article on terraform on git running via Jenkins.
I hope this will surely help DevOps enthusiasts.
Thank you !