For      Developers

By: Ashley Hutson
Twitter: @asheliahut

Slides: https://github.com/asheliahut/terraform-for-developers-talk

How My Infrastructure Became Unmanageable

  • My work at McGraw Hill Education
    • Building large scale applications
    • Microservices
  • Have you ever had an application get to complex and needing many cloud services tying them all together?
  • SECRET management and how to get those in a secure location easily
  • Wanting to share my infrastructure designs with teammates and others.

Why Teach This to an Application Developer?

  • There is a high likely chance if you work at a large company there is an SRE team/Dev Ops person but they may build things that you can help with or improve
  • Mostly terraform code needs small variable changes per application and understanding it is important.
  • You should want to know what pieces you are putting together to build your application without pulling in extra hardware saving money

What is Terraform

  • A tool to manage infrastructure as code
  • Allows for creating sharable components to build out and tie each piece of cloud components together.
  • It is platform agnostic with support for (AWS, GCP, Azure, OpenStack and many more)
  • Easy way to make a pizza get delivered on every new production build :P 

What Version?

  • Terraform has an odd versioning scheme
    • Current version is v0.15
  • v0.12+ Is not backwards compatible with v0.11

How Do I Install?

  • The best way to install is using TFSwitch
    • https://tfswitch.warrensbox.com/Install
  • Once installed with TFSwitch
    • Just run
      • tfswitch
    • Or
      • tfswitch <version>
echo $PATH
/this/is/going/to/be/long/help/me:/next/path/is/long:/why/does/path/never/end

How Do I Manual Install?

  • Start by downloading terraform for your Operating System
  • Make sure terraform is available in your PATH
    • Linux: /usr/local/bin or /usr/bin
    • Mac: Use homebrew or add to PATH manually
    • Windows: Manually add to your PATH
echo $PATH
/this/is/going/to/be/long/help/me:/next/path/is/long:/why/does/path/never/end

What is HCL?

HCL = Hasicorp Configuration Language

 

This is a layer on top of JSON

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
  required_version = "~> 0.13"
  backend "s3" {
    bucket               = "terraform-state-aak"
    region               = "us-east-1"
    encrypt              = "true"
    dynamodb_table       = "my-locking-table"
    role_arn             = "arn:aws:iam::111111111111:role/deploy-infrastructure"
    key                  = "app.tfstate"
    workspace_key_prefix = "env"
  }
}

module "cloudwatch" {
  source      = "./modules/cloudwatch"
  name        = var.cloud_log_name
}

Base HCL Rules

  • All code is written in stanzas or blocks
  • Blocks consist of Key = Value pairs.
  • Comments begin with # for single line and multi line are /* */
  • Lists are [] and maps are {}
    • Maps are like json key value
    • Lists are like non associative arrays for php
  • All strings must be "double quotes"

Source: https://www.linode.com/docs/applications/configuration-management/introduction-to-hcl/

Terraform File Structure

Main.tf

This is the starting point for all code to run 

The terraform code will always start with a provider or variables defined they can share a same space but it is not prefered. Always separate variables from resources and modules.

Variables.tf

This is where all base high level variables are defined

All variables used for your application as a whole are stored in this file. You may think though that this is a security issue because of secrets but there are ways around that. This file does not store computed values either.

Outputs.tf

Allows resources and variables to be accessed outside scope.

The outputs file houses a structure similar to that of variables but each have a type of output and the value references the resource or module from which the values come. This allows for pulling nested resources.

output "alb_name" {
  value = "${module.service_alb.alb_name}"
}

Modules Directory

This is storage of each piece of infrastructure broken down.

If creating a new AWS application with an RDS node and elasticache tied to your EC2 instance you would want 1 module for RDS, 1 module for elasticache, and 1 module for EC2.

.terraform directory

This holds local run data of terraform.

This directory is vital and important for running terraform locally on a computer. If you choose to go further and integrate with a CI provider or hook up to a backend this becomes less of an important piece.

Components of Terraform

  • Variables
  • Providers
  • Modules
  • Resources
  • Templates
  • Data Sources

Variables

  • They are just like any other programming language like PHP
  • You use them to set a value that can be overridden or used to reference and keep your code DRY
  • Descriptions are not mandatory but highly encouraged
  • Only a value is needed to be set
  • These are objects with a key value pair of value = "value"

Variables Example

# Single Variable
variable "aws_region" {
  description = "AWS Region"
  default     = "us-east-1"
}

# List type
variable "application_ports" {
  type = list(object({
    internal = number,
    external = number,
    protocol = string
  }))
  default = [
    {
      internal = 8080
      external = 80
      protocol = "tcp"
    },
    {
      internal = 8081
      external = 443
      protocol = "tcp"
    }
  ]
}

How to Reference

To Access the scope of a variable you just need to pull it from where it lives.

module "cloudwatch" {
  source      = "./modules/cloudwatch"
  name        = var.cloud_log_name
}

Ternary

Change interpolation of variables based on conditional data

module "elasticache" {
  source                  = "./modules/elasticache"
  project_name            = var.project_name
  environment_name        = var.env
  engine_version          = var.engine_version
  instance_type           = "${var.env == "production" ? var.prod_size : var.dev_size}"
}

In v0.11 this was used with many hacks many of these were fixed in v0.12

Conditionals

This was added in 0.13 that allowed to validate the variables used.

variable "cluster_min_size" {
  description = "Mimimum number of running EC2 instances in the cluster."
  type        = number
  default     = 0

  validation {
    condition     = var.cluster_min_size >= 0
    error_message = "Value MUST be a positive integer."
  }
}

Sensitive Data

This was added in 0.14 that allowed to obfuscate data in outputs.

variable "user_information" {
  type = object({
    name    = string
    address = string
  })
  sensitive = true
}
resource "some_resource""a" {
  name    = var.user_information.name
  address = var.user_information.address
}

Functions

There are many built in terraform functions for computing and doing actions to variables.

  • Numeric
    • abs
    • ceil
  • String
    • chomp
    • join
    • regex
  • Hash
    • sha1
  • So many more

Providers

  • These are used to reference the cloud system or local system in which you are trying to run against.
    • AWS/GCP
  • There are no providers that can just do all clouds so you must do each separately if trying to deploy to multi cloud.
  • These can be used for more than just base infrastructure. You can create providers for any type of API (example: dominos or datadog)

Infastructure

These are the most common types of providers used when writing Terraform code.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
    google = {
      source = "hashicorp/google"
      version = "3.65.0"
    }
  }
  required_version = "~> 0.15"
}

provider "aws" {
  access_key = var.aws_access_key
  secret_key = var.aws_secret_key
  region     = var.aws_region
}

provider "google" {
  credentials = file("account.json")
  project     = var.gcp_project
  region      = var.aws_region
}

PLEASE REMEMBER TO SET VERSIONS!

Misc

Providers can be found for infrastructure monitoring tools that are backed by APIs

 

This can be powerful to have a single place for all component needs for your application down to your source control

provider "datadog" {
  api_key = var.datadog_api_key
  app_key = var.datadog_app_key
}

# Configure the PagerDuty provider
provider "pagerduty" {
  token = var.pagerduty_token
}

# Configure the GitHub Provider
provider "github" {
  token        = var.github_token
  organization = var.github_organization
}

# Configure the Cloudflare provider
provider "cloudflare" {
  email = var.cloudflare_email
  token = var.cloudflare_token
}

3rd Party

There are people who will wrap other apis into providers so you can call. Remember if it has a JSON payload it is easily replicable in HCL.

provider "dominos" {
  first_name    = var.dom_first_name
  last_name     = var.dom_last_name
  email_address = var.dom_email
  phone_number  = var.dom_phone

  credit_card {
    number = var.dom_card.num
    cvv    = var.dom_card.cvv
    date   = var.dom_card.date
    zip    = var.dom_card.zip
  }
}

# External DB provider for big data
provider "snowflake" {
  account = "..."
  role    = "..."
  region  = "..."
}

Modules

  • After having some providers set up the next step is to figure out each component of the infrastructure and how to break it apart
  • Each module is set up with the same file structure as your top level.
    • Main.tf
    • variables.tf
    • outputs.tf
  • Outputs become more valuable here as you will be refrencing these across modules in your application top level

Main.tf

These are lists of resources and templates

resource "aws_db_parameter_group" "rds_parameter_group" {
    name   = ${var.name"-"var.environment"-rds-pg-11"}
    family = "postgres11"
}

data "template_file" "event_rule" {
    template = file("${path.module}/event-rule.json")
    vars {
        cluster_arn = aws_ecs_cluster.default.id
    }
}

Examples Directory

Include this as a bonus

Fill out a full definition of the module and how to use it in a file that would call the module.

External Modules

How to call official modules

# Pull from terraform source and don't write own modules
module "ec2_cluster" {
  source                 = "terraform-aws-modules/ec2-instance/aws"
  version                = "~> 2.0"

  name                   = "my-cluster"
  instance_count         = 5

  ami                    = "ami-ebd02392"
  instance_type          = "t2.micro"
  key_name               = "user1"
  monitoring             = true
  vpc_security_group_ids = ["sg-12345678"]
  subnet_id              = "subnet-eddcdzz4"

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}

External Modules

How to call custom modules

module "cool_thing" {
  source = "git::git@github.com:enthought/terraform-modules.git/cool_thing?ref=v0.1.0"
  var_one = "foo"
  var_two = "bar"
}

Resources

  • This is the definition of each piece needed for a piece of infrastructure. It amounts to an api call definiton of JSON.
  • A resource can be an EC2 instance or it can be a S3 bucket all the way to a permissions object for them

Resource Example

resource "aws_instance" "web" {
  ami           = "ami-a1b2c3d4"
  instance_type = "t2.micro"
}

resource "aws_iam_role" "lambda_iam_role" {
    name               = ${var.name"-"var.environment"-ecs-scale-role"}
    assume_role_policy = data.aws_iam_policy_document.lambda_role_document.json
}

# Create a new monitor
resource "datadog_monitor" "default" {
  # ...
}

# Create a new timeboard
resource "datadog_timeboard" "default" {
  # ...
}

Resources Follow Docs

Lifecycle Events

  • prevent_destroy : bool
  • create_before_destroy : bool
  • ignore_changes: list of keys
  • timeouts: object (this is not always available for all resources)

prevent_destroy

Stops resouce from being destroyed on destroy

This value can save you many times over by applying it to a database. Making sure your data is safe and secure.

create_before_destroy

This is important from stopping downtime

This is used heavily for replacing new EC2 Instances or Google apps this allows for new instances to be created or auto scaling groups and then the flip to the resource will occur before destroying the old one. One less problem during deploys!

ignore_changes

If you want to manage something outside terraform

This is used when using a service that provides automatic updates for instance if you use automated updates for RDS you can ignore the engine_version so when you push the next time it will go through just fine without complaining the version isn't in sync.

timeouts

This can be super important for slow APIs (aws cough cough)

There are many times throughout the day where cloud provider apis get overwhelmed and placing something like a new EC2 instance can take a long time this allows for the operation to wait it out longer before deeming the apply bad.

timeouts {
    create = "60m"
    delete = "2h"
  }
}

Templates

  • This is a way to store the definition of a file that can be interpolated differently based on variables passed to create a resource.
  • The best example of this is for either Kubernetes or ECS when defining a task definition.
  • These definitions needs to pull and store variables that can be different based on environments.

Template Example

[
    {
        "cpu": 10,
        "essential": true,
        "image": "datadog/agent",
        "memoryReservation": 128,
        "name": "dd-agent",
        "portMappings": [
            {
                "containerPort": 8125,
                "protocol": "udp"
            }
        ],
        "privileged": true,
        "environment": [
            {
                "name": "DD_API_KEY",
                "value": "${datadog_license}"
            },
            {
                "name": "DD_DOCKER_LABELS_AS_TAGS",
                "value": "true"
            },
            {
                "name": "DD_PROCESS_AGENT_ENABLED",
                "value": "true"
            }
        ]
    }
]

Data Sources

  • These are pulled from the provider and will being with "data" these allow for pulling data from a provider to use as a variable
  • The best examples of this is pulling parameter store variables to use as secret values or pulling new data from a shared source
    • I still recommend locking all versions but you could even pull latest amis from aws.

Data Example

# Find the latest available AMI that is tagged with Component = web
data "aws_ami" "web" {
  filter {
    name   = "state"
    values = ["available"]
  }

  filter {
    name   = "tag:Component"
    values = ["web", "myapp"]
  }

  most_recent = true
}

Terraform State

  • This is an important file that holds all of the values computed from after running terraform. 
  • Keeping this file locally is frowned upon because it is always good to know what your infrastructure was before and other people can't access it.
  • Define a backend for your state!

Backend

A "backend" in Terraform determines how state is loaded and how an operation such as apply is executed. This abstraction enables non-local file state storage, remote execution, etc.

 

These are placed at the top of your main.tf or in a seperate file called backend.tf

Make file secure!

terraform {
  backend "consul" {
    address = "demo.consul.io"
    scheme  = "https"
    path    = "example_app/terraform_state"
  }
}

terraform {
  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key"
    region = "us-east-1"
  }
}

The First Deploy

  • terraform init --backend=true
    • supplying backend = true will force terraform to use the backend you define or you can choose to run it locally and move the file to the backend location later
  • terraform plan
  • terraform apply

Basics Over! Let's Go Crazy?

  • We need to go over a few more topics really before diving in to our first project.
  • Terraform is for providing updates to infrastructure first and foremost. If you choose to use terraform to push code it is best to split it into a seperate terraform codebase.
  • Creating environments and destroying them are simple that's why we make sure to "plan" for them.

Terraform Commands!

Apply

  • Runs your terraform and creates the infrastructure and saves the state
  • Does not rollback!

Destroy

  • Destroys all infrastructure unless you specify specific resources
  • Does not rollback!

Plan

  • This is the most important command you will ever run
  • It goes through and pre determines many values and show exactly what it will do in an apply
  • This should ALWAYS be ran before apply.

Import

  • This feature is amazing and allows you to pull in pre defined resources in your terraform into your state.
  • If you have infrastructure already in your cloud you can define it as it is and run import for that resource and it will pull the values into your state.

If done correctly

  • Each piece should be able to come up and down at will and not worry about issues
  • When new applies are done to modify old values like changing an instance size there is always a lifetime cycle event
lifecycle {
    create_before_destroy = true
}

How to Deploy Code With Terraform

  • Terraform was never meant to directly deploy application code.
  • In this day and age we have docker!
  • Docker places all your code into a single piece of infrastructure that can be pulled and executed.
  • Updates to application code are as simple as pushing a new docker tag.

*If using a proper deploy tooling for applications you may not want to deploy code with terraform

Deploying Docker ECR

#!/bin/bash
set -o pipefail
set -e

# Move To terraform Directory
cd $( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

# Set Environment Variables that need to be calculated
source ./set-env.sh

# Log in to Elastic Container Service
eval `aws ecr get-login --region ${CCD_AWS_REGION} --no-include-email`

PHP_REPOSITORY=${CCD_AWS_ACCOUNT_ID}.dkr.ecr.${CCD_AWS_REGION}.amazonaws.com/app-name/php

# Build Container Images
docker build -t $PHP_REPOSITORY -f ../build/php/Dockerfile ../

# Tag Container Images && Push New Containers
docker tag $PHP_REPOSITORY $NODE_REPOSITORY:${APP_VERSION}
docker push $PHP_REPOSITORY:${APP_VERSION}

Now Let's Build An Application For Serverless Google Cloud Run

Deploying Docker GCP

# install gcloud
gcloud auth configure-docker

# build your container
cd /path/to/dockerfile
docker build .

# [SOURCE_IMAGE] is the local image name or image ID 
# [PROJECT-ID] is your GCP account project id
docker tag [SOURCE_IMAGE] gcr.io/[PROJECT-ID]/php-app
docker push gcr.io/[PROJECT-ID]/php-app

Create main.tf

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "3.65.0"
    }
  }
  backend "gcs" {
    bucket  = var.backend_bucket
    prefix  = var.backend_prefix
  }
}

provider "google" {
  credentials = sensitive(file("account.json"))
  project     = sensitive(var.project_id)
  region      = var.region
}

module "cloudrun" {
  source             = "./modules/cloudrun"
  service_name       = var.service_name
  project_id         = var.project_id
  location           = var.region
  url                = var.url
  container_location = var.container_location
}

Create variables.tf

variable "backend_bucket" {
  type        = string
  default     = "app_backend"
  description = "The backend bucket location"
}

variable "backend_prefix" {
  type        = string
  default     = "terraform/state/prod"
  description = "The prefix inside the bucket to go down into allowing you to swap environments if you wanted."
}

variable "project_id" {
  type        = string
  default     = "project_id"
  description = "The project id associated to the gcp account."
}

variable "region" {
  type        = string
  default     = "us-central1"
  description = "The gcp region to set the application."
}

variable "service_name" {
  type        = string
  default     = "cool-php-app-service"
  description = "The gcp name for the cloud run service."
}

variable "url" {
  type        = string
  default     = "https://URL_ASSOCIATED_TO_ACCOUNT.com"
  description = "The url the application should run at."
}

variable "container_location" {
  type        = string
  default     = "gcr url"
  description = "The gcr location of our applications docker container."
}

Create Cloud Run Module

resource "google_cloud_run_service" "default" {
  name     = var.service_name
  location = var.location

  metadata {
    namespace = var.project_id
  }

  spec {
    containers {
      image = var.container_location
    }
  }
}

# The Service is ready to be used when the "Ready" condition is True
# Due to Terraform and API limitations this is best accessed through a local variable
locals {
  cloud_run_status = {
    for cond in google_cloud_run_service.default.status[0].conditions : cond.type => cond.status
  }
}

resource "google_cloud_run_domain_mapping" "default" {
location = var.location
name = var.url

  metadata {
    namespace = var.project_id
  }

  spec {
    route_name = google_cloud_run_service.default.name
  }
}

Create Cloud Run Variables

variable "project_id" {
  type        = string
  description = "The project id associated to the gcp account."
}

variable "region" {
  type        = string
  default     = "us-central1"
  description = "The gcp region to set the application."
}

variable "service_name" {
  type        = string
  default     = "cool-php-app-service"
  description = "The gcp name for the cloud run service."
}

variable "url" {
  type        = string
  default     = "https://URL_ASSOCIATED_TO_ACCOUNT.com"
  description = "The url the application should run at."
}

variable "container_location" {
  type        = string
  default     = "gcr url"
  description = "The gcr location of our applications docker container."
}

Create Cloud Run Outputs

output "isReady" {
  value = local.cloud_run_status["Ready"] == "True"
}

Formatting built in with terraform fmt

Questions!?

I'll totally answer anything I can