Terraform for_each. cloudinit_config and user_data

Terraform
1

WS User Data with Multiple Files using Template function

I want to create multiple aws ec2 instances resources (NOTE: 4 instances, The example only shows 2) with, a different set of configurations, a core user_data script and a custom user-data scripts for each type of the aws instances to also be run via user_data. I used this provided terraform solution. AWS User Data with Multiple Files using Templatefile
Based on the configuration’s files below, the common.sh is being copied and executed on both instances. However, the configure.sh is being copied to both instances’ but it’s only executed on the GritfyWeb server.
Please advise.

main.tf

data "cloudinit_config" "example" {
  for_each = var.servers

  part {
    filename     = "common.sh"
    content_type = "text/x-shellscript"
    content = templatefile("${path.cwd}/scripts/common.sh", {
    })
  }
  part {
    filename     = "configure.sh"
    content_type = "text/x-shellscript"
    #content = templatefile("${path.cwd}/scripts/each.value.appl_name}", {
    content = templatefile("/home/ec2-user/stage2/scripts/${each.value.appl_name}", {
    })
  }
}

resource "aws_instance" "server" {
  for_each = var.servers
  ami = each.value.ami_id
  instance_type = each.value.instance_type
  subnet_id = each.value.subnet_id
  security_groups = [ "sg-04b403faf33f968ad" ]
  key_name = var.key_name

user_data = data.cloudinit_config.example[each.key].rendered

  tags = {
    Name = "${each.value.environment}-server"
  } 

}

variables.tf

variable "servers" {
  description = "Map of servers names to configuration."
  type        = map(any)

  default = {
    client-webapp = {
      ami_id                  = "ami-06640050dc3f556bb",
      instance_type           = "t2.micro",
      subnet_id               = "subnet-02ef6d33efadfef5a"
      environment             = "GritfyApp"
      appl_name               = "GritfyApp-test.tpl"
    },
    internal-webapp = {
      ami_id                     = "ami-057b76da478f6105c",
      instance_type           = "t3.micro",
      subnet_id               = "subnet-02ef6d33efadfef5a"
      environment             = "GrityWeb-dev"
      appl_name               = "GrityWeb-dev.tpl"
    }
  }
}

variable "aws_region" {
  description = "AWS region for all resources."
  type        = string
  default     = "us-east-2"
}

variable "key_name" {
  description = "Key Name"
  type        = string
  default     = "em-test"
}

variable "security_group_id" {
  description = "ID of existing security group whose rules we will manage"
  type        = string
  default     = "sg-04b403faf33f968ad"
}

user_data_scripts

ls scripts/
scripts/
common.sh  GritfyApp-test.tpl  GrityWeb-dev.tpl
2

Hi @mmauney,

From what you’ve described I think it’s not yet clear whether the problem you’ve observed is related to Terraform behavior or to cloud-init behavior. For that reason I don’t have a direct answer for you yet but I do have some ideas about what you could try to gather more information.

I see in your configuration that your intention is to have one instance of data.cloudinit_config.example for each element of var.servers, and then to choose the content of the configure.sh script based on each object’s appl_name attribute. My first step here then would be to inspect the results of that data source to see if the configuration matches what you intend.

If you run terraform console at your shell prompt you can access Terraform’s interactive evaluation console, which will let you evaluate arbitrary expressions against the objects in your root Terraform module. Since we’re interested in resource-related data it’s important to terraform apply first so that the final results of those resources will be snapshotted in the state.

Once you have a state snapshot including the results of applying this configuration, you could type the following at the console command prompt:

data.cloudinit_config.example

A resource or data block with for_each set appears in expressions as a mapping using the same keys as your source collection, and so I’d expect to see a mapping with the keys client-webapp and internal-webapp, each of which contains an object value representing the result of an instance of data.cloudinit_config.example.

To produce an example without recreating all of your environment I made a simpler version of your configuration that doesn’t actually refer to the templates and instead just uses some fixed content in each of the cloud-init parts. For me, evaluating data.cloudinit_config.example in the console gives this result:

> data.cloudinit_config.example
{
  "client-webapp" = {
    "base64_encode" = true
    "boundary" = "MIMEBOUNDARY"
    "gzip" = true
    "id" = "945689793"
    "part" = tolist([
      {
        "content" = "common content"
        "content_type" = "text/x-shellscript"
        "filename" = "common.sh"
        "merge_type" = ""
      },
      {
        "content" = "GritfyApp-test.tpl content"
        "content_type" = "text/x-shellscript"
        "filename" = "configure.sh"
        "merge_type" = ""
      },
    ])
    "rendered" = "H4sIAAAAAAAA/7TOwUvDMBTH8Xsg/0Po/XV6Ejp2mG6IhyrIFDxm6ev6IHkJySu0/724IRbEk+wY+P7yPg+RBVngMCdsTBi9ULJZVoEm7NbmGEfubJ43VfvU7u9f3p5329ePSn294B1zociNua1vtNIKYBlp9f33jkqKheTcWhHrhoAsa9OTR7YBN5WLIUSuy1D9zA7Zcukxw55d7IhPjbk7kiyCs1lwktUEZUDvi8uURKuWAv7iXW4Yd1n/k8s9ncaM1xU/ZpJ+3qYEgkVqSf4vPYBWnwEAAP//in8wWcsBAAA="
  }
  "internal-webapp" = {
    "base64_encode" = true
    "boundary" = "MIMEBOUNDARY"
    "gzip" = true
    "id" = "1350101568"
    "part" = tolist([
      {
        "content" = "common content"
        "content_type" = "text/x-shellscript"
        "filename" = "common.sh"
        "merge_type" = ""
      },
      {
        "content" = "GrityWeb-dev.tpl content"
        "content_type" = "text/x-shellscript"
        "filename" = "configure.sh"
        "merge_type" = ""
      },
    ])
    "rendered" = "H4sIAAAAAAAA/7TOwUrEMBDG8Xsg7xB6n66ehC57UHcRD1WQVfGYptPtQDIJyVTatxd3ERfEk3gM/L/M7zayIAvsl4SNCZMXSjbLKtCM/dp0ceLe5mVTtfft7ubx+WF7/fRWqc8XvGAuFLkxl/WFVloBnEdaff29pZJiITm2VsS6MSDL2gzkkW3ATeViCJHrMlbfs322XAbMsGMXe+JDY646krPgaBacZTVDGdH74jIl0aqlgD94pxvGndZ/5PJAhynj/4rvMsnyih30+F5L8r/ZAbT6CAAA//8qslj3yQEAAA=="
  }
}

Notice that for each of the two instances of the data resource there’s an object with a part attribute which contains a list of objects representing the part blocks. Each one shows the final value of content, which in my case is just a placeholder string but in your case will be the result of rendering the templates.

The main question to answer to start here is: does the content for each of your parts of each of your instances contain the content you expected? Are all of the scripts valid shell syntax and a valid sequence of commands?

A common reason for the symptoms you’ve described is that one of the scripts contains an error, so when cloud-init tries to run it the execution fails. When that happens, cloud-init writes information to its own logs inside your EC2 instance, but Terraform cannot see or react to that failure because cloud-init runs outside of Terraform’s control. The goal of this debugging, then, is to verify that all of your scripts are valid and contain the content you expected.

3

Hello,

Thanks for your response. I ran the data.cloudinit_config.example via the console and did see any command errors. However, I did see errors in the /var/log/cloud-init-output.log from the GritfyApp-server.
I’ve attached the cloud-init logs from both servers and the data.cloudinit_config.example output.

Thanks

.

(Attachment cloud-init-output-GrityWeb-dev-server.log is missing)

(Attachment data.cloudinit_config.example.log is missing)

(Attachment cloud-init-output-GritfyApp-server.log is missing)

(Attachment cloud-init-GrityWeb-dev-server.log is missing)

(Attachment cloud-init-GritfyApp-server.log is missing)

4

cloud-init-GritfyApp-server.txt (139.1 KB)
cloud-init-GrityWeb-dev-server.txt (203.9 KB)
cloud-init-output-GritfyApp-server.txt (4.3 KB)
cloud-init-output-GrityWeb-dev-server.txt (7.2 KB)
data.cloudinit_config.example.txt (2.8 KB)

5

I found the problem. ```
#content = templatefile(“${path.cwd}/scripts/each.value.appl_name}”,
Missing bracket {

Thanks again