Https://github.com/hashicorp/terraform/issues/32754

Terraform Providers AWS
1

Terraform Version

Terraform v1.3.7
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.54.0
+ provider registry.terraform.io/hashicorp/local v2.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/tls v4.0.4

Terraform Configuration Files

variable "key_name" {
  description = "SSH Key Name For Authentication"
  type        = string
  default     = "ubuntu"
}

resource "tls_private_key" "ubuntu" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "aws_key_pair" "generated_key" {
  key_name   = var.key_name
  public_key = tls_private_key.ubuntu.public_key_openssh

}

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}


resource "aws_instance" "ubuntu" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = var.instance_type
  key_name      = aws_key_pair.generated_key.key_name

  network_interface {
    network_interface_id = aws_network_interface.ubuntu.id
    device_index         = 0
  }

  metadata_options {
    http_endpoint = "disabled"
  }

  connection {
    user        = "ubuntu"
    type        = "ssh"
    host        = self.public_ip
    private_key = tls_private_key.ubuntu.private_key_pem
    timeout     = "1m"
  }

  provisioner "remote-exec" {
    inline = [
      "apt get update -y"
    ]
  }

  depends_on = [
    aws_key_pair.generated_key
  ]
}

Debug Output

Expected Behavior

Terraform Apply should work through fine and remote_exec should connect and execute

Actual Behavior

Throws an error as shown which is an SSH error when remote_exec tries to connect.

╷
│ Error: file provisioner error
│
│   with aws_instance.web-template,
│   on ec2.tf line 51, in resource "aws_instance" "web-template":
│   51:   provisioner "file" {
│
│ timeout - last error: SSH authentication failed (ubuntu@35.175.205.156:22): ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported
│ methods remain
╵

However, If I disable the following lines , it all works smoothly, terraform apply works and remote_exec connects and executes the script.

  metadata_options {
    http_endpoint = "disabled"
  }

Additonally, the SSH key generated is unable to connect and throws the same error.

Steps to Reproduce

terraform init
terraform apply

Additional Context

I need to build a bastion host with IMDS disabled by default as a security requirement and hence I need to use the following metadata configuration in the aws_instance resource

metadata_options {
   http_endpoint = "disabled"
 }

What I fail to understand is why or rather how is this step/feature interfering with SSH communications ? Why does remote_exec need to contact IMDS service when all it really needs is an SSH private key which is being provided.

References

Other similar issues I looked at prior to filing this error