Remote-exec with bastion host is not working by using null_resource for OCI

Terraform
1

Hi,
I am trying to use terraform remote-exec within a null_resource and it works fine when we have direct connectivity to instance. Now i wanted to connect the bastion host first before running the scripts in private instances which is failing.

The err seems to me like terraform is not trying to connect the bastion host instead its trying directly to reach the private instance IP’s which is wrong.

PFB the err and code sample,

│ Error: file provisioner error
│
│ with module.instances[“test-remote-exec-02”].null_resource.shell-remote-exec[0],
│ on modules\main.tf line 183, in resource “null_resource” “shell-remote-exec”:
│ 183: provisioner “file” {
│
│ timeout - last error: dial tcp :22: i/o timeout

Code sample,

provisioner “remote-exec” {
connection {
type = “ssh”
agent = false
timeout = “10m”
host = oci_core_instance.instance.private_ip
user = “opc”
private_key = file(var.private_key)
bastion_host = var.bastion_ip
bastion_user = “opc”
bastion_certificate = “”
bastion_private_key = file(“${path.root}/scripts/ssh-key”)
}

Please suggest.

2

Hi @ulags.n,

This error message is just exposing directly an error from the underlying network library that Terraform uses, but I believe the :22 part of this message is the address that the provisioner was attempting to connect to, which doesn’t seem to include an IP address.

I’m not sure what would cause that to happen, but I wonder if either oci_core_instance.instance.private_ip or var.bastion_ip is an empty string in your configuration but the provisioner is not validating that properly and is instead trying to connect to a wrong location, such as localhost.

Can you check what both of those expressions resolve to (for example, using terraform console) and see if they both seem like sensible IP addresses?

3

Hi @apparentlymart,
Thank you for the response. The oci_core_instance.instance.private_ip returns the OCI instance private IP and that is the one the provisioner is trying to connect directly instead of bastion_ip first(i am passing the right public ip through variable for bastion_ip variable).

The IP addresses are sensible only and the problem is terraform is not trying to connect the bastion IP first before trying to reach the private IP’s.

NOTE: I was able to manually connect to bastion server and then SSH to private IP’s without any issues.

4

Hi @ulags.n,

How have you determined that Terraform is connecting to the instance IP address instead of the bastion? The error message you shared doesn’t mention any IP addresses at all, so I’m not sure how you’ve determined which IP address Terraform tried to connect to in this case.

5

Hi @apparentlymart ,
I could see the instance private ip being printed during terraform apply and I have removed it while pasting it here. Hope that helps.

6

That’s a really confusing thing to do. At least make it obvious where a piece of information has been redacted.

Also, you have said your issue is with the remote-exec provisioner, and shown code using that, but the error message reveals it is the file provisioner that is generating an error:

You are making it harder for people to help you, by omitting relevant information!

7

Hi @maxb,
Thank you for the response. Let’s not get diverted from actual issue hope I have explained it now. I am reiterating it again the terraform is not trying to reach the bastion public ip first instead it’s trying to connect directly the private IP’s.

The logs are samples I posted to support the issue.remote-exec or provisioner block I have used the same connection block.

Please let me know if any more gaps/clarifications needs further inputs.

8

You’ve ignored the point of my previous message. The error you posted is not from the remote-exec provisioner.

9

I did mention both file and remote-exec provisioner is making use of the same connection block. You can use connection block at resource level and also at provisioner level.

Below is the file provisioner and connection block for the same,

provisioner “file” {
source = “${path.root}/scripts/default.sh”
destination = “/home/user/script.sh”

connection {
  type        = "ssh"
  agent       = false
  timeout     = "10m"
  host        = oci_core_instance.instance.private_ip
  user        = "user"
  private_key = file("${path.root}/scripts/ssh-key")

  bastion_host        = var.bastion_ip
  bastion_user        = "user"
  bastion_private_key = file("${path.root}/scripts/ssh-key")
} }

Error while its trying to connect the private ip instead of bastion host first,

│ Error: file provisioner error
│
│ with module.instances[“test-remote-exec-01”].null_resource.shell-remote-exec[0],
│ on modules\compute\instance\main.tf line 131, in resource “null_resource” “shell-remote-exec”:
│ 131: provisioner “file” {
│
│ timeout - last error: dial tcp 10.110.1.235:22: i/o timeout

10

Thank you for sharing the actual configuration that applies to your error.

Though it’s mostly clear in this particular case, please bear in mind Welcome to the forum - please reformat your message in future.

Though this may seem obvious, the behaviour you describe would be easily explained if var.bastion_ip was accidentally set to an empty string.

Therefore, please double check it really is set to a real value.

11

Hi @maxb ,
I have explained it above. Bastion IP has set to a public ip and not an empty string. Question is why terraform is trying to reach the private IP directly instead of bastion ip ?

12

I know you believe var.bastion_ip is set to a public IP, but the easiest explanation for the observed behaviour would be if that is actually not true, and it has accidentally taken on an empty value. Hence I am asking you to investigate that possibility.

13

Assuming that doesn’t resolve the problem, please enable maximum logging, using the TF_LOG=trace environment variable, and show us the portion of the logs that relate to running the provisioner.