PG backend - no stored state found after init

[Sehaco]

Hey there, I have problems understanding how to get a remote backend to work. The plan is to not store tfstates in gitlab and instead in a pgdb (S3, AWS, Cloud, …, not possible)

For a first test I created a pg-container on my host using the steps below. It is up and running and after the init I see the schema has been created properly. But the plan fails because there is no remote state found. And sure there is no remote state yet… What did I misunderstood in the how to?

Any help is greatly appreciated.

The password is random. Please ignore the hardcoded pass in the data source, I will provide variables later in an gitignored autofile.

Example provider.tf

terraform {
  required_providers {
    vsphere = {
      source = "hashicorp/vsphere"
      version = "2.0.2"
    }
  }
  backend "pg" {}
}

provider "vsphere" {
  user              = var.user
  password          = var.password
  vsphere_server    = var.vsphere_server

  allow_unverified_ssl = false
}

data "terraform_remote_state" "pgdb" {
  backend = "pg"
  config = {
    conn_str = "postgres://postgres:Asdf1234@localhost/terraform_backend?sslmode=disable"
  }
}

Create a new and clean db

docker run --name pg_tfstate -e POSTGRES_PASSWORD=Asdf1234 -e POSTGRES_USER=postgres -e POSTGRES_DB=terraform_backend --publish 127.0.0.1:5432:5432 -d postgres:15.0-alpine

Check workspace

terraform workspace list
* default

Initialise the backend

terraform init -backend-config="conn_str=postgres://postgres:Asdf1234@localhost/terraform_backend?sslmode=disable"

Initializing the backend...

Successfully configured the backend "pg"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/vsphere versions matching "2.0.2"...
- Installing hashicorp/vsphere v2.0.2...
- Installed hashicorp/vsphere v2.0.2 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

[...]

Plan it

terraform plan                                                                                                    
data.terraform_remote_state.pgdb: Reading...
data.vsphere_datacenter.datacenter: Reading...
data.vsphere_custom_attribute.owner_name: Reading...
data.vsphere_custom_attribute.admin_contact: Reading...
data.vsphere_custom_attribute.admin_contact: Read complete after 0s [id=310]
data.vsphere_custom_attribute.owner_name: Read complete after 0s [id=304]
data.vsphere_datacenter.datacenter: Read complete after 0s [id=datacenter-49618]
data.vsphere_host.esxi_hosts[0]: Reading...
data.vsphere_compute_cluster.compute_cluster[0]: Reading...
data.vsphere_network.networks[0]: Reading...
data.vsphere_datastore.datastores[0]: Reading...
data.vsphere_virtual_machine.template[0]: Reading...
data.vsphere_network.networks[0]: Read complete after 0s [id=network-49634]
data.vsphere_datastore.datastores[0]: Read complete after 0s [id=datastore-349728]
data.vsphere_compute_cluster.compute_cluster[0]: Read complete after 0s [id=domain-c345684]
data.vsphere_host.esxi_hosts[0]: Read complete after 0s [id=host-345706]
data.vsphere_virtual_machine.template[0]: Read complete after 0s [id=42152b35-6ab4-8e14-4251-3afa13092c42]
╷
│ Error: Unable to find remote state
│ 
│   with data.terraform_remote_state.pgdb,
│   on provider.tf line 19, in data "terraform_remote_state" "pgdb":
│   19: data "terraform_remote_state" "pgdb" {
│ 
│ No stored state was found for the given workspace in the given backend.
╵

And why is the connection string in there? I thought the purpose of using -backend-config is to prevent that from happening?
.terraform/terraform.tfstate

{
    "version": 3,
    "serial": 1,
    "lineage": "1fef30f4-f401-d745-2722-e41ff99600ee",
    "backend": {
        "type": "pg",
        "config": {
            "conn_str": "postgres://postgres:Asdf1234@localhost/terraform_backend?sslmode=disable",
            "schema_name": null,
            "skip_index_creation": null,
            "skip_schema_creation": null,
            "skip_table_creation": null
        },
        "hash": 1949691007
    },
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

[jbardin]

Hi @Sehaco,

Are you attempting to store the state for the given config in a pg backend, or are you attempting to access an existing state from another workspace’s pg backend?

The backend block configures where you store your state, and the terraform_remote_state data resource is for accessing another remote state. These two are unrelated, so you probably only need to have one depending on your goal, and they should definitely not be configured to point to the same location.

[Sehaco]

hey @jbardin

this is a fresh and clean db with an uninitialized test instance. I was trying to figure out how to store the tfstate in a remote backend instead of doing it locally with the purpose of team shared instances.

I thought both blocks are needed for that. Or do I always have to manually add and delete the backend {} and data blocks?

If I get this right storing is done via backend {} an accessing via data "terraform_remote_state"?

So if I want to check whether there is an existing config I have to use the data block and if I want to make a change I have to remove it and add the backend block?

[jbardin]

The backend config is statically defined, and doesn’t normally change, so you would usually just leave the backend "pg" definition in the config.

The terraform_remote_state data resource is specifically used to access root module outputs from another state file. If you are not accessing outputs from a different state file, then you should not use this.

I’m not certain what you mean by “check whether there is an existing config”, you are defining the configuration to use here, there should only be one.

[Sehaco]

@jbardin

The backend config is statically defined, and doesn’t normally change, so you would usually just leave the backend "pg" definition in the config.

That’s how I understood. I thought this block is needed for the first “init” of an instance only, hence the use of -backend-config. So I misunderstood and now wish I could use variables just like for the provider

@jbardin

The terraform_remote_state data resource is specifically used to access root module outputs from another state file.

Alright, got it. This would be useful if my resource would depend on another one and it’s outputs, right? But this isn’t the case in my test. Thank you for clarifying!

@jbardin

I’m not certain what you mean by “check whether there is an existing config”, you are defining the configuration to use here, there should only be one.

I was mixing up words. I was referring to the states of an instance. But after your explanation how it really works, therefore the removal of the data block, I was able to send the state info to the db. And after deleting the lock-file /.terraform-folder, I was able to make changes to an existing instance after a new init as expected.

Thank you very much for your help! I very much appreciate it!