Warning: Failed to decode resource from state

Terraform
1

Hello evryone, really needing your brains. I believe AWS upgraded its version recently. For the last 2-3 days I’ve been having all kinda errors when I do ‘tf plan’ or ‘tf apply’. I did upgrade my TF Cloud version and AWS version as well. This is the error I’m getting mostly.

╷
│ Warning: Deprecated attribute
│
│ on cloudfront.tf line 3, in resource “aws_cloudfront_distribution” “www_neuropharmmagen”:
│ 3: domain_name = aws_s3_bucket.www_neuropharmagen_com.website_endpoint
│
│ The attribute “website_endpoint” is deprecated. Refer to the provider
│ documentation for details.
╵
╷
│ Warning: Deprecated attribute
│
│ on route53.tf line 59, in resource “aws_route53_record” “www_neurofarmagen_es”:
│ 59: name = aws_s3_bucket.www_neurofarmagen_es.website_endpoint
│
│ The attribute “website_endpoint” is deprecated. Refer to the provider
│ documentation for details.
╵
╷
│ Warning: Argument is deprecated
│
│ with module.production.aws_eip.nat_eip,
│ on .terraform/modules/production/pgx/vpc.tf line 45, in resource “aws_eip” “nat_eip”:
│ 45: vpc = true
│
│ use domain attribute instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.www_neuropharmagen_com,
│ on s3.tf line 20, in resource “aws_s3_bucket” “www_neuropharmagen_com”:
│ 20: resource “aws_s3_bucket” “www_neuropharmagen_com” {
│
│ Use the aws_s3_bucket_website_configuration resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.www_neuropharmagen_com,
│ on s3.tf line 22, in resource “aws_s3_bucket” “www_neuropharmagen_com”:
│ 22: acl = “private”
│
│ Use the aws_s3_bucket_acl resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with module.nfg-prod.aws_eip.nat_eip,
│ on .terraform/modules/nfg-prod/nfg/vpc.tf line 44, in resource “aws_eip” “nat_eip”:
│ 44: vpc = true
│
│ use domain attribute instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.neurofarmagen_es,
│ on s3.tf line 31, in resource “aws_s3_bucket” “neurofarmagen_es”:
│ 31: acl = “private”
│
│ Use the aws_s3_bucket_acl resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.neurofarmagen_es,
│ on s3.tf line 29, in resource “aws_s3_bucket” “neurofarmagen_es”:
│ 29: resource “aws_s3_bucket” “neurofarmagen_es” {
│
│ Use the aws_s3_bucket_website_configuration resource instead
╵
╷
│ Warning: Deprecated attribute
│
│ on route53.tf line 48, in resource “aws_route53_record” “neurofarmagen_es”:
│ 48: name = aws_s3_bucket.neurofarmagen_es.website_endpoint
│
│ The attribute “website_endpoint” is deprecated. Refer to the provider
│ documentation for details.
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.neuropharmagen_com,
│ on s3.tf line 11, in resource “aws_s3_bucket” “neuropharmagen_com”:
│ 11: resource “aws_s3_bucket” “neuropharmagen_com” {
│
│ Use the aws_s3_bucket_website_configuration resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.neuropharmagen_com,
│ on s3.tf line 13, in resource “aws_s3_bucket” “neuropharmagen_com”:
│ 13: acl = “private”
│
│ Use the aws_s3_bucket_acl resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.www_neurofarmagen_es,
│ on s3.tf line 38, in resource “aws_s3_bucket” “www_neurofarmagen_es”:
│ 38: resource “aws_s3_bucket” “www_neurofarmagen_es” {
│
│ Use the aws_s3_bucket_website_configuration resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.www_neurofarmagen_es,
│ on s3.tf line 40, in resource “aws_s3_bucket” “www_neurofarmagen_es”:
│ 40: acl = “private”
│
│ Use the aws_s3_bucket_acl resource instead
╵
╷
│ Warning: Deprecated attribute
│
│ on cloudfront.tf line 50, in resource “aws_cloudfront_distribution” “neuropharmagen”:
│ 50: domain_name = aws_s3_bucket.neuropharmagen_com.website_endpoint
│
│ The attribute “website_endpoint” is deprecated. Refer to the provider
│ documentation for details.
╵
╷
│ Warning: Resource targeting is in effect
│
│ You are creating a plan with the -target option, which means that the
│ result of this plan may not represent all of the changes requested by the
│ current configuration.
│
│ The -target option is not for routine use, and is provided only for
│ exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
╵
╷
│ Warning: Argument is deprecated
│
│ with module.production.aws_s3_bucket.aina-labs_com,
│ on .terraform/modules/production/pgx/s3.tf line 113, in resource “aws_s3_bucket” “aina-labs_com”:
│ 113: resource “aws_s3_bucket” “aina-labs_com” {
│
│ Use the aws_s3_bucket_website_configuration resource instead
╵
╷
│ Warning: Argument is deprecated
│
│ with module.production.aws_eip.nat_eip,
│ on .terraform/modules/production/pgx/vpc.tf line 45, in resource “aws_eip” “nat_eip”:
│ 45: vpc = true
│
│ use domain attribute instead
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.module.ecs.aws_ecs_cluster.this[0]” from
│ prior state: unsupported attribute “capacity_providers”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.module.vpc.aws_route_table.public[0]” from
│ prior state: unsupported attribute “instance_id”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.module.vpc.aws_vpc.this[0]” from prior
│ state: unsupported attribute “enable_classiclink”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.module.vpc.aws_route_table.private[0]” from
│ prior state: unsupported attribute “instance_id”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.transit_vpc.aws_route_table.private[0]” from prior
│ state: unsupported attribute “instance_id”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.transit_vpc.aws_route_table.public[0]” from prior
│ state: unsupported attribute “instance_id”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “aws_iam_role.musc” from prior state: unsupported attribute
│ “role_last_used”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.aws_iam_role.rds_enhanced_monitoring” from
│ prior state: unsupported attribute “role_last_used”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.aws_secretsmanager_secret.db_password” from
│ prior state: unsupported attribute “rotation_enabled”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.aws_iam_role.nfg_ec2_role” from prior
│ state: unsupported attribute “role_last_used”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.aws_autoscaling_group.app_asg” from prior
│ state: unsupported attribute “tags”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.aws_secretsmanager_secret.app_creds” from
│ prior state: unsupported attribute “rotation_enabled”
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding “module.nfg-prod.aws_iam_role.nfg_ecs_role” from prior
│ state: unsupported attribute “role_last_used”
╵
Failed generating plan JSON
Exit code: 1

Failed to marshal plan to json: error marshaling prior state: unsupported attribute “role_last_used”
Operation failed: failed generating plan JSON: failed running command (exit 1)

My TF version:
Terraform v1.7.0
on darwin_amd64

Please help and thank you!

2

Hi @mahmud.rahimberganov,

Are you using a current release of Terraform? That may account for some of the errors, but it looks like you made a change to the provider as well.

The changes to resource attributes are something you need to coordinate when you upgrade the provider to a new major version. The major releases have upgrade guides in the documentation, listing any breaking changes and modifications you need to make to the configuration.

3

Thank you for your responce!

This is my provider.tf

terraform {
cloud {
organization = “PG”

workspaces {
  name = "production"
}

}

required_providers {
aws = {
source = “hashicorp/aws”
version = “~> 5.0”
}
}
}

Configure the AWS Provider

provider “aws” {
region = “us-east-1”
}

4

Thanks for the update @mahmud.rahimberganov,

There is an existing issue similar to this here: Terraform cannot convert targeted plan to json output · Issue #23297 · hashicorp/terraform · GitHub.

What commands exactly did you use to generate the plan and display the json format?

5

Yes, you’re right. I am having a similar error. I was using this tf code for over a year now, never had an issue. The ‘dev’ environment is working fine, although they use the same code. I don’t know what changed, why it started failing. Evrythinhg started few days ago, when I did ‘tf init’ it said I needed to upgrade. After I upgraded, smth happened. Can’t figure it out why

Thank you!

6

If you have recently upgraded the provider across a major version, some other actions may be necessary, which would be laid out in the Terraform AWS Provider Version 5 Upgrade Guide

To begin with you have a number of warnings about deprecated attributes which need to be removed from the configuration. Regarding the error, what commands exactly did you use to generate the plan and display the json format?

7

I just did ‘tf init’ and ‘tf plan -target module.production’

8

I tried to remove from the state
╵
~/github/terraform/terraform/production (the_latest :heavy_check_mark:) tf state rm “module.transit_vpc.aws_route_table.public[0]”
╷
│ Error: Incompatible Terraform version
│
│ The local Terraform version (1.7.0) does not meet the version requirements for remote workspace Precision-Genetics/production (~> 1.4.0).
│
│ If you’re sure you want to upgrade the state, you can force Terraform to continue using the -ignore-remote-version flag. This may result in an unusable workspace.
╵

Error loading the state: Error checking remote Terraform version

Please ensure that your Terraform state exists and that you’ve
configured it properly. You can use the “-state” flag to point
Terraform at another state file.
~/github/terraform/terraform/production (the_latest :heavy_check_mark:)

9

You must be able to apply a clean plan without the use of -target in order to complete the provider upgrade. I don’t know what your intent is here, but removing that resource from the state will cause Terraform to forget the data and plan to recreate it (which will probably cause the provider to return an error in most cases when the resource instance conflicts with existing resources)

As for the other error, you have a required Terraform version of 1.4, so you cannot upgrade until you change the requirement to match.

10

this my version local:
~/github/terraform/terraform/production (the_latest :heavy_check_mark:) tf --version
Terraform v1.7.0
on darwin_amd64

code:
required_providers {
aws = {
source = “hashicorp/aws”
version = “~> 5.0”
}
}

TF Cloud: 1.4.3

Which one should I change?

THanks again!

11

My intent is to apply on to production env, not the other one in prod folder. Ihave two environments here
And, I just did tf plan without targeting, and got these errors:

Error: Cycle: module.nfg-prod.module.alb.var.create_lb (expand), module.nfg-prod.module.alb.var.putin_khuylo (expand), module.nfg-prod.module.alb.local.create_lb (expand), module.nfg-prod.module.alb.output.lb_zone_id (expand), module.nfg-prod.module.alb.output.lb_dns_name (expand), module.nfg-prod.aws_route53_record.www, module.nfg-prod.aws_route53_zone.primary, module.nfg-prod.aws_route53_record.cert_dns_validation (expand), module.nfg-prod.aws_route53_record.cert_dns_validation[“nfg.precisiongenetics.com”], module.nfg-prod.module.alb.output.lb_arn (expand), module.nfg-prod.aws_lb_listener.lb_listener, module.nfg-prod.aws_acm_certificate_validation.cert_validation (destroy deposed f585cf37), module.nfg-prod.aws_route53_record.cert_dns_validation[“nfg.aina-labs.com”] (destroy), module.nfg-prod.aws_acm_certificate.cert, module.nfg-prod.aws_acm_certificate_validation.cert_validation, module.nfg-prod.module.alb (expand), module.nfg-prod.module.alb.aws_lb.this (expand)

NFG-prod is my other env in this folder

12

You will need to fix your configuration before you can move forward. You cannot have cycles in the configuration, and you should be able to apply the configuration as a whole. If you have unrelated parts of the configuration which are not applied together, they should be separated into individual modules, not managed with -target. Note the warnings from using the -target flag:

│ The -target option is not for routine use, and is provided only for
│ exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
13

But, what should be fixed though? tf fmt and tf validate gives me ‘success’. And I did apply the same code to my other folder, which has ‘dev’ and ‘staging’. Prefectly fine! Do you think I need to fix ‘nfg-prod’ env code?

~/github/terraform/terraform/production (the_latest :heavy_check_mark:) tf validate
╷
│ Warning: Deprecated attribute
│
│ on cloudfront.tf line 3, in resource “aws_cloudfront_distribution” “www_neuropharmmagen”:
│ 3: domain_name = aws_s3_bucket.www_neuropharmagen_com.website_endpoint
│
│ The attribute “website_endpoint” is deprecated. Refer to the provider documentation for details.
│
│ (and 3 more similar warnings elsewhere)
╵
╷
│ Warning: Argument is deprecated
│
│ with aws_s3_bucket.neuropharmagen_com,
│ on s3.tf line 11, in resource “aws_s3_bucket” “neuropharmagen_com”:
│ 11: resource “aws_s3_bucket” “neuropharmagen_com” {
│
│ Use the aws_s3_bucket_website_configuration resource instead
│
│ (and 10 more similar warnings elsewhere)
╵
Success! The configuration is valid, but there were some validation warnings as shown above

14

And I do agree, I should keep them separately

15

Hi @mahmud.rahimberganov,

Your original question included a set of warnings about deprecated arguments, and one error about incorrect data in the Terraform state.

The warnings and the error are related in that they are both caused by an upgrade to a new version of the hashicorp/aws provider, but I expect you should be able to resolve the error without first resolving the warnings.

To resolve the error, you will need to run terraform apply with no -target option to allow the AWS provider to upgrade the state format for every resource currently tracked in your Terraform state. The error is caused by the fact that your -target option excluded some resources from the planning step, which therefore prevented Terraform from asking the provider to upgrade their state representations. You must complete a full plan and apply after a provider major version upgrade to ensure that everything in the state has been upgraded.

Once you’ve completed your full apply to upgrade the state, you can then work on these deprecation warnings separately. Each deprecation warning includes a small note about what to do, although some of them just refer to the provider upgrade guides, which are here:

I’ve included all three links above because I’m not sure what version of the provider you were previously running. If you were previously running a 3.x version, for example, then you’ll need to read both the 4.x and the 5.x upgrade guide to complete your upgrade to the 5.x series.

However, because these are just warnings rather than errors, you can work on them gradually rather than having to fix them all at once.

16

Hello gentlemen,
Thank you so much for your help/advise. Will try to run apply and let you know what happens.

17

just ran ‘tf apply’.
this is my main.tf:

module “production” {
source = “git@github.com:precisiongenetics/terraform-modules.git//pgx”

env = “production”
transit_vpc_id = module.transit_vpc.vpc_id
transit_gateway_id = module.tgw.ec2_transit_gateway_id
tgw_route_table_id = module.tgw.ec2_transit_gateway_route_table_id
tgw_cidr = module.transit_vpc.vpc_cidr_block
transit_vpc_tgwa_id = module.tgw.ec2_transit_gateway_vpc_attachment_ids[0]

ecs_ssh_key = “precision-prod”

private_subnets = [“10.2.1.0/24”, “10.2.2.0/24”, “10.2.3.0/24”]
public_subnets = [“10.2.101.0/24”, “10.2.102.0/24”, “10.2.103.0/24”]
vpc_cidr = “10.2.0.0/16”

main_cluster_count = 5
health_cluster_count = 3
tasks_count = {
“beat” = “1”
“reports” = “2”
“kahuna” = “3”
“jobs” = “3”
“aina” = “3”
“migrator” = “1”
“email” = “2”
“health” = “5”
“health_worker” = “7”
}

aws_account_id = “732191791819”
database_name = “questis”
database_username = “questis_prod”
database_snapshot_arn = “arn:aws:rds:us-east-1:732191791819:cluster-snapshot:deploy-night-march-14-2023”
}

module “nfg-prod” {
source = “git@github.com:precisiongenetics/terraform-modules.git//nfg?ref=tests”

env = “prod”
subdomain_prefix = “nfg.”
image_tag = “production”

tgw_cidr = module.transit_vpc.vpc_cidr_block
transit_gateway_id = module.tgw.ec2_transit_gateway_id
tgw_route_table_id = module.tgw.ec2_transit_gateway_route_table_id
transit_vpc_tgwa_id = module.tgw.ec2_transit_gateway_vpc_attachment_ids[0]
ecs_ssh_key = “precision-prod”

private_subnets = [“10.12.1.0/24”, “10.12.2.0/24”, “10.12.3.0/24”]
public_subnets = [“10.12.101.0/24”, “10.12.102.0/24”, “10.12.103.0/24”]
vpc_cidr = “10.12.0.0/16”
}

and this is the error I got:

╷
│ Error: Cycle: module.nfg-prod.module.alb.output.lb_zone_id (expand), module.nfg-prod.module.alb.output.lb_dns_name (expand), module.nfg-prod.aws_route53_record.www, module.nfg-prod.module.alb.var.putin_khuylo (expand), module.nfg-prod.module.alb.var.create_lb (expand), module.nfg-prod.module.alb.local.create_lb (expand), module.nfg-prod.module.alb.aws_lb.this (expand), module.nfg-prod.aws_route53_record.cert_dns_validation (expand), module.nfg-prod.aws_route53_record.cert_dns_validation[“nfg.precisiongenetics.com”], module.nfg-prod.aws_acm_certificate.cert, module.nfg-prod.aws_acm_certificate_validation.cert_validation, module.nfg-prod.module.alb (expand), module.nfg-prod.module.alb.output.lb_arn (expand), module.nfg-prod.aws_lb_listener.lb_listener, module.nfg-prod.aws_acm_certificate_validation.cert_validation (destroy deposed d1c59e30), module.nfg-prod.aws_route53_record.cert_dns_validation[“nfg.aina-labs.com”] (destroy), module.nfg-prod.aws_route53_zone.primary
│
│

18

I have the same main.tf in ‘dev’ folder where I keep files for ‘dev’ and ‘staging’. They run with no errors

19

Without the configuration it’s very hard to debug this, but the inclusion of destroy and destroy deposed nodes indicates that these could be leftover from some prior refactoring, but were not cleaned out correctly due to repeated use of -target.

You may be able to -target those resources specifically to aid in their removal, or it may be easier to manually delete the actual objects and remove them from the state file.

If the module.nfg-prod.aws_route53_record.cert_dns_validation resource does still exist in the configuration, and still contributes to cycles during evaluation, you will need to inspect the configuration and refactor some of the references to break the cycle. It’s hard to say what that might entail without spending some time with full access to the configuration.

20

Thank you very much JBardin!