We use random_password resource to generate password, Seems “>”, “<”, and “&” are always converted into unicode in terraform state file. But random_password do not generated uncoide iteself. these special characters are shown as is. We do not know if other special characters will be converted too (based on my experiment, seems not). Anyone knows why, and how to avoid that?
This is a valid JSON representation of those characters. Any real JSON parser code will successfully parse the \u sequences - therefore it shouldn’t be a problem. If it is, the fix is to use a real JSON parser to read the state rather than trying to ad-hoc cut a sequence of characters out of the file using basic string manipulation.
The reason these characters are being escaped, is because Go’s default handling of JSON is to escape a few characters such that if it is embedded with an HTML document, it will still be processed correctly: https://pkg.go.dev/encoding/json#HTMLEscape
<, >, &, U+2028 and U+2029 characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029 so that the JSON will be safe to embed inside HTML tags.
Hi maxb,
Thanks for the quick replay and very useful info. Here is the resource we are using. Do you think it is their bug? (it is from microsoft azure library). The reason I am asking is the password set in our Azure database seems has unicode in it. Well, we cannot unencrypt the password to see what it really is, but we have to reset the password manually using non-unicode one, and connection is successful. So we suspect it is the problem.
resource "azurerm_postgresql_flexible_server" "example" {
name = "example-psqlflexibleserver"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
version = "12"
delegated_subnet_id = azurerm_subnet.example.id
private_dns_zone_id = azurerm_private_dns_zone.example.id
administrator_login = "psqladmin"
administrator_password = "H@Sh1CoR3!" #here we use the password generated from random_password resource we have.
zone = "1"
storage_mb = 32768
sku_name = "GP_Standard_D4s_v3"
depends_on = [azurerm_private_dns_zone_virtual_network_link.example]}
I think we should put aside the question of how this is recorded in the state file because that is an implementation detail of Terraform and not something that should matter for you as a user of Terraform.
Can you instead describe what you are trying to achieve and what problem you are encountering while doing so? Whatever you are trying to do should be done some other way than by reading the latest state snapshot, because that artifact is for Terraform’s own use and not something intended for you to access directly.
We manually open the .tf state file and find the password in state file and use it for database connection. If there is any Unicode inside, we have to convert it into ascii. This is apparently a bad approach. But have not found a better way to do it. If you have any suggestions, please let me know. Thank you.
Have you tried to use the decoded version of the state file password before changing it?
If it didn’t work, it may be a problem with the provider accepting characters that aren’t really supported by the backend.
I know the AWS provider validates the characters in the RDS resource password but not sure if the Azure RM one does.
I’m assuming you’ve already tested your code with some manually crafted string combinations (with and without <, > and &) to make sure it’s a provider issue and not your module’s problem.
You can either use Terraform outputs to print out the password (which may be tricky depending on how it’s being applied and how the output is logged - because that’s a sensitive value you need to use nonsensitive) or you can inspect the Terraform state using the CLI command terraform state show ADDRESS (reference)
If you need to use any value generated by Terraform for ongoing work outside of Terraform then the typical answer is to define an output value in your root module which explicitly exports that value for external use.
I assume this is a value that Terraform will consider to be “sensitive” and so it won’t show it in the terraform apply output by default, but after terraform apply is complete you can retrieve the raw value (without any escaping) using a command like this:
terraform output -raw name
Where the “name” at the end matches the name of the output value you declared. For example if you write an output "password" block then the name would be “password”.
We need to use the generated passwords in different environments such as azure, gcp, ibm, and aws. After lot of testing, we narrowed the special characters down to “@^!” so far to make it safe to use. But it is obviously too limited. We will test more combinations. It is even worse than Unicode problem, but it is another topic.