Boundary setup initial auth failing when using AWS kms

Hi,

I trying to setup non dev boundary on AWS using ubuntu OS. When I try using AWS KMS for key management. My initial auth login fails with status code 403 . Authentication fail.

My configuration.
boundary.hcl

disable_mlock = true

controller {
name = “controller”
description = “A default controller”

graceful_shutdown_wait_duration = “10s”

database {
url = “”
max_open_connections = 10
}

}

worker {
name = “worker”
description = “A default worker”
}

listener “tcp” {
address = “”
purpose = “api”

tls_disable = true

cors_enabled = true
cors_allowed_origins = [“*”]
}

listener “tcp” {
address = “”
purpose = “cluster”
}

listener “tcp” {
address = “”
purpose = “proxy”
}

listener “tcp” {
address = “”
purpose = “ops”
tls_disable = true
}

kms “awskms” {
purpose = “root,worker-auth,recovery”
region = “ap-south-1”
kms_key_id = “kms key id”
access_key = “”
secret_key = “”

}

Is your controller running properly? If you’re getting an auth login failure it could mean the controller has some problem in its configuration – check the output from the controller process and see if it’s complaining about anything.

Hi, Thanks for quick response.

The controller and worker seems to be up and running.

I get the below error log

{“id”:“23Of1DQ0vE”,“source”:“https://hashicorp.com/boundary/ip-10-13-1-29/controller+worker",“specversion”:“1.0”,“type”:“observation”,“data”:{“latency-ms”:0.297026,“request_info”:{“id”:“gtraceid_7iHzYXgN1SzqBhg0F8NG”,“method”:“POST”,“path”:“/v1/auth-methods/ampw_LZOFMSk0nx:authenticate”,“client_ip”:“117.253.181.207”},“start”:“2022-09-23T15:48:31.833902172Z”,“status”:403,“stop”:“2022-09-23T15:48:31.834199198Z”,“version”:“v0.1”},“datacontentype”:“application/cloudevents”,“time”:"2022-09-23T15:48:31.834237008Z”}

PFA

Sharing few more lines of log . Hopefully helpful to debug.

Also highlighting a log which when running boundary server -config /etc/boundary.d/boundary.hcl.

pki auth rotation ticking not running. Could this be the issue?

{“id”:“wTcxZxbKhP”,“source”:“https://hashicorp.com/boundary/ip-10-13-1-29/controller+worker",“specversion”:“1.0”,“type”:“system”,“data”:{“version”:“v0.1”,“op”:“github.com/hashicorp/boundary/internal/observability/event.(*HclogLoggerAdapter).writeEvent”,“data”:{“@original-log-level”:“none”,“@original-log-name”:“awskms”,“msg”:"configuring client automatic mTLS”,“purpose”:“root”}},“datacontentype”:“application/cloudevents”,“time”:“2022-09-23T16:05:49.304243285Z”}
{“id”:“TnIuQKogHk”,“source”:“https://hashicorp.com/boundary/ip-10-13-1-29/controller+worker",“specversion”:“1.0”,“type”:“system”,“data”:{“version”:“v0.1”,“op”:“worker.(Worker).startAuthRotationTicking”,“data”:{“msg”:"using kms worker authentication; pki auth rotation ticking not running”}},“datacontentype”:“application/cloudevents”,“time”:“2022-09-23T16:05:49.306460085Z”}
{“id”:“mF0OC6HcUU”,“source”:“https://hashicorp.com/boundary/ip-10-13-1-29/controller+worker",“specversion”:“1.0”,“type”:“system”,“data”:{“version”:“v0.1”,“op”:“github.com/hashicorp/boundary/internal/observability/event.(*HclogLoggerAdapter).writeEvent”,“data”:{“@original-log-level”:“none”,“@original-log-name”:“awskms”,“args”:[“/tmp/533080319/boundary-plugin-kms-awskms-8M1WX”],“msg”:"starting plugin”,“path”:“/tmp/533080319/boundary-plugin-kms-awskms-8M1WX”,“purpose”:“root”}},“datacontentype”:“application/cloudevents”,“time”:“2022-09-23T16:05:49.306918783Z”}
{“id”:“0Borl2YNr3”,“source”:“https://hashicorp.com/boundary/ip-10-13-1-29/controller+worker",“specversion”:“1.0”,“type”:“system”,“data”:{“version”:“v0.1”,“op”:“github.com/hashicorp/boundary/internal/observability/event.(*HclogLoggerAdapter).writeEvent”,“data”:{“@original-log-level”:“none”,“@original-log-name”:“awskms”,“msg”:"plugin started”,“path”:“/tmp/533080319/boundary-plugin-kms-awskms-8M1WX”,“pid”:16438,“purpose”:“root”}},“datacontentype”:“application/cloudevents”,“time”:“2022-09-23T16:05:49.307037575Z”}

Do you only get that 403 when you authenticate? The AWS KMS is used during database init and at other times so if you were able to init the database successfully then the AWS KMS should be working.

My guess is that the credentials you’re using are not correct. Are you using the information for the initial credential that was output when you ran the init?

Yes, Database init was successful and I am using the initial credential that was outputted.

The same setup is working with AEAD encryption type

Did you get any output to the event log? Generally if there is an error it will be written to the event log that the controller outputs.

Hi ,

I found out the issue. 403 block was happening due to CORS error . I have made it
cors_allowed_origins = [“*”]. I understand that this could pose a security risk. But I also wanted suggestions on this.