HCSEC-2021-12 - Codecov Security Event and HashiCorp GPG Key Exposure

Frequently Asked Questions (added April 27 2021)

Incident response activities are ongoing, and relevant updates and outcomes will be shared promptly when available via https://discuss.hashicorp.com/c/security.

Has any HashiCorp customer data been disclosed?

There is no evidence of HashiCorp customer data disclosure at this point in time.

Was HashiCorp source code and/or binaries maliciously modified?

There is no evidence of malicious modification to HashiCorp code or binaries at this point in time.

What steps should HashiCorp customers/users consider taking?

In general, HashiCorp customers/users should ensure that they download HashiCorp products only from the official release channel accessible directly at https://releases.hashicorp.com or linked from HashiCorp web properties.

In environments where HashiCorp product downloads are manually or automatically validated using the SHA256SUM files and associated signatures, process or configuration updates may be necessary to reflect the change in HashiCorp’s GPG key.

HashiCorp has provided separate Terraform-specific guidance. Customers should consider upgrading to Terraform v0.11.15, v0.12.31, v0.13.7, v0.14.11, and v0.15.1 which have been released and use the new GPG key for provider validation.

What exactly was HashiCorp’s exposure?

The Codecov Bash Uploader and associated affected components as described in the Codecov disclosure was enabled for a small subset of HashiCorp-owned source code repositories and associated build pipelines.

Per the Codecov disclosure, the unauthorized alterations to their Bash Uploader enabled a third party to potentially export information stored in their users’ continuous integration (CI) environments. This information (specifically, repository names/locations and environment variables) potentially could have been sent to a third-party server outside of Codecov’s infrastructure.

On review of the CI environments for the affected HashiCorp repositories, a number of environment variables containing sensitive secrets (including the HashiCorp GPG private key used for signing release hashes) were determined to be potentially exposed.

How have HashiCorp sources, builds, releases, and/or binaries been verified?

The immediate focus has been on verification of existing builds, releases, and binaries. Analysis of logs, signatures, and storage system metadata in conjunction with comparison to known-good copies did not uncover evidence of malicious modification.

The secondary focus has been on verification of source code. Activity around HashiCorp source code repositories for the window of exposure has been and will continue to be reviewed.

There is no evidence of malicious modification to HashiCorp code or binaries at this point in time.

What was the timeline?

The Codecov disclosure was posted on April 15, 2021 and stated unauthorized third-party access occurred between January 31, 2021 and April 1, 2021.

HashiCorp began response immediately on review of the Codecov disclosure on April 15, 2021. Various investigative and remedial activities have been undertaken and continue.

HashiCorp rotated and revoked the exposed GPG key, re-signed the majority of existing product releases with the new GPG key, and published a public security bulletin on April 22, 2021.

HashiCorp released updated Terraform binaries with updated GPG keys on April 26, 2021.