How does terraform apply and deploy helm packages with encrypted stuff (helm secrets)

Terraform Providers Kubernetes
1

In the our helm repository, we have a ci/cd with github actions, when we change helm templates or values, then ci will package helm stuff and push it to GCR as an image. see it as following github actions.

name: STAGING-ci-helm
permissions:
  contents: read
  id-token: write
on:
  push:
    branches:
      - staging
  pull_request:
env:
  CHART_NAME: aaaa
  IMAGE_TAG: 0.0.1
jobs:
  job:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: helm lint
        run: |
          helm lint -f aaaa/values-staging.yaml ./aaaa
      - uses: google-github-actions/auth@v0
        with:
          workload_identity_provider: '${{ secrets.WI_POOL_PROVIDER_ID }}'
          service_account: '${{ secrets.PACKAGER_GSA_ID }}'
          token_format: 'access_token'
      - uses: google-github-actions/setup-gcloud@v0
        with:
          version: latest
      - name: login to artifact registry
        run: |
          gcloud auth configure-docker ${{ secrets.ARTIFACT_REGISTRY_HOST_NAME }} --quiet
      - name: helm package
        run: |
          helm package ./aaaa --version $IMAGE_TAG --app-version $IMAGE_TAG
      - name: helm push
        if: ${{ github.event_name == 'push' }}
        run: |
          helm push aaaa-$IMAGE_TAG.tgz oci://${{ secrets.ARTIFACT_REGISTRY_HOST_NAME }}/${{ secrets.PROJECT_ID }}/${{ secrets.ARTIFACT_REGISTRY_REPOSITORY }}

Then in the our terraform repository, we can run a github actions workflow “Terraform Apply” to deploy our cluster with helm provider.

name: "Terraform Staging Apply"

on:
  push:
    branches:
      - staging
  pull_request:
    types:
        - closed
    branches:
      - staging

jobs:
  terraform:
    name: "Terraform Apply"
    runs-on: ubuntu-latest
    permissions: write-all
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.STAGING_AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.STAGING_AWS_SECRET_ACCESS_KEY }}
    defaults:
      run:
        working-directory: ./ci
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

      - name: Terraform Init
        id: init
        run: terraform init

      - name: Terraform Apply
        if: github.ref == 'refs/heads/staging' && github.event_name == 'push'
        run: terraform apply -auto-approve -input=false -refresh=false

In order to combine terraform and helm, so we define 'resource “helm_release” ’ to glue terraform and helm, helm_release will refer to var.aaaa_repository and var.aaaa_version and retrieve our helm package.

resource "helm_release" "aaaa" {
  count = var.install_aaaa_helm_release ? 1 : 0

  # Chart information.
  repository = var.aaaa_repository
  version    = var.aaaa_version
  chart      = "aaaa"

  name              = "aaaa"
  namespace         = var.k8s_namespace
  create_namespace  = !local.is_namespace_default
  dependency_update = true
  force_update      = false
  lint              = true
  recreate_pods     = false
  wait              = false

  values = [
    local_file.helm_values.content
  ]

All the above works well.

Now I will make some changes to support helm secrets, we encrypts some stuff such as credentials with helm secrets (actually encrypted by sops), actually I can manually run “helm secrets install” at local, it works well, that means helm secrets works well.

However, because I change some stuff with encrypted stuff, I know “helm secrets install” or “helm secrets templates” works. Now in the above case, in the above ci/cd pipeline, we use “helm package”, then terraform call package, I am not sure how to make terraform to work with a helm package that has some stuff that has been encrypted by “helm secrets”?

An example as following, I use helm secrets (sops) to encrypt item dockerconfigjson

dockerconfigjson: ENC[AES256_GCM,data:hNWO6G/lMK2kf11kpQjW/VCs40KHWpLgha13cxQ==,iv:vIkA4FXh1iYBYVoH5Xtyp2SNo5XzPatiVlAxnfhcpSA=,tag:rNXGgovTC+VdkpwOkfUrjQ==,type:str]
sops:
    kms:
        - arn: arn:aws:kms:ca-central-1:88888888888:key/055555555-66666-7777-8888-xxxxxxxx
          created_at: "2023-10-12T18:17:08Z"
          enc: AQICAHhlPQ/aFcm7KLIoh1yCQAUgRDt67BhGjRw1f/mkc8yXaQFqweLYkq2dRjjmaUprSV7KAAAAfAgEQgDvYaldhb3vQrH5T4vOvdl6t4p+PHRFYCSo55YiEFWNeYTT1S3d1ufVeBvK9RLlq268FDDLul+FxA1pXfQ==
          aws_profile: ""
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age: []
    lastmodified: "2023-10-12T20:35:14Z"
    mac: ENC[AES256_GCM,data:lOcdqVQOxTJvYNoyHRZ2M4zcyoMZqAmO8bWLzlOiiezKGofC9s9+/sRVXONr4l8HcWo9pYGMfn3DM3YA9zE=,iv:Ht9i1ou+xJ0le2QqlNllKUPgMC3utKkKHPE2jibugrM=,tag:cz1R7TDbM39vMwuqsMFTuQ==,type:str]
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.8.0

Then in the a secret, we refer to that encrypted dockerconfigjson. When I use “helm secrets install”, it works well.

apiVersion: v1
kind: Secret
metadata:
  name: dockercredential
  namespace: {{ .Release.Namespace | quote }}
data:
	.dockerconfigjson: {{ .Values.dockerconfigjson }}
type: kubernetes.io/dockerconfigjson

In short, now we are using “helm package” to package the helm stuff, some manifest (such as the above Secret dockercredential) in the helm template has been encrypted by helm secrets (sops), how to change terraform ‘resource “helm_release” “aaaa”’ stuff so that terraform can fit in encrypted stuff and deploy helm successfully?