Cannot get container / pod ENV to populate from JSON file in config stanza

the1dv · October 31, 2022, 5:38am

We are trialling Waypoint to replace our fairly convoluted Terraform deployment of containers.
I have raised a few issues on Github regarding our specific use case as they seem like bugs but essentially I am trying to write environment variables from our Terraform deployment into our Waypoint deployment by way of a temporary variables.json file generated by Terraform.

The issue we have is that no matter how we try this, Waypoint never seems to inject the Variables (or the .env file that we tried) and there are no logs to help us understand why.

NEW_RELIC_APP_NAME is an environment variable that is set in our variables.json file that is being added by waypoint.hcl:

app "tymlez-middleware" {
  config {
    env = jsondecode(file("variables.json"))
  }

  build {
    use "docker-pull" {
      image =  join("/", ["australia-southeast1-docker.pkg.dev", var.gcp_project_id, "tymlez-images", "${var.app_name}-tymlez-middleware"])
      tag   = var.git_sha
      #tag = gitrefpretty()
      auth {
        username = "_json_key"
        password = var.gcp_service_account
      }
    }
    registry {
      use "docker" {
        image = join("/", ["australia-southeast1-docker.pkg.dev", var.gcp_project_id, "tymlez-images", "${var.app_name}-tymlez-middleware-waypoint"])
        tag   = "latest"
        auth {
          username = "_json_key"
          password = var.gcp_service_account
        }
      }
    }
  }
  deploy {
    use "kubernetes" {
      namespace = "default"
      probe_path = "/healthcheck"
    }
  }
  release {
    use "kubernetes" {
      load_balancer = true
    }
  }
}

As you can see here, these environment variables are not getting set.

I’m including our Dockerfile here, we don’t use Entrypoint on it but instead CMD, is this an issue?

Are there any other reasons why this file isn’t being populated into the ENV of the container? The JSON is flat (no nesting - just a single object with each variable) - it would decode into a map in HCL.

FROM node:16.17-alpine3.15
RUN apk add --no-cache curl

ARG GIT_SHA=latest
ENV GIT_SHA="${GIT_SHA}"
ENV PORT=3000

RUN addgroup -S tymlez
RUN adduser -S -G tymlez tymlez
RUN chown -R tymlez:tymlez /home/tymlez/
USER tymlez

WORKDIR /app
RUN chown -R tymlez:tymlez /app

COPY --chown=tymlez:tymlez *.* ./
COPY --chown=tymlez:tymlez .yarn ./.yarn
COPY --chown=tymlez:tymlez yarn.lock ./yarn.lock

COPY --chown=tymlez:tymlez packages/common-libs ./packages/common-libs
COPY --chown=tymlez:tymlez packages/backend-libs ./packages/backend-libs
COPY --chown=tymlez:tymlez cli/tymlez-dev-cli ./cli/tymlez-dev-cli
COPY --chown=tymlez:tymlez packages/platform-api-interfaces ./packages/platform-api-interfaces
COPY --chown=tymlez:tymlez packages/trustchain-sdk ./packages/trustchain-sdk


COPY --chown=tymlez:tymlez patches ./patches

COPY --chown=tymlez:tymlez services/platform-middleware ./services/platform-middleware

RUN yarn
WORKDIR /app/services/platform-middleware
RUN yarn build
RUN yarn cache clean
CMD ["printenv"]
EXPOSE 3000
CMD ["yarn", "start:prod"]

2022-10-31 15:33:10.905 AEDT

tymlez-middleware
NEW_RELIC_APP_NAME. Not starting!
2022-10-31 15:33:10.905 AEDT

tymlez-middleware
Set app_name in your newrelic.js or newrelic.cjs file or set environment variable
2022-10-31 15:33:10.905 AEDT

tymlez-middleware
Error: New Relic requires that you name this application!
2022-10-31 15:33:10.904 AEDT

tymlez-middleware
New Relic for Node.js was unable to bootstrap itself due to an error:
2022-10-31 15:33:08.701 AEDT

tymlez-middleware
2022-10-31T04:33:08.701Z [DEBUG] entrypoint.url.agent: connected to hub: addr=52.12.2.131:443
2022-10-31 15:33:04.616 AEDT

tymlez-middleware
2022-10-31T04:33:04.616Z [DEBUG] entrypoint.url.agent: connected successfully: status=connected latency=207.894002ms skew=84.380232ms
2022-10-31 15:33:04.616 AEDT

tymlez-middleware
2022-10-31T04:33:04.616Z [DEBUG] entrypoint.url.agent: connection latency: latency=207.894002ms
2022-10-31 15:33:04.036 AEDT

tymlez-middleware
2022-10-31T04:33:04.036Z [DEBUG] entrypoint.url.agent: connecting to hub: addr=52.12.2.131:443
2022-10-31 15:33:03.674 AEDT

tymlez-middleware
2022-10-31T04:33:03.672Z [INFO] entrypoint.child: starting child process: args=["/usr/local/bin/docker-entrypoint.sh", "yarn", "start:prod"] cmd=/usr/local/bin/docker-entrypoint.sh
2022-10-31 15:33:03.674 AEDT

tymlez-middleware
2022-10-31T04:33:03.672Z [DEBUG] entrypoint.child: child command received
2022-10-31 15:33:03.674 AEDT

tymlez-middleware
2022-10-31T04:33:03.672Z [DEBUG] entrypoint.child: starting child command watch loop
2022-10-31 15:33:03.673 AEDT

tymlez-middleware
2022-10-31T04:33:03.672Z [INFO] entrypoint.config.watcher: env vars changed, sending new child command
2022-10-31 15:33:03.673 AEDT

tymlez-middleware
2022-10-31T04:33:03.672Z [DEBUG] entrypoint.config.watcher.watchloop: new configuration computed
2022-10-31 15:33:03.179 AEDT

tymlez-middleware
2022-10-31T04:33:03.179Z [DEBUG] entrypoint.url: refreshing data
2022-10-31 15:33:03.178 AEDT

tymlez-middleware
2022-10-31T04:33:03.178Z [DEBUG] entrypoint.url: discovering hubs
2022-10-31 15:33:03.172 AEDT

tymlez-middleware
2022-10-31T04:33:03.171Z [DEBUG] entrypoint.url: url service enabled, configuring: addr=https://control.hzn.network service_port=3000 labels=waypoint/workspace=default,waypoint.hashicorp.com/app=tymlez-middleware,waypoint.hashicorp.com/project=tymlez-platform,waypoint.hashicorp.com/workspace=default,waypoint.hashicorp.com/instance-id=01GGP50YX5S39MAKTW0WE8ZW36,:deployment=v2,:deployment-order=01ggp4rehfc9ajpajr18r1qqnr
2022-10-31 15:33:03.171 AEDT

tymlez-middleware
2022-10-31T04:33:03.171Z [DEBUG] entrypoint.config.watcher.watchloop: new config variables received, scheduling refresh
2022-10-31 15:33:03.171 AEDT

tymlez-middleware
2022-10-31T04:33:03.171Z [DEBUG] entrypoint.config_recv: new configuration received
2022-10-31 15:33:03.171 AEDT

tymlez-middleware
2022-10-31T04:33:03.171Z [DEBUG] entrypoint.config_recv: first config received, switching config state to true
2022-10-31 15:33:02.999 AEDT

waypoint
2022-10-31T04:33:02.999Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/GetVersionInfo response: error=<nil> duration=414.72µs
2022-10-31 15:33:02.999 AEDT

waypoint
2022-10-31T04:33:02.999Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/GetVersionInfo request
2022-10-31 15:33:02.998 AEDT

waypoint
2022-10-31T04:33:02.998Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/EntrypointConfig request
2022-10-31 15:33:02.998 AEDT

waypoint
2022-10-31T04:33:02.998Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/EntrypointLogStream request
2022-10-31 15:33:02.996 AEDT

waypoint
2022-10-31T04:33:02.996Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/GetVersionInfo response: error=<nil> duration=133.145µs
2022-10-31 15:33:02.996 AEDT

waypoint
2022-10-31T04:33:02.996Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/GetVersionInfo request
2022-10-31 15:33:02.924 AEDT

waypoint
2022-10-31T04:33:02.924Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/ConvertInviteToken response: error=<nil> duration=3.064951ms
2022-10-31 15:33:02.921 AEDT

waypoint
2022-10-31T04:33:02.921Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/ConvertInviteToken request
2022-10-31 15:32:51.499 AEDT

waypoint
2022-10-31T04:32:51.499Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/EntrypointConfig response: error="context canceled" duration=15.201411034s
2022-10-31 15:32:51.499 AEDT

waypoint
2022-10-31T04:32:51.499Z [INFO] waypoint.server.grpc: /hashicorp.waypoint.Waypoint/EntrypointLogStream response: error="rpc error: code = Canceled desc = context canceled" duration=15.200992667s
2022-10-31 15:32:51.497 AEDT

tymlez-middleware
Error initializing Waypoint entrypoint: exit status 1
2022-10-31 15:32:51.497 AEDT

tymlez-middleware
2022-10-31T04:32:51.497Z [WARN] entrypoint.log: error sending logs: error="rpc error: code = Internal desc = SendMsg called after CloseSend"
2022-10-31 15:32:51.497 AEDT

tymlez-middleware
2022-10-31T04:32:51.497Z [ERROR] entrypoint.config_recv: error receiving configuration, exiting: err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2022-10-31 15:32:51.497 AEDT

tymlez-middleware
2022-10-31T04:32:51.497Z [INFO] entrypoint.child: child process exited on its own: err="exit status 1"
2022-10-31 15:32:51.496 AEDT

tymlez-middleware
2022-10-31T04:32:51.496Z [WARN]  entrypoint.child: subprocess exited: args=["/usr/local/bin/docker-entrypoint.sh", "yarn", "start:prod"] cmd=/usr/local/bin/docker-entrypoint.sh err="exit status 1"

{
insertId: "n3u9y9dj86sm6qwg"
labels: {6}
logName: "projects/tymlez-dev-240712/logs/stderr"
receiveTimestamp: "2022-10-31T04:32:55.689887427Z"
resource: {2}
severity: "ERROR"
textPayload: "2022-10-31T04:32:51.496Z [WARN]  entrypoint.child: subprocess exited: args=["/usr/local/bin/docker-entrypoint.sh", "yarn", "start:prod"] cmd=/usr/local/bin/docker-entrypoint.sh err="exit status 1""
timestamp: "2022-10-31T04:32:51.496922138Z"
}
2022-10-31 15:32:51.420 AEDT

tymlez-middleware
}

the1dv · November 1, 2022, 4:26am

I have since updated our docker to use ENTRYPOINT instead and put a printenv command in there.

I have raised 2 bugs for both approached I have tried that have some additional information in them:

github.com/hashicorp/waypoint

Cannot set environment variables using jsondecode in Config Stanza

opened 02:08AM - 01 Nov 22 UTC

voycey

new

I assume this is related to #4135 I have tried the following: ```hcl app …"tymlez-middleware" { config { env = jsondecode(file("variables.json")) # env = { # "APM_ENABLED": true, # "DATABASE_HOST": "1.1.1.1", # "DATABASE_PASSWORD": "password", # "DATABASE_PORT": 5432, # "DATABASE_TYPE": "postgres", # "DATABASE_USERNAME": "sqluser@postgres.com", # "NEW_RELIC_APPLICATION_LOGGING_FORWARDING_ENABLED": true, # "NEW_RELIC_APPLICATION_LOGGING_LOCAL_DECORATING_ENABLED": true, # "NEW_RELIC_APP_NAME": "dev-platform-platform-terraform-tymlez-__container__name__" # } } build { use "docker-pull" { ``` Variables set just fine when hardcoded as a map. When using the same file as `jsondecode(file("variables.json"))` this fails to create or insert the variables. I have tested that the file deserializes correctly into a map via Terraform Console: ``` > jsondecode(file("variables.json")) { "APM_ENABLED" = true "ASSET_BUCKET_NAME" = "asset_bucket_name" ... ``` In the logs the container starts up and I have a `printenv` call in there to print the environment - I can see the WAYPOINT vars but none of the values from `variables.json` Is this a limitation or am I doing something wrong? I have now tried 5 different methods to get the container environment populated with dynamic values and unfortunately all of them have failed in some way :(

github.com/hashicorp/waypoint

Cannot write environment file via config stanza?

opened 01:29AM - 31 Oct 22 UTC

voycey

new

When trying to write a file containing environment variables (standard .env form…at) via the config stanza I am getting the following error: ``` **2022-10-31T01:07:16.648Z [ERROR] entrypoint.config.watcher.watchloop: error evaluating expression: var=.env error=".env:1,15-19: Variables not allowed; Variables may not be used here."** ``` Is waypoint trying to parse this file for some reason? It shouldn't care what the contents of that file are surely? Code I am using is: ```hcl app "tymlez-middleware" { config { file = { ".env" = file("${path.cwd}/.env") } } ... ``` Also tried in case serializing it helps (no luck): ```hcl app "tymlez-middleware" { config { file = { ".env" = filebase64("${path.cwd}/.env_b64") } } ... ```