Nomad job - google cloud storage artifact

Hi everyone,

We have a nomad task that is supposed to start a java application and artifact block to download the jar file from a private google cloud storage.
The problem we face is that the authentication to Google Cloud Storage fails with
“failed to download artifact “gcs::”: googleapi: Error 403: Insufficient Permission, insufficientPermissions”

We tried using:
GOOGLE_CREDENTIALS using stringified credentials string
GOOGLE_APPLICATION_CREDENTIALS point to the service account json file available to the task

None of above worked showing 403 insufficient permissions.

This is our current config:
task “app” {

  driver = "java"
  config {
    jar_path    = "local/our-file.jar"
    jvm_options = ["-Xmx2048m", "-Xms256m"]
  env {
  artifact {
    source = "gcs::"

What is more interesting is that we downloaded the go-getter module, and ran it locally - it did work given that either GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_OAUTH_TOKEN were present as env variables.

Does anyone have experience using artifact block pointing to non-public Google Cloud Storage bucket?

Thank you!

1 Like

Hi @ivan - for private GCS our go-getter library is a bit less convenient to use than private S3, etc.

The basic problem is the environment variables need to be set on the Nomad Client agent - not on the job spec. If you’re running Nomad under systemd, you’d add them to the unit file, e.g.


Starting with Nomad 1.5, you’ll also need to allow the artifact download sandbox to have access to this environment variable by setting artifact.set_environment_variables in Client configuration - nomad/upgrade-specific.mdx at v1.5.0-rc.1 · hashicorp/nomad · GitHub

Hi Seth,

Many thanks for your clarification!
We followed the steps using GOOGLE_APPLICATION_CREDENTIALS (since the OAUTH Token is subject to expiration). We’re using Nomad 1.4.4.

  1. In client.hcl:
    client {
    enabled = true
    node_class = “hashistack”

options {
“driver.raw_exec.enable” = “1”
“docker.privileged.enabled” = “true”
“env.denylist” = “CUSTOM_VAR”
“set_environment_variables” = “GOOGLE_APPLICATION_CREDENTIALS”
2) In systemd service definition
ExecStart=/usr/local/bin/nomad agent -config=“/etc/nomad.d/nomad.hcl”
ExecReload=/bin/kill -HUP $MAINPID

  1. we restarted the nomad service
  2. we ran our job - same problem:
    “failed to download artifact “gcs::”: googleapi: Error 403: Insufficient Permission, insufficientPermissions”

Can you please advise/or if you have a working end-to-end example:

  2. if you plan to introduce the same integration capabilities for Cloud Storage like AWS S3 (EC2 IAM Instance profiles) - so that go-getter library can download the GCS artifact using the Compute Engine Service account directly?

Reason I ask - is because having credentials stored directly on the host is not a recommended security practice by Google.

Thank you in advance!