Transform FE/BE nomad job to use sidecar proxies

Hi,

I have successfully deployed a simple job in nomad that has a couple of groups each running one tasks: a frontend and a backend:

// vim: set ft=hcl
job "randomer" {
  datacenters = ["dc1"]

  group "frontend" {
    count = 1

    network {
      port "fe-randomer-port" {
        to = 7050
      }
    }

    service {
      name = "fe-randomer"
      port = "7050"

      check {
        type     = "http"
        path     = "/ping"
        method   = "GET"
        interval = "1s"
        timeout  = "1s"
      }
    }

    task "frontend" {
      driver = "raw_exec"

      config {
        command = "/Users/drio/dev/tufts/consul-local/services/randomer/bin/frontend-randomer"
      }
    }
  }

  group "backend" {
    count = 1

    network {
      port "port" {
        to = 7050
      }
    }

    service {
      name = "be-randomer"
      port = "7050"

      check {
        type     = "http"
        path     = "/ping"
        method   = "GET"
        interval = "1s"
        timeout  = "1s"
      }
    }

    task "backend" {
      driver = "raw_exec"

      config {
        command = "/Users/drio/dev/tufts/consul-local/services/randomer/bin/backend-randomer"
      }
    }
  }
}

Now I want to proxy the traffic between the FE and the BE.

The new nomad job config looks like this:

// vim: set ft=hcl
job "randomer" {
  datacenters = ["dc1"]

  group "backend" {
    count = 1

    network {
      mode = "bridge"
    }

     service {
      name = "be-randomer"
      port = "7050"

      connect {
        sidecar_service {}
      }


      check {
        type     = "http"
        path     = "/ping"
        method   = "GET"
        interval = "1s"
        timeout  = "1s"
      }
    }

    task "backend" {
      driver = "raw_exec"

      config {
        command = "/Users/drio/dev/tu/consul-local/services/randomer/bin/backend-randomer"
      }
    }
  }
 

  group "frontend" {
    count = 1

    network {
      mode = "bridge"
      port "fe-randomer-port" {
        static = 7050
        to = 7050
      }
    }

service {
      name = "fe-randomer"
      port = "7050"

      connect {
        sidecar_service {
          proxy {
            upstreams {
              destination_name = "backend-randomer"
              local_bind_port  = 7061
            }
          }
        }
      }

      check {
        type     = "http"
        path     = "/ping"
        method   = "GET"
        interval = "1s"
        timeout  = "1s"
      }
    }

    task "frontend" {
      driver = "raw_exec"

      config {
        command = "/Users/drio/dev/tu/consul-local/services/randomer/bin/frontend-randomer"
      }
    }
  }

}

Submission output:

Notice the * Constraint “missing drivers”: 1 nodes excluded by filter not sure what that is since I am running a raw_exec driver.

==> 2022-09-15T18:06:17-04:00: Monitoring evaluation "019ee3a5"
    2022-09-15T18:06:17-04:00: Evaluation triggered by job "randomer"
    2022-09-15T18:06:17-04:00: Evaluation within deployment: "b87f592c"
    2022-09-15T18:06:17-04:00: Evaluation status changed: "pending" -> "complete"
==> 2022-09-15T18:06:17-04:00: Evaluation "019ee3a5" finished with status "complete" but failed to place all allocations:
    2022-09-15T18:06:17-04:00: Task Group "backend" (failed to place 1 allocation):
      * Constraint "missing drivers": 1 nodes excluded by filter
    2022-09-15T18:06:17-04:00: Task Group "frontend" (failed to place 1 allocation):
      * Constraint "missing drivers": 1 nodes excluded by filter
    2022-09-15T18:06:17-04:00: Evaluation "4449b038" waiting for additional capacity to place remainder
==> 2022-09-15T18:06:17-04:00: Monitoring deployment "b87f592c"
  ⠇ Deployment "b87f592c" in progress...

    2022-09-15T18:06:22-04:00
    ID          = b87f592c
    Job ID      = randomer
    Job Version = 9
    Status      = running
    Description = Deployment is running

    Deployed
    Task Group  Desired  Placed  Healthy  Unhealthy  Progress Deadline
    backend     1        0       0        0          N/A
    frontend    1        0       0        0          N/A^C

Can you tell me what I am missing here?

Thank you,
-drd

You have to enable the raw_exec driver in your client config: Drivers: Raw Exec | Nomad by HashiCorp

Thank you but I don’t think that is the problem. I am running this in dev mode and I can see nomad detects the raw_exec driver:

    2022-09-16T07:58:43.854-0400 [INFO]  agent: detected plugin: name=raw_exec type=driver plugin_version=0.1.0

Also, the non-proxied nomad job (listed above) works fine and uses the raw_exec driver.

Hi @drio from the path of your command it looks like you are using a mac? If so Connect isn’t going to work - it requires bridge mode (which you have configured), but bridge mode requires Linux, because it is based on Linux network namespaces.

If you run the agent with

# dev mode with convenience configuration for connect 
sudo nomad agent -dev-connect

you’ll likely see a warning about operating system support.

1 Like

Thank you, Seth. Yes, I am in mac. That makes sense.

I think I can still test the proxies if I switch my jobs to use the docker driver right?