podman logs 13a2cfa1d64f6c471ed84d682ea1fcb28c8b75fe62779eb13e71e3158866d1ac
==> No configuration files loaded
==> Starting Nomad agent...
==> Nomad agent configuration:

       Advertise Addrs: HTTP: 10.89.0.91:4646; RPC: 10.89.0.91:4647; Serf: 10.89.0.91:4648
            Bind Addrs: HTTP: [0.0.0.0:4646]; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648
                Client: true
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: true
               Version: 1.4.2

==> Nomad agent started! Log data will stream in below:

    2022-11-12T07:06:44.080Z [DEBUG] agent.plugin_loader: starting plugin: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman args=["/usr/lib/nomad/plugins/nomad-driver-podman"]
    2022-11-12T07:06:44.080Z [DEBUG] agent.plugin_loader: plugin started: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman pid=15
    2022-11-12T07:06:44.080Z [DEBUG] agent.plugin_loader: waiting for RPC address: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman
    2022-11-12T07:06:44.109Z [DEBUG] agent.plugin_loader.nomad-driver-podman: plugin address: plugin_dir=/usr/lib/nomad/plugins address=/tmp/plugin3841138431 network=unix timestamp=2022-11-12T07:06:44.109Z
    2022-11-12T07:06:44.110Z [DEBUG] agent.plugin_loader: using plugin: plugin_dir=/usr/lib/nomad/plugins version=2
    2022-11-12T07:06:44.123Z [DEBUG] agent.plugin_loader: plugin process exited: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman pid=15
    2022-11-12T07:06:44.123Z [DEBUG] agent.plugin_loader: plugin exited: plugin_dir=/usr/lib/nomad/plugins
    2022-11-12T07:06:44.124Z [DEBUG] agent.plugin_loader.docker: using client connection initialized from environment: plugin_dir=/usr/lib/nomad/plugins
    2022-11-12T07:06:44.124Z [DEBUG] agent.plugin_loader.docker: using client connection initialized from environment: plugin_dir=/usr/lib/nomad/plugins
    2022-11-12T07:06:44.124Z [DEBUG] agent.plugin_loader: starting plugin: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman args=["/usr/lib/nomad/plugins/nomad-driver-podman"]
    2022-11-12T07:06:44.124Z [DEBUG] agent.plugin_loader: plugin started: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman pid=22
    2022-11-12T07:06:44.124Z [DEBUG] agent.plugin_loader: waiting for RPC address: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman
    2022-11-12T07:06:44.124Z [DEBUG] agent.plugin_loader.stdio: received EOF, stopping recv loop: plugin_dir=/usr/lib/nomad/plugins err="rpc error: code = Unavailable desc = error reading from server: EOF"
    2022-11-12T07:06:44.137Z [DEBUG] agent.plugin_loader.nomad-driver-podman: plugin address: plugin_dir=/usr/lib/nomad/plugins address=/tmp/plugin3564943711 network=unix timestamp=2022-11-12T07:06:44.137Z
    2022-11-12T07:06:44.137Z [DEBUG] agent.plugin_loader: using plugin: plugin_dir=/usr/lib/nomad/plugins version=2
    2022-11-12T07:06:44.138Z [DEBUG] agent.plugin_loader.nomad-driver-podman: http baseurl: plugin_dir=/usr/lib/nomad/plugins @module=podman url=unix:/run/podman/podman.sock timestamp=2022-11-12T07:06:44.138Z
    2022-11-12T07:06:44.138Z [DEBUG] agent.plugin_loader.nomad-driver-podman: http baseurl: plugin_dir=/usr/lib/nomad/plugins @module=podman url=unix:/run/podman/podman.sock timestamp=2022-11-12T07:06:44.138Z
    2022-11-12T07:06:44.139Z [DEBUG] agent.plugin_loader.stdio: received EOF, stopping recv loop: plugin_dir=/usr/lib/nomad/plugins err="rpc error: code = Unavailable desc = error reading from server: EOF"
    2022-11-12T07:06:44.140Z [DEBUG] agent.plugin_loader: plugin process exited: plugin_dir=/usr/lib/nomad/plugins path=/usr/lib/nomad/plugins/nomad-driver-podman pid=22
    2022-11-12T07:06:44.140Z [DEBUG] agent.plugin_loader: plugin exited: plugin_dir=/usr/lib/nomad/plugins
    2022-11-12T07:06:44.140Z [INFO]  agent: detected plugin: name=podman type=driver plugin_version=0.4.0
    2022-11-12T07:06:44.140Z [INFO]  agent: detected plugin: name=qemu type=driver plugin_version=0.1.0
    2022-11-12T07:06:44.140Z [INFO]  agent: detected plugin: name=java type=driver plugin_version=0.1.0
    2022-11-12T07:06:44.140Z [INFO]  agent: detected plugin: name=docker type=driver plugin_version=0.1.0
    2022-11-12T07:06:44.140Z [INFO]  agent: detected plugin: name=raw_exec type=driver plugin_version=0.1.0
    2022-11-12T07:06:44.140Z [INFO]  agent: detected plugin: name=exec type=driver plugin_version=0.1.0
    2022-11-12T07:06:44.141Z [INFO]  nomad.raft: initial configuration: index=1 servers="[{Suffrage:Voter ID:2e23ccec-20fc-a3e1-500d-e7824c018a78 Address:10.89.0.91:4647}]"
    2022-11-12T07:06:44.141Z [INFO]  nomad: serf: EventMemberJoin: 13a2cfa1d64f.global 10.89.0.91
    2022-11-12T07:06:44.141Z [INFO]  nomad: starting scheduling worker(s): num_workers=1 schedulers=["service", "batch", "system", "sysbatch", "_core"]
    2022-11-12T07:06:44.141Z [DEBUG] nomad: started scheduling worker: id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef index=1 of=1
    2022-11-12T07:06:44.141Z [INFO]  nomad: started scheduling worker(s): num_workers=1 schedulers=["service", "batch", "system", "sysbatch", "_core"]
    2022-11-12T07:06:44.141Z [ERROR] client.cpuset.v2: failed to enabled minimum set of cgroup controllers; disabling cpuset management: error="write /sys/fs/cgroup/cgroup.subtree_control: no such file or directory"
    2022-11-12T07:06:44.141Z [INFO]  client: using state directory: state_dir=/tmp/NomadClient3035827923
    2022-11-12T07:06:44.142Z [INFO]  client: using alloc directory: alloc_dir=/tmp/NomadClient2533225955
    2022-11-12T07:06:44.142Z [INFO]  client: using dynamic ports: min=20000 max=32000 reserved=""
    2022-11-12T07:06:44.142Z [DEBUG] client.fingerprint_mgr: built-in fingerprints: fingerprinters=["arch", "bridge", "cgroup", "cni", "consul", "cpu", "host", "memory", "network", "nomad", "signal", "storage", "vault", "env_digitalocean", "env_aws", "env_gce", "env_azure"]
    2022-11-12T07:06:44.142Z [INFO]  client.fingerprint_mgr.cgroup: cgroups are available
    2022-11-12T07:06:44.142Z [DEBUG] client.fingerprint_mgr: CNI config dir is not set or does not exist, skipping: cni_config_dir=/opt/cni/config
    2022-11-12T07:06:44.149Z [INFO]  nomad.raft: entering follower state: follower="Node at 10.89.0.91:4647 [Follower]" leader-address= leader-id=
    2022-11-12T07:06:44.150Z [DEBUG] worker: running: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
    2022-11-12T07:06:44.151Z [INFO]  nomad: adding server: server="13a2cfa1d64f.global (Addr: 10.89.0.91:4647) (DC: dc1)"
    2022-11-12T07:06:44.151Z [DEBUG] nomad.keyring.replicator: starting encryption key replication
    2022-11-12T07:06:44.151Z [DEBUG] client.fingerprint_mgr: fingerprinting periodically: fingerprinter=cgroup initial_period=15s
    2022-11-12T07:06:44.165Z [INFO]  client.fingerprint_mgr.consul: consul agent is available
    2022-11-12T07:06:44.165Z [DEBUG] client.fingerprint_mgr.cpu: detected core count: cores=1
    2022-11-12T07:06:44.165Z [WARN]  client.fingerprint_mgr.cpu: failed to detect set of reservable cores: error="openat2 /sys/fs/cgroup/nomad.slice/cpuset.cpus.effective: no such file or directory"
    2022-11-12T07:06:44.165Z [INFO]  client.fingerprint_mgr.cpu: fallback to default cpu total compute, set client config option cpu_total_compute to override
    2022-11-12T07:06:44.165Z [DEBUG] client.fingerprint_mgr.network: link speed detected: interface=eth0 mbits=10000
    2022-11-12T07:06:44.167Z [DEBUG] client.fingerprint_mgr.network: detected interface IP: interface=eth0 IP=10.89.0.91
    2022-11-12T07:06:44.167Z [DEBUG] client.fingerprint_mgr.network: unable to read link speed: path=/sys/class/net/lo/speed device=lo
    2022-11-12T07:06:44.167Z [DEBUG] client.fingerprint_mgr.network: link speed could not be detected, falling back to default speed: interface=lo mbits=1000
    2022-11-12T07:06:44.167Z [DEBUG] client.fingerprint_mgr: fingerprinting periodically: fingerprinter=consul initial_period=15s
    2022-11-12T07:06:44.172Z [DEBUG] client.fingerprint_mgr: fingerprinting periodically: fingerprinter=vault initial_period=15s
    2022-11-12T07:06:44.176Z [DEBUG] client.fingerprint_mgr.env_gce: could not read value for attribute: attribute=machine-type error="Get \"http://169.254.169.254/computeMetadata/v1/instance/machine-type\": dial tcp 169.254.169.254:80: connect: connection refused"
    2022-11-12T07:06:44.176Z [DEBUG] client.fingerprint_mgr.env_gce: error querying GCE Metadata URL, skipping
    2022-11-12T07:06:44.176Z [DEBUG] client.fingerprint_mgr.env_azure: could not read value for attribute: attribute=compute/azEnvironment error="Get \"http://169.254.169.254/metadata/instance/compute/azEnvironment?api-version=2019-06-04&format=text\": dial tcp 169.254.169.254:80: connect: connection refused"
    2022-11-12T07:06:44.176Z [DEBUG] client.fingerprint_mgr.env_digitalocean: failed to request metadata: attribute=region error="Get \"http://169.254.169.254/metadata/v1/region\": dial tcp 169.254.169.254:80: connect: connection refused"
    2022-11-12T07:06:44.176Z [DEBUG] client.fingerprint_mgr: detected fingerprints: node_attrs=["arch", "bridge", "cgroup", "consul", "cpu", "host", "network", "nomad", "signal", "storage"]
    2022-11-12T07:06:44.176Z [INFO]  client.plugin: starting plugin manager: plugin-type=csi
    2022-11-12T07:06:44.177Z [INFO]  client.plugin: starting plugin manager: plugin-type=driver
    2022-11-12T07:06:44.177Z [INFO]  client.plugin: starting plugin manager: plugin-type=device
    2022-11-12T07:06:44.177Z [DEBUG] client.device_mgr: exiting since there are no device plugins
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: starting plugin: driver=podman path=/usr/lib/nomad/plugins/nomad-driver-podman args=["/usr/lib/nomad/plugins/nomad-driver-podman"]
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: plugin started: driver=podman path=/usr/lib/nomad/plugins/nomad-driver-podman pid=28
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: waiting for RPC address: driver=podman path=/usr/lib/nomad/plugins/nomad-driver-podman
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: initial driver fingerprint: driver=qemu health=undetected description=""
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: initial driver fingerprint: driver=java health=undetected description=""
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr.docker: could not connect to docker daemon: driver=docker endpoint=unix:///var/run/docker.sock error="Get \"http://unix.sock/version\": dial unix /var/run/docker.sock: connect: no such file or directory"
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: initial driver fingerprint: driver=docker health=undetected description="Failed to connect to docker daemon"
    2022-11-12T07:06:44.177Z [DEBUG] client.driver_mgr: initial driver fingerprint: driver=raw_exec health=healthy description=Healthy
    2022-11-12T07:06:44.178Z [DEBUG] client.driver_mgr: initial driver fingerprint: driver=exec health=healthy description=Healthy
    2022-11-12T07:06:44.178Z [DEBUG] client.plugin: waiting on plugin manager initial fingerprint: plugin-type=driver
    2022-11-12T07:06:44.178Z [DEBUG] client.plugin: waiting on plugin manager initial fingerprint: plugin-type=device
    2022-11-12T07:06:44.178Z [DEBUG] client.plugin: finished plugin manager initial fingerprint: plugin-type=device
    2022-11-12T07:06:44.178Z [DEBUG] client.server_mgr: new server list: new_servers=[0.0.0.0:4647, 10.89.0.91:4647] old_servers=[]
    2022-11-12T07:06:44.180Z [DEBUG] consul.sync: sync complete: registered_services=3 deregistered_services=0 registered_checks=3 deregistered_checks=0
    2022-11-12T07:06:44.190Z [DEBUG] client.driver_mgr.nomad-driver-podman: plugin address: driver=podman address=/tmp/plugin1551708441 network=unix timestamp=2022-11-12T07:06:44.190Z
    2022-11-12T07:06:44.190Z [DEBUG] client.driver_mgr: using plugin: driver=podman version=2
    2022-11-12T07:06:44.191Z [DEBUG] client.driver_mgr.nomad-driver-podman: http baseurl: driver=podman @module=podman url=unix:/run/podman/podman.sock timestamp=2022-11-12T07:06:44.191Z
    2022-11-12T07:06:44.220Z [DEBUG] client.driver_mgr: initial driver fingerprint: driver=podman health=healthy description=ready
    2022-11-12T07:06:44.220Z [DEBUG] client.driver_mgr: detected drivers: drivers="map[healthy:[raw_exec exec podman] undetected:[java docker qemu]]"
    2022-11-12T07:06:44.220Z [DEBUG] client.plugin: finished plugin manager initial fingerprint: plugin-type=driver
    2022-11-12T07:06:44.220Z [INFO]  client: started client: node_id=e8ea9810-5fba-2457-9dc8-75aeaca6b964
    2022-11-12T07:06:44.220Z [DEBUG] http: UI is enabled
    2022-11-12T07:06:44.220Z [DEBUG] http: UI is enabled
    2022-11-12T07:06:44.234Z [DEBUG] consul.sync: sync complete: registered_services=1 deregistered_services=0 registered_checks=1 deregistered_checks=0
    2022-11-12T07:06:45.290Z [WARN]  nomad.raft: heartbeat timeout reached, starting election: last-leader-addr= last-leader-id=
    2022-11-12T07:06:45.293Z [INFO]  nomad.raft: entering candidate state: node="Node at 10.89.0.91:4647 [Candidate]" term=2
    2022-11-12T07:06:45.293Z [DEBUG] nomad.raft: voting for self: term=2 id=2e23ccec-20fc-a3e1-500d-e7824c018a78
    2022-11-12T07:06:45.294Z [DEBUG] nomad.raft: calculated votes needed: needed=1 term=2
    2022-11-12T07:06:45.294Z [DEBUG] nomad.raft: vote granted: from=2e23ccec-20fc-a3e1-500d-e7824c018a78 term=2 tally=1
    2022-11-12T07:06:45.294Z [INFO]  nomad.raft: election won: term=2 tally=1
    2022-11-12T07:06:45.294Z [INFO]  nomad.raft: entering leader state: leader="Node at 10.89.0.91:4647 [Leader]"
    2022-11-12T07:06:45.294Z [INFO]  nomad: cluster leadership acquired
    2022-11-12T07:06:45.298Z [INFO]  nomad.core: established cluster id: cluster_id=94f24a30-3b56-6854-8d7e-c7dbcae7c7bf create_time=1668236805297264840
    2022-11-12T07:06:45.298Z [INFO]  nomad: eval broker status modified: paused=false
    2022-11-12T07:06:45.299Z [INFO]  nomad: blocked evals status modified: paused=false
    2022-11-12T07:06:45.305Z [DEBUG] nomad.autopilot: autopilot is now running
    2022-11-12T07:06:45.305Z [DEBUG] nomad.autopilot: state update routine is now running
    2022-11-12T07:06:45.307Z [INFO]  nomad.keyring: initialized keyring: id=dd74b0a2-8a9c-834f-044c-fd8cfa6ec685
    2022-11-12T07:06:45.308Z [DEBUG] client: updated allocations: index=1 total=0 pulled=0 filtered=0
    2022-11-12T07:06:45.308Z [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=0
    2022-11-12T07:06:45.308Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=0 errors=0
    2022-11-12T07:06:45.383Z [DEBUG] http: request complete: method=PUT path=/v1/jobs duration=1.062168482s
    2022-11-12T07:06:45.389Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=25fa1693-dc6f-9f74-6baa-f39bde7caf80 type=service namespace=default job_id=countdash node_id="" triggered_by=job-register
    2022-11-12T07:06:45.392Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=25fa1693-dc6f-9f74-6baa-f39bde7caf80 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 2) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Created Deployment: "c4d0401c-d812-62d9-ea2d-4b238158a8b0"
  | Desired Changes for "api": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  | Desired Changes for "dashboard": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  
    2022-11-12T07:06:45.393Z [DEBUG] worker: created evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"a83f9bdd-936a-f14a-b714-f6e3a2ca338a\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:06:45.393Z [DEBUG] worker.service_sched: failed to place all allocations, blocked eval created: eval_id=25fa1693-dc6f-9f74-6baa-f39bde7caf80 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef blocked_eval_id=a83f9bdd-936a-f14a-b714-f6e3a2ca338a
    2022-11-12T07:06:45.393Z [DEBUG] worker: submitted plan for evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=25fa1693-dc6f-9f74-6baa-f39bde7caf80
    2022-11-12T07:06:45.393Z [DEBUG] worker.service_sched: setting eval status: eval_id=25fa1693-dc6f-9f74-6baa-f39bde7caf80 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:06:45.394Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"25fa1693-dc6f-9f74-6baa-f39bde7caf80\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:06:45.394Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=25fa1693-dc6f-9f74-6baa-f39bde7caf80 type=service namespace=default job_id=countdash node_id="" triggered_by=job-register
    2022-11-12T07:06:45.572Z [DEBUG] client: updated allocations: index=10 total=0 pulled=0 filtered=0
    2022-11-12T07:06:45.573Z [INFO]  client: node registration complete
    2022-11-12T07:06:45.573Z [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=0
    2022-11-12T07:06:45.573Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=0 errors=0
    2022-11-12T07:06:45.573Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=a83f9bdd-936a-f14a-b714-f6e3a2ca338a type=service namespace=default job_id=countdash node_id="" triggered_by=queued-allocs
    2022-11-12T07:06:45.573Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=a83f9bdd-936a-f14a-b714-f6e3a2ca338a job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 2) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Desired Changes for "api": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  | Desired Changes for "dashboard": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  
    2022-11-12T07:06:45.574Z [DEBUG] worker: submitted plan for evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=a83f9bdd-936a-f14a-b714-f6e3a2ca338a
    2022-11-12T07:06:45.574Z [DEBUG] worker.service_sched: setting eval status: eval_id=a83f9bdd-936a-f14a-b714-f6e3a2ca338a job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:06:45.574Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"a83f9bdd-936a-f14a-b714-f6e3a2ca338a\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:06:45.574Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=a83f9bdd-936a-f14a-b714-f6e3a2ca338a type=service namespace=default job_id=countdash node_id="" triggered_by=queued-allocs
    2022-11-12T07:06:45.575Z [DEBUG] client: state updated: node_status=ready
    2022-11-12T07:06:45.575Z [DEBUG] client.server_mgr: new server list: new_servers=[10.89.0.91:4647] old_servers=[0.0.0.0:4647, 10.89.0.91:4647]
    2022-11-12T07:06:45.575Z [DEBUG] client: updated allocations: index=14 total=2 pulled=2 filtered=0
    2022-11-12T07:06:45.575Z [DEBUG] client: allocation updates: added=2 removed=0 updated=0 ignored=0
    2022-11-12T07:06:45.597Z [DEBUG] client: allocation updates applied: added=2 removed=0 updated=0 ignored=0 errors=0
    2022-11-12T07:06:45.646Z [DEBUG] client.alloc_runner.runner_hook: received result from CNI: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 result="{\"Interfaces\":{\"eth0\":{\"IPConfigs\":[{\"IP\":\"172.26.64.2\",\"Gateway\":\"172.26.64.1\"}],\"Mac\":\"56:2b:a6:25:ac:d3\",\"Sandbox\":\"/var/run/netns/74c46165-f80f-bc79-2852-c2c228c3c4a2\"},\"nomad\":{\"IPConfigs\":null,\"Mac\":\"4e:01:d4:34:4f:38\",\"Sandbox\":\"\"},\"vethe619b90f\":{\"IPConfigs\":null,\"Mac\":\"0a:ca:1b:f2:70:cf\",\"Sandbox\":\"\"}},\"DNS\":[{}],\"Routes\":[{\"dst\":\"0.0.0.0/0\"}]}"
    2022-11-12T07:06:45.647Z [DEBUG] client.alloc_runner.task_runner: lifecycle start condition has been met, proceeding: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api
    2022-11-12T07:06:45.661Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: starting plugin: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api path=/usr/sbin/nomad args=["/usr/sbin/nomad", "logmon"]
    2022-11-12T07:06:45.661Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: plugin started: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api path=/usr/sbin/nomad pid=145
    2022-11-12T07:06:45.661Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: waiting for RPC address: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api path=/usr/sbin/nomad
    2022-11-12T07:06:45.663Z [DEBUG] consul.sync: sync complete: registered_services=1 deregistered_services=0 registered_checks=0 deregistered_checks=0
    2022-11-12T07:06:45.675Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon.nomad: plugin address: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api address=/tmp/plugin1769148564 network=unix @module=logmon timestamp=2022-11-12T07:06:45.675Z
    2022-11-12T07:06:45.675Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: using plugin: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api version=2
    2022-11-12T07:06:45.678Z [INFO]  client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api path=/tmp/NomadClient2533225955/74c46165-f80f-bc79-2852-c2c228c3c4a2/alloc/logs/.connect-proxy-count-api.stdout.fifo @module=logmon timestamp=2022-11-12T07:06:45.678Z
    2022-11-12T07:06:45.678Z [INFO]  client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api @module=logmon path=/tmp/NomadClient2533225955/74c46165-f80f-bc79-2852-c2c228c3c4a2/alloc/logs/.connect-proxy-count-api.stderr.fifo timestamp=2022-11-12T07:06:45.678Z
    2022-11-12T07:06:45.681Z [DEBUG] client.alloc_runner.task_runner.task_hook.envoy_bootstrap: bootstrapping Consul connect-proxy: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api task=connect-proxy-count-api service=count-api
    2022-11-12T07:06:45.681Z [DEBUG] client.alloc_runner.task_runner.task_hook.envoy_bootstrap: check for SI token for task: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api task=connect-proxy-count-api exists=false
    2022-11-12T07:06:45.681Z [INFO]  client.alloc_runner.task_runner.task_hook.envoy_bootstrap: bootstrapping envoy: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api namespace="" proxy_id=_nomad-task-74c46165-f80f-bc79-2852-c2c228c3c4a2-group-api-count-api-9001-sidecar-proxy service=count-api gateway="" bootstrap_file=/tmp/NomadClient2533225955/74c46165-f80f-bc79-2852-c2c228c3c4a2/connect-proxy-count-api/secrets/envoy_bootstrap.json grpc_addr=unix://alloc/tmp/consul_grpc.sock admin_bind=127.0.0.2:19001 ready_bind=127.0.0.1:19101
    2022-11-12T07:06:45.771Z [INFO]  client.driver_mgr.nomad-driver-podman: Pulling image: driver=podman @module=podman image=docker.io/envoyproxy/envoy:v1.21.1 timestamp=2022-11-12T07:06:45.767Z
    2022-11-12T07:06:45.806Z [DEBUG] client.alloc_runner.runner_hook: received result from CNI: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 result="{\"Interfaces\":{\"eth0\":{\"IPConfigs\":[{\"IP\":\"172.26.64.3\",\"Gateway\":\"172.26.64.1\"}],\"Mac\":\"72:8e:fd:5b:57:90\",\"Sandbox\":\"/var/run/netns/9aeb97b4-dff8-6a99-749f-2a48a2b11065\"},\"nomad\":{\"IPConfigs\":null,\"Mac\":\"4e:01:d4:34:4f:38\",\"Sandbox\":\"\"},\"vethfe489e90\":{\"IPConfigs\":null,\"Mac\":\"4e:9f:fb:eb:9d:f9\",\"Sandbox\":\"\"}},\"DNS\":[{}],\"Routes\":[{\"dst\":\"0.0.0.0/0\"}]}"
    2022-11-12T07:06:45.807Z [DEBUG] client.alloc_runner.task_runner: lifecycle start condition has been met, proceeding: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard
    2022-11-12T07:06:45.808Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: starting plugin: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard path=/usr/sbin/nomad args=["/usr/sbin/nomad", "logmon"]
    2022-11-12T07:06:45.808Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: plugin started: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard path=/usr/sbin/nomad pid=242
    2022-11-12T07:06:45.808Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: waiting for RPC address: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard path=/usr/sbin/nomad
    2022-11-12T07:06:45.817Z [DEBUG] consul.sync: sync complete: registered_services=1 deregistered_services=0 registered_checks=0 deregistered_checks=0
    2022-11-12T07:06:45.822Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon.nomad: plugin address: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard @module=logmon address=/tmp/plugin1904368906 network=unix timestamp=2022-11-12T07:06:45.822Z
    2022-11-12T07:06:45.822Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: using plugin: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard version=2
    2022-11-12T07:06:45.824Z [INFO]  client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard @module=logmon path=/tmp/NomadClient2533225955/9aeb97b4-dff8-6a99-749f-2a48a2b11065/alloc/logs/.connect-proxy-count-dashboard.stdout.fifo timestamp=2022-11-12T07:06:45.824Z
    2022-11-12T07:06:45.824Z [INFO]  client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard @module=logmon path=/tmp/NomadClient2533225955/9aeb97b4-dff8-6a99-749f-2a48a2b11065/alloc/logs/.connect-proxy-count-dashboard.stderr.fifo timestamp=2022-11-12T07:06:45.824Z
    2022-11-12T07:06:45.825Z [DEBUG] client.alloc_runner.task_runner.task_hook.envoy_bootstrap: bootstrapping Consul connect-proxy: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard task=connect-proxy-count-dashboard service=count-dashboard
    2022-11-12T07:06:45.825Z [DEBUG] client.alloc_runner.task_runner.task_hook.envoy_bootstrap: check for SI token for task: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard task=connect-proxy-count-dashboard exists=false
    2022-11-12T07:06:45.825Z [INFO]  client.alloc_runner.task_runner.task_hook.envoy_bootstrap: bootstrapping envoy: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard namespace="" proxy_id=_nomad-task-9aeb97b4-dff8-6a99-749f-2a48a2b11065-group-dashboard-count-dashboard-http-sidecar-proxy service=count-dashboard gateway="" bootstrap_file=/tmp/NomadClient2533225955/9aeb97b4-dff8-6a99-749f-2a48a2b11065/connect-proxy-count-dashboard/secrets/envoy_bootstrap.json grpc_addr=unix://alloc/tmp/consul_grpc.sock admin_bind=127.0.0.2:19001 ready_bind=127.0.0.1:19101
    2022-11-12T07:06:45.881Z [DEBUG] client: updated allocations: index=16 total=2 pulled=0 filtered=2
    2022-11-12T07:06:45.881Z [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=2
    2022-11-12T07:06:45.881Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=2 errors=0
    2022-11-12T07:06:45.885Z [INFO]  client.driver_mgr.nomad-driver-podman: Pulling image: driver=podman @module=podman image=docker.io/envoyproxy/envoy:v1.21.1 timestamp=2022-11-12T07:06:45.885Z
    2022-11-12T07:06:46.133Z [DEBUG] client: updated allocations: index=17 total=2 pulled=0 filtered=2
    2022-11-12T07:06:46.133Z [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=2
    2022-11-12T07:06:46.134Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=2 errors=0
    2022-11-12T07:06:46.574Z [DEBUG] client: state changed, updating node and re-registering
    2022-11-12T07:06:46.578Z [INFO]  client: node registration complete
    2022-11-12T07:06:48.842Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="321.661µs"
    2022-11-12T07:06:49.894Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:55658
    2022-11-12T07:06:51.707Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="824.284µs"
    2022-11-12T07:06:58.848Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="397.585µs"
    2022-11-12T07:06:59.892Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:55836
    2022-11-12T07:07:01.712Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="595.7µs"
    2022-11-12T07:07:08.868Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration=10.63547ms
    2022-11-12T07:07:09.892Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:57494
    2022-11-12T07:07:11.720Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration=2.283673ms
    2022-11-12T07:07:18.871Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="562.522µs"
    2022-11-12T07:07:19.894Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:49952
    2022-11-12T07:07:21.725Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration=1.57991ms
    2022-11-12T07:07:28.875Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="749.804µs"
    2022-11-12T07:07:29.895Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:52164
    2022-11-12T07:07:31.731Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="770.297µs"
    2022-11-12T07:07:38.882Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="384.945µs"
    2022-11-12T07:07:39.897Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:52630
    2022-11-12T07:07:41.738Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="436.901µs"
    2022-11-12T07:07:45.770Z [ERROR] client.alloc_runner.task_runner: running driver failed: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api error="rpc error: code = Unknown desc = failed to create image: docker.io/envoyproxy/envoy:v1.21.1: failed to start task, unable to pull image docker.io/envoyproxy/envoy:v1.21.1 : Error reading response: context deadline exceeded (Client.Timeout or context cancellation while reading body)"
    2022-11-12T07:07:45.773Z [INFO]  client.alloc_runner.task_runner: not restarting task: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api reason="Error was unrecoverable"
    2022-11-12T07:07:45.774Z [DEBUG] client.alloc_runner: task failure, destroying all tasks: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 failed_task=connect-proxy-count-api
    2022-11-12T07:07:45.775Z [DEBUG] client.alloc_runner.task_runner: task run loop exiting: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=web
    2022-11-12T07:07:45.789Z [DEBUG] consul.sync: sync complete: registered_services=0 deregistered_services=1 registered_checks=0 deregistered_checks=0
    2022-11-12T07:07:45.886Z [ERROR] client.alloc_runner.task_runner: running driver failed: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard error="rpc error: code = Unknown desc = failed to create image: docker.io/envoyproxy/envoy:v1.21.1: failed to start task, unable to pull image docker.io/envoyproxy/envoy:v1.21.1 : Error reading response: context deadline exceeded (Client.Timeout or context cancellation while reading body)"
    2022-11-12T07:07:45.886Z [INFO]  client.alloc_runner.task_runner: not restarting task: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard reason="Error was unrecoverable"
    2022-11-12T07:07:45.886Z [DEBUG] client.alloc_runner: task failure, destroying all tasks: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 failed_task=connect-proxy-count-dashboard
    2022-11-12T07:07:45.887Z [DEBUG] client.alloc_runner.task_runner: task run loop exiting: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=dashboard
    2022-11-12T07:07:45.891Z [DEBUG] consul.sync: sync complete: registered_services=0 deregistered_services=1 registered_checks=0 deregistered_checks=0
    2022-11-12T07:07:48.888Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="725.609µs"
    2022-11-12T07:07:49.782Z [WARN]  client.alloc_runner.task_runner.task_hook.logmon.nomad: timed out waiting for read-side of process output pipe to close: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api @module=logmon timestamp=2022-11-12T07:07:49.780Z
    2022-11-12T07:07:49.783Z [WARN]  client.alloc_runner.task_runner.task_hook.logmon.nomad: timed out waiting for read-side of process output pipe to close: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api @module=logmon timestamp=2022-11-12T07:07:49.781Z
    2022-11-12T07:07:49.792Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon.stdio: received EOF, stopping recv loop: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api err="rpc error: code = Unavailable desc = error reading from server: EOF"
    2022-11-12T07:07:49.794Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: plugin process exited: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api path=/usr/sbin/nomad pid=145
    2022-11-12T07:07:49.795Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: plugin exited: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api
    2022-11-12T07:07:49.795Z [DEBUG] client.alloc_runner.task_runner: task run loop exiting: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=connect-proxy-count-api
    2022-11-12T07:07:49.796Z [INFO]  client.alloc_runner: waiting for task to exit: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2 task=web
    2022-11-12T07:07:49.797Z [INFO]  client.gc: marking allocation for GC: alloc_id=74c46165-f80f-bc79-2852-c2c228c3c4a2
    2022-11-12T07:07:49.889Z [WARN]  client.alloc_runner.task_runner.task_hook.logmon.nomad: timed out waiting for read-side of process output pipe to close: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard @module=logmon timestamp=2022-11-12T07:07:49.889Z
    2022-11-12T07:07:49.890Z [WARN]  client.alloc_runner.task_runner.task_hook.logmon.nomad: timed out waiting for read-side of process output pipe to close: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard @module=logmon timestamp=2022-11-12T07:07:49.889Z
    2022-11-12T07:07:49.891Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon.stdio: received EOF, stopping recv loop: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard err="rpc error: code = Unavailable desc = error reading from server: EOF"
    2022-11-12T07:07:49.894Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: plugin process exited: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard path=/usr/sbin/nomad pid=242
    2022-11-12T07:07:49.895Z [DEBUG] client.alloc_runner.task_runner.task_hook.logmon: plugin exited: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard
    2022-11-12T07:07:49.895Z [DEBUG] client.alloc_runner.task_runner: task run loop exiting: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=connect-proxy-count-dashboard
    2022-11-12T07:07:49.895Z [INFO]  client.alloc_runner: waiting for task to exit: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065 task=dashboard
    2022-11-12T07:07:49.895Z [INFO]  client.gc: marking allocation for GC: alloc_id=9aeb97b4-dff8-6a99-749f-2a48a2b11065
    2022-11-12T07:07:49.898Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:59156
    2022-11-12T07:07:49.987Z [DEBUG] nomad.client: adding evaluations for rescheduling failed allocations: num_evals=1
    2022-11-12T07:07:49.988Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=1206b367-29cc-d2cf-6773-4ccc2bcf8a0f type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:07:49.988Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=1206b367-29cc-d2cf-6773-4ccc2bcf8a0f job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 0) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Desired Changes for "api": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0)
  | Desired Changes for "dashboard": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0)
  
    2022-11-12T07:07:49.989Z [DEBUG] worker.service_sched: setting eval status: eval_id=1206b367-29cc-d2cf-6773-4ccc2bcf8a0f job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:07:49.989Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"1206b367-29cc-d2cf-6773-4ccc2bcf8a0f\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:07:49.989Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=1206b367-29cc-d2cf-6773-4ccc2bcf8a0f type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:07:49.989Z [DEBUG] client: updated allocations: index=20 total=2 pulled=0 filtered=2
    2022-11-12T07:07:49.990Z [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=2
    2022-11-12T07:07:49.990Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=2 errors=0
    2022-11-12T07:07:50.243Z [DEBUG] nomad.client: adding evaluations for rescheduling failed allocations: num_evals=1
    2022-11-12T07:07:50.247Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=dfeb8df4-6c15-86d9-7aed-91a76eb92029 type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:07:50.248Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=dfeb8df4-6c15-86d9-7aed-91a76eb92029 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 0) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Desired Changes for "api": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0)
  | Desired Changes for "dashboard": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0)
  
    2022-11-12T07:07:50.250Z [DEBUG] worker.service_sched: setting eval status: eval_id=dfeb8df4-6c15-86d9-7aed-91a76eb92029 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:07:50.251Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"dfeb8df4-6c15-86d9-7aed-91a76eb92029\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:07:50.252Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=dfeb8df4-6c15-86d9-7aed-91a76eb92029 type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:07:50.253Z [DEBUG] client: updated allocations: index=22 total=2 pulled=0 filtered=2
    2022-11-12T07:07:50.253Z [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=2
    2022-11-12T07:07:50.254Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=2 errors=0
    2022-11-12T07:07:51.245Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577 type=service namespace=default job_id=countdash node_id="" triggered_by=deployment-watcher
    2022-11-12T07:07:51.248Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 0) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Desired Changes for "api": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0)
  | Desired Changes for "dashboard": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0)
  
    2022-11-12T07:07:51.251Z [DEBUG] worker: created evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"c000a5d8-b5d8-3928-bc1c-3383ec4e2352\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:07:51.252Z [DEBUG] worker.service_sched: found reschedulable allocs, followup eval created: eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef followup_eval_id=c000a5d8-b5d8-3928-bc1c-3383ec4e2352
    2022-11-12T07:07:51.252Z [DEBUG] worker: created evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"051dd03f-3468-4cf0-c811-f7bcbd370c13\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:07:51.252Z [DEBUG] worker.service_sched: found reschedulable allocs, followup eval created: eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef followup_eval_id=051dd03f-3468-4cf0-c811-f7bcbd370c13
    2022-11-12T07:07:51.257Z [DEBUG] worker: submitted plan for evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577
    2022-11-12T07:07:51.257Z [DEBUG] worker.service_sched: setting eval status: eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:07:51.260Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"e9c222ba-11aa-ef76-8a8a-bcaabeb39577\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:07:51.261Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=e9c222ba-11aa-ef76-8a8a-bcaabeb39577 type=service namespace=default job_id=countdash node_id="" triggered_by=deployment-watcher
    2022-11-12T07:07:51.269Z [DEBUG] client: updated allocations: index=24 total=2 pulled=2 filtered=0
    2022-11-12T07:07:51.270Z [DEBUG] client: allocation updates: added=0 removed=0 updated=2 ignored=0
    2022-11-12T07:07:51.271Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=2 ignored=0 errors=0
    2022-11-12T07:07:51.277Z [DEBUG] client: updated allocations: index=27 total=2 pulled=2 filtered=0
    2022-11-12T07:07:51.278Z [DEBUG] client: allocation updates: added=0 removed=0 updated=2 ignored=0
    2022-11-12T07:07:51.278Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=2 ignored=0 errors=0
    2022-11-12T07:07:51.745Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="564.689µs"
    2022-11-12T07:07:58.896Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="736.94µs"
    2022-11-12T07:07:59.903Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:43702
    2022-11-12T07:08:01.753Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="573.536µs"
    2022-11-12T07:08:08.903Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="762.308µs"
    2022-11-12T07:08:09.904Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:53272
    2022-11-12T07:08:11.764Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration=3.915493ms
    2022-11-12T07:08:14.256Z [INFO]  client.driver_mgr: driver health state has changed: driver=podman previous=healthy current=undetected description=disabled
    2022-11-12T07:08:14.259Z [ERROR] client.driver_mgr.nomad-driver-podman: Could not get podman info: driver=podman error="Get \"http://u/v1.0.0/libpod/info\": dial unix /run/podman/podman.sock: connect: no such file or directory" @module=podman timestamp=2022-11-12T07:08:14.256Z
    2022-11-12T07:08:15.260Z [DEBUG] client: state changed, updating node and re-registering
    2022-11-12T07:08:15.266Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=2b27f247-b500-ada8-d03c-4256cc88d4ef type=service namespace=default job_id=countdash node_id=e8ea9810-5fba-2457-9dc8-75aeaca6b964 triggered_by=node-update
    2022-11-12T07:08:15.267Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=2b27f247-b500-ada8-d03c-4256cc88d4ef job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 2) (destructive 0) (inplace 0) (stop 2) (disconnect 0) (reconnect 0)
  | Desired Changes for "api": (place 1) (inplace 0) (destructive 0) (stop 1) (migrate 0) (ignore 0) (canary 0)
  | Desired Changes for "dashboard": (place 1) (inplace 0) (destructive 0) (stop 1) (migrate 0) (ignore 0) (canary 0)
  
    2022-11-12T07:08:15.271Z [DEBUG] worker: created evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"251792ed-4800-ddb3-36b7-f4b60dc6c9c4\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:08:15.271Z [DEBUG] worker.service_sched: failed to place all allocations, blocked eval created: eval_id=2b27f247-b500-ada8-d03c-4256cc88d4ef job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef blocked_eval_id=251792ed-4800-ddb3-36b7-f4b60dc6c9c4
    2022-11-12T07:08:15.277Z [DEBUG] worker: submitted plan for evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=2b27f247-b500-ada8-d03c-4256cc88d4ef
    2022-11-12T07:08:15.277Z [DEBUG] worker.service_sched: setting eval status: eval_id=2b27f247-b500-ada8-d03c-4256cc88d4ef job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:08:15.278Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"2b27f247-b500-ada8-d03c-4256cc88d4ef\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:08:15.278Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=2b27f247-b500-ada8-d03c-4256cc88d4ef type=service namespace=default job_id=countdash node_id=e8ea9810-5fba-2457-9dc8-75aeaca6b964 triggered_by=node-update
    2022-11-12T07:08:15.279Z [INFO]  client: node registration complete
    2022-11-12T07:08:15.279Z [DEBUG] client: evaluations triggered by node registration: num_evals=1
    2022-11-12T07:08:15.281Z [DEBUG] client: updated allocations: index=33 total=2 pulled=2 filtered=0
    2022-11-12T07:08:15.282Z [DEBUG] client: allocation updates: added=0 removed=0 updated=2 ignored=0
    2022-11-12T07:08:15.283Z [DEBUG] client: allocation updates applied: added=0 removed=0 updated=2 ignored=0 errors=0
    2022-11-12T07:08:15.775Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=c000a5d8-b5d8-3928-bc1c-3383ec4e2352 type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:08:15.778Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=c000a5d8-b5d8-3928-bc1c-3383ec4e2352 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 2) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Desired Changes for "dashboard": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  | Desired Changes for "api": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  
    2022-11-12T07:08:15.780Z [DEBUG] worker: created evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"cf8f8bb1-0e02-17ab-e80c-14d1ec5d1a57\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:08:15.780Z [DEBUG] worker.service_sched: failed to place all allocations, blocked eval created: eval_id=c000a5d8-b5d8-3928-bc1c-3383ec4e2352 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef blocked_eval_id=cf8f8bb1-0e02-17ab-e80c-14d1ec5d1a57
    2022-11-12T07:08:15.781Z [DEBUG] worker.service_sched: setting eval status: eval_id=c000a5d8-b5d8-3928-bc1c-3383ec4e2352 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:08:15.781Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"c000a5d8-b5d8-3928-bc1c-3383ec4e2352\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:08:15.782Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=c000a5d8-b5d8-3928-bc1c-3383ec4e2352 type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:08:15.888Z [DEBUG] worker: dequeued evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=051dd03f-3468-4cf0-c811-f7bcbd370c13 type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:08:15.890Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=051dd03f-3468-4cf0-c811-f7bcbd370c13 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef
  results=
  | Total changes: (place 2) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
  | Desired Changes for "api": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  | Desired Changes for "dashboard": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
  
    2022-11-12T07:08:15.894Z [DEBUG] worker: created evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"3983309f-d3d4-0083-fa50-e9b38a182921\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:08:15.894Z [DEBUG] worker.service_sched: failed to place all allocations, blocked eval created: eval_id=051dd03f-3468-4cf0-c811-f7bcbd370c13 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef blocked_eval_id=3983309f-d3d4-0083-fa50-e9b38a182921
    2022-11-12T07:08:15.895Z [DEBUG] worker.service_sched: setting eval status: eval_id=051dd03f-3468-4cf0-c811-f7bcbd370c13 job_id=countdash namespace=default worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef status=complete
    2022-11-12T07:08:15.896Z [DEBUG] worker: updated evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval="<Eval \"051dd03f-3468-4cf0-c811-f7bcbd370c13\" JobID: \"countdash\" Namespace: \"default\">"
    2022-11-12T07:08:15.896Z [DEBUG] worker: ack evaluation: worker_id=0b1a206e-3ebd-fc3f-b9e1-372a670f2fef eval_id=051dd03f-3468-4cf0-c811-f7bcbd370c13 type=service namespace=default job_id=countdash node_id="" triggered_by=alloc-failure
    2022-11-12T07:08:18.910Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=client duration="440.331µs"
    2022-11-12T07:08:19.906Z [DEBUG] nomad: memberlist: Stream connection from=10.89.0.90:32812
    2022-11-12T07:08:21.772Z [DEBUG] http: request complete: method=GET path=/v1/agent/health?type=server duration="387.609µs"