Nats cluster + consul connect

Nomad
, ,
1

Hi, I’m trying to run a nats clustered job which will be accessible by other services using consul connect

here is my nomad job file:

job "nats" {
  datacenters = ["dc1"]

  type = "service"

  constraint {
    attribute = "${node.class}"
    value     = "nats"
  }

  group "cluster" {

    count = 3


    update {
      max_parallel = 1
    }

    migrate {
      max_parallel     = 1
      health_check     = "checks"
      min_healthy_time = "5s"
      healthy_deadline = "30s"
    }

    ephemeral_disk {
      size = 5000
    }

    network {
      mode = "bridge"
      port "nats" {
        to = 4222
      }
      port "cluster" {
        to = 6222
      }
    }

    service {
      name     = "nats"
      provider = "consul"
      port     = "nats"
    }

    service {
      name         = "nats-cluster"
      port         = "cluster"
      address_mode = "alloc"

      connect {
        sidecar_service {}
      }
    }


    task "nats" {
      driver = "docker"

      config {
        image = "nats:2.10.4-alpine"
        args  = ["-c", "/local/cluster.conf"]
        ports = ["nats", "cluster"]
      }

      template {
        destination = "local/cluster.conf"
        change_mode = "noop"
        data        = <<EOH
server_name: {{ env "NOMAD_TASK_NAME" }}
listen: 4222
monitor_port: 8222
max_payload: 31457280

accounts: { 
  SYS: { 
    users: [ 
      { user: "sys", pass: "secret" }
    ]
  }
  ADMIN {
    users: [
      { user: admin, password: secret }
    ]
    jetstream: enabled
  }
}
system_account: SYS

jetstream {
  store_dir: /data
}

cluster {
  name: stream
  listen: 0.0.0.0:6222
  routes: [
    {{- range service "nats-cluster" }}
    # here goes the nats cluster nodes without node itself address (e.g. for node-1)
    # nats://node-2:6222
    # nats://node-3:6222
    {{- end}}
  ]
}
        EOH
      }
    }
  }
}

the template on each node differs (the node itself info should not be in routes)
how could I do that?

how could I have cluster port (6222) in a service mesh by consul so nodes talk to each other based on consul.

2

Hi @r6m,

Could you explain the problem you’re currently experiencing please?

Thanks,
jrasell and the Nomad team

3

@jrasell sorry, updated the question

4

@jrasell if I use host volumes for nats jetstream /data path, I should create 3 separate tasks in a group, of 3 groups with 1 task for nats?

5

@r6m did you ever get a working configuration? If so, could you please share it?
Thanks