How do you setup Nomad in local development

Hi,

I am using a mac and would like to use nomad for dev on the mac before using it in prod.

Goal: run our app in dev 1 docker node (mount to src dir) and 1 db mysql

I am trying to setup a host_volume in the client section of the config file
use this host_volume to mount the docker db data like in the tutorial

when i start nomad, it cant find the existing directory

is there any tutorial to run nomad in dev locally … is it possible??

What’s your problem? It’s easier if you provide someway to reproduce your problem.

It’s possible to run the nomad agent locally, of course. It will be also possible to work with host volumes.

Could you share the tutorial you`re following?

I have following this tutorial Local Nomad Cluster Using Vagrant - #9 by DerekStrickland from DerekStrickland comment

  • used the Vagrantfile from Nomad repo root
  • add to nomad-client config vmCfg.vm.synced_folder '~/opt/volumes', '/opt/volumes' to mount local shares

On the nomad-client01 client.hcl

log_level    = "DEBUG"
datacenter   = "dc1"
data_dir     = "/etc/nomad.d/data"
enable_debug = true

client {
  enabled = true
  server_join {
    retry_join = ["192.168.56.11"]
  }

  host_volume "db-entrypoint" {
    path      = "/opt/volumes/db-entrypoint"
    read_only = false
  }

  host_volume "db-conf" {
    path      = "/opt/volumes/db-conf"
    read_only = false
  }

  host_volume "db-data" {
    path      = "/opt/volumes/db-data"
    read_only = false
  }

  host_volume "db-log" {
    path      = "/opt/volumes/db-log"
    read_only = false
  }

  host_volume "redis-data" {
    path      = "/opt/volumes/redis-data"
    read_only = false
  }
}

plugin "docker" {
  config {
    allow_privileged = true
  }
}

I have been able to start a redis job (redis.nomad)

job "redis" {
  type = "service"

  group "redis" {
    count = 1

    volume "redis-data" {
      type      = "host"
      read_only = false
      source    = "redis-data"
    }

    task "redis-server" {
      driver = "docker"

      volume_mount {
        volume      = "redis-data"
        destination = "/data"
        read_only   = false
      }

      config {
        image   = "redis:latest"
        ports   = ["db"]
        command = "redis-server"
        args    = ["/usr/local/etc/redis/redis.conf"]
        mount {
          type   = "bind"
          source = "local"
          target = "/usr/local/etc/redis"
        }
      }

      resources {
        cpu    = 500
        memory = 1024
      }

      service {
        name = "sakura-redis-server"
        port = "db"

        provider = "nomad"

        check {
          type     = "tcp"
          interval = "10s"
          timeout  = "2s"
        }
      }

      template {
        data        = <<EOH
maxmemory-policy noeviction
requirepass dev
EOH
        destination = "local/redis.conf"
        # source        = "local/redis.conf.tpl"
        # destination   = "local/redis.conf"
        # change_mode   = "signal"
        # change_signal = "SIGINT"
      }

    }

    network {
      port "db" {
        # to     = 3306
        static = 6379
      }
    }
  }
}

mysql or mariadb job is not working

job "mysql" {
  type = "service"

  group "mariadb-server" {
    count = 1

    volume "db-conf" {
      type      = "host"
      read_only = false
      source    = "db-conf"
    }

    volume "db-data" {
      type      = "host"
      read_only = false
      source    = "db-data"
    }

    volume "db-log" {
      type      = "host"
      read_only = false
      source    = "db-log"
    }

    volume "db-entrypoint" {
      type      = "host"
      read_only = false
      source    = "db-entrypoint"
    }

    restart {
      attempts = 10
      interval = "5m"
      delay    = "25s"
      mode     = "delay"
    }

    # task "pre-mariadb-server" {
    #   driver = "docker"
    #   # user   = "1000:1000"

    #   volume_mount {
    #     volume      = "db-data"
    #     destination = "/var/lib/mysql"
    #     read_only   = false
    #   }

    #   volume_mount {
    #     volume      = "db-log"
    #     destination = "/var/log/mysql"
    #     read_only   = false
    #   }

    #   volume_mount {
    #     volume      = "db-conf"
    #     destination = "/etc/mysql/conf.d"
    #     read_only   = false
    #   }

    #   volume_mount {
    #     volume      = "db-entrypoint"
    #     destination = "/docker-entrypoint-initdb.d"
    #     read_only   = false
    #   }

    #   config {
    #     # image = "mysql"
    #     image = "mariadb:10.3"
    #     # fix volume permissions to use mysql uid/gid 999
    #     #  /var/log/mysql /etc/mysql/conf.d /docker-entrypoint-initdb.d
    #     command = "sh"
    #     args    = ["-c", "chown -R 999:999 /var/lib/mysql /var/log/mysql"]
    #   }

    #   # https://github.com/hashicorp/nomad/issues/8892 - need for fixing permissions
    #   lifecycle {
    #     hook    = "prestart"
    #     sidecar = false
    #   }

    #   resources {
    #     cpu    = 500
    #     memory = 2048
    #   }
    # }

    task "mariadb-server" {
      driver = "docker"
      user   = "mysql"

      volume_mount {
        volume      = "db-data"
        destination = "/var/lib/mysql"
        read_only   = false
      }

      volume_mount {
        volume      = "db-log"
        destination = "/var/log/mysql"
        read_only   = false
      }

      volume_mount {
        volume      = "db-conf"
        destination = "/etc/mysql/conf.d"
        read_only   = false
      }

      # volume_mount {
      #   volume      = "db-entrypoint"
      #   destination = "/docker-entrypoint-initdb.d"
      #   read_only   = false
      # }

      env {
        MYSQL_ROOT_PASSWORD = "passw0rd"
        MYSQL_DATABASE      = "mydb"
        MYSQL_USER          = "myuser"
        MYSQL_PASSWORD      = "passw0rd"
      }

      config {
        # image = "mysql:latest"
        image = "mariadb:10.3"
        ports = ["db"]
      }

      resources {
        cpu    = 500
        memory = 4096
      }

      service {
        name = "sakura-mariadb-server"
        port = "db"

        provider = "nomad"

        check {
          type     = "tcp"
          interval = "10s"
          timeout  = "2s"
        }
      }
    }

    network {
      port "db" {
        # to     = 3306
        static = 3306
      }
    }
  }
}

In nomad-client01 /opt/volumes is owned by user:vagrant(uid 1000), group:vagrant(gid 1000).

Mysql or mariadb docker images are using user/group mysql(999:999).

So far I did not find a way to solve this permission problem.

Do you have any tutorials?

In a dev team, how do you setup a local dev environment (ie: redis + mysql + node) in a Nomad dev cluster (running on the laptop or in the cloud)… with hotreload while edit your code

Currently using docker-compose… but the team would like to move to Nomad in dev before going to production