Executing bat file from nomad (exit code 0)

Hi,

Iv been having issues with a particular job which perform a raw_exec. For some reason with this particular job/ executable I always get exit code 0 immediately and the job seemingly completes.

The problem is that the executable is in fact meant to run continuously and does not (exit code 0) immediately when run manually.

  task "gitea" {
        driver = "raw_exec"

        config {
          command = "C:/Users/myuser/Documents/gitea/gitea"
        }

//remove unneccessary parts
}

Again this seems to work as expected in that it executed but immediately exits.

Similarly if i run a .bat file instead the behaviour seems to be unusual:

cd C:\Users\myuser\Documents\gitea && cmd /k gitea
@pause

when i execute this .bat file from nomad it actually performs each line and does infact not only not pause but seems to auto-complete immediately still:

The job logs show the following within a second of executing:

C:\Users\myuser\Documents\nomad_0.10.2_windows_amd64\data\alloc\7187bce4-b747-dc16-26b3-7ef4d8d85f67\startgitea>cd C:\Users\myuser\Documents\gitea   && cmd /k gitea 

C:\Users\myuser\Documents\gitea>Press any key to continue . . . 

the overview

Feb 04, '20 19:24:37 +0000	Terminated	Exit Code: 0
Feb 04, '20 19:24:37 +0000	Started	Task started by client

So does anyone have any idea why this particular job just immediately might return exit code 0 rather than hanging on the execution of the application as im trying to keep gitea alive as a permanent service/job via nomad.

I have also observed similar weird behavior on Windows’ poweshell executions. What I discovered completely by accident is that if the command finishes too quickly, Nomad “loses” track of it.
What I did is to add two task to the group, the 2nd one just some time consuming command for about 10 seconds (say the timeout cmd or something). Then things worked.

I have no rational explanation for why it happened this way, and have not followed up since, I was only doing a demo.

Hi @DHT! When you’re running raw_exec jobs, Nomad sets up an executor that monitors the process. But from the .bat script you’re running there, that’s not pausing execution because it’s expecting an interactive prompt (which isn’t supported for Nomad jobs).

So it looks to me that the problem is the invocation of gitea itself. Maybe you could provide some more of the “unneccesary parts” of the job file? For example, it looks like from https://docs.gitea.io/en-us/windows-service/ that you should be passing arguments to gitea.exe like web --config C:/path/to/app.ini?

Hi,

Thanks for taking the time to review my question.

 task "gitea" {
        driver = "raw_exec"

        config {
          command = "C:/Users/myuser/Documents/gitea/gitea.exe"
         args    = ["web","--config","C:/Users/myuser/Documents/gitea/custom/conf/app.ini"]
       
  }

This is the task im trying to execute in full, im still unable to resolve that particular issue. Further on windows it seems my nomad client is unable to resolve the docker driver also despite docker desktop being installed so im completely stuck and unable to use nomad for the task at hand.

The error i recieve when using docker is “Docker is configured with Linux containers; only Windows containers are supported” which is…not what i intended as the point of using docker on windows would be to run gitea…in a linux container…so thats a…less than ideal error to put it politely.

In regards to the raw_exec i tried @shantanugadgil tip today however i had the same issue, the task executes and completes with exit code 0 within a matter of 1 second and then restarts. Further it doesent cause gitea to even write to the log file it usually generates when gitea.exe is run manually, meaning it appears to exit before any task has even been performed by the exe. However if i provide an invalid path it throws a warning, so seems like it is actually performing the task but just exiting before it has time to even execute at all.

here is a complete example with the same behaviour:

job "docs" {
  datacenters = ["dc1"]

  group "example" {
    task "server" {
      driver = "raw_exec"

      config {
        command = "gitea"
        args = [
          "--config", "C:/Users/myuser/Documents/gitea/custom/conf/app.ini",
        ]
      }

      resources {
        network {
          mbits = 1
          port "http" {
            static = "3000"
          }
        }
      }
    }
  }
}

Further on windows it seems my nomad client is unable to resolve the docker driver also despite docker desktop being installed so im completely stuck and unable to use nomad for the task at hand.

Just a heads up, Docker for Windows (the desktop application) isn’t really all that well supported for us. Keep an eye on https://github.com/hashicorp/nomad/issues/2633 for more on that.

A couple more things to look at here:

  • Are you looking at the alloc logs via the Nomad CLI? If so, you may want to try the -stderr flag to see if it’s writing to stderr at all. docker logs co-mingles these.
  • You do need the full path to gitea in your configuration, as you had in the earlier example:
     config {
        command = "gitea"
        args = [
          "--config", "C:/Users/myuser/Documents/gitea/custom/conf/app.ini",
        ]
      }
  • Do you have full debug logs from Nomad itself? Maybe there’s some clue there.

Hi Tgross,

Thanks again for the response and the heads up about docker support.

I used the gitea command without the path in the second example as i added it to the env path to try to reduce the potential number of things that could go wrong. But the result is the same.

For the logs of the raw_exec, as im leaving docker for now as its not my preferred choice for this task anyway. Im viewing them via the nomad web interface but i think because it exit codes 0 there is no error to actually view, it seems to complete without any problem and so the only thing i can actually see is the events viewer:

|Feb 07, '20 22:31:42 +0000|Terminated|Exit Code: 0

|Feb 07, '20 22:31:42 +0000|Started|Task started by client

based on this it does seem to be particularly a problem with keeping gitea alive, it seems clear it can find the exe as if i provide an invalid path/file it causes a failure due to the file not begin found. But after contacting gitea support via discord and stepping through the process it seems that the task im executing should be all that is required and works when running directly on the client just not via nomad which is very confusing.

Also the logs on the client from nomad / allocation folder seem to be completely empty -

Update:

I solved the issue by creating a gitea service and instead of calling it directly, use nomad to trigger the service to start. Its a less than ideal solution but iv been making 0 progress in debugging this so just leaving this here in case someone else encounters this problem.

config {
        command = "net"
        args = [
          "start", "mygitea-service",
        ]
      }