I am running a small device farm which consists of a number of custom IOT devices which have RS232 consoles. Each device is attached to a raspberry pi which is running as a Nomad client. There are a number of different classes of device, say ClassA and ClassB. I’m using Nomad to schedule tests to the RPIs. The tests connect to the devices via the serial port and therefore it is not possible to have two tests running at the same time on any RPI.
Right now I have client metadata which specifies which class of device is attached to each RPI and I can use constraints on the job to ensure that the jobs are scheduled to RPIs with the correct devices attached. The only way I’ve found to ensure that only one job can be scheduled to each RPI at a time is to inflate the CPU/RAM requirements so that only one fits. This has the downside that nothing else can run on those devices whilst the tests are in progress - if we scheduled a system job, for example, then all our tests would fail to deploy.
I’ve tried to write a sort of generic device driver which will allow a task to consume a device, and then release it when the job completes, but this is not possible as (quoting the device driver documentation) “device plugins do not currently have an interface for persisting state to the Nomad client. Instead, the device plugin API emphasizes fingerprinting devices and reporting their status. After helping to provision a task with a scheduled device, a device plugin does not have any responsibility (or ability) to monitor the task.”. I could include a “release device” task at the end of each job, but that feels fragile.
Is there a way I can ensure that only one such job is scheduled on a client at a time, without arbitrarily restricting the scheduling of other jobs?
Thanks very much in advance,
Richard.