Hi,
I’ve migrated my containers from docker-compose management to nomad management, each container has its own job now.
I am suffering issues with memory management, I have some kotlin-jvm containers that throws jvm exception on startup regarding ridiculous amount of memory that they cannot allocate (for example 1.5 gb of memory), it doesn’t make sense because when I am running those containers via “docker run” command they occupy around 200-300 MB of ram,what could be the issue here?
Hi @dekel.honen. Thanks for using Nomad!
Are you using the Java driver? It has the ability to configure jvm_options to be passed when invoking Java. I wonder if setting Xms and Xmx would solve your problem. Xms “sets the initial and minimum Java heap size” and Xmx “sets the maximum Java heap size”. Seems like a good thing to try maybe?
Hi,
I am using the docker driver since I am working with kotlin-jvm containers, any other solution?
Ok. In that case, you should be able to configure this in your Dockerfile before you even get to Nomad.
Here’s an article on how to configure JVM memory settings using the JAVA_OPTS environment variable.
If for some reason, you don’t have the ability to modify the docker image, you could try setting an environment variable, or passing an arg to the command. It’s a little bit contextual based on what you are doing in your Dockerfile versus your jobspec, but the basic idea is you need to find a way to pass this setting to the JVM itself. If you are still stuck, you can paste your Dockerfile and your jobspec with secrets removed and I can try to help.
Hi,
I’ve tried your solution and wrote:
ENV JAVA_OPTS="-Xms128M -Xmx128M"
in the docker file, it runs on 100MB max in docker stats command but when I am trying to run it via nomad job(even with 300MB of memory) it crashes with the following reason:
There is insufficient memory for Java Runtime Environment to continue.
Native memory allocation failed to map 268435456 bytes for Failed commit area…
So I guess the solution is not working sadly, any other suggestion?