Hello everyone,
I see there’s a lack of good options to manage and deploy Spark Streaming applications: all of the available ones for Spark jobs are very much targeted towards batch jobs.
So I wonder if making a plugin to Waypoint would be fitting. It’s not a web application, so it does not have load balancer URIs or something like that. My idea would be to make it a wrapper over Livy or the spark-submit command.