Hello everyone, I hope to be in the right place (if not, I apologize in advance).
I am struggling with a configuration problem … I want to deploy a number of web services and applications on an AWK EKS cluster and would like to do so using Terraform.
My project is simple: I expose web pages (both static and dynamic), REST API and also WebSocket connectivity, from different third-level domains, with url rewrite (in some cases) and in https.
It all works on my development environment (a microk8s instance running on ubuntu) using nginx as ingress controller for k8s, but when I started thinking about which AWS services I should be using I got stuck …
A complete scenario with EKS require a VPC with at least 4 subnets (2 publics and 2 privates), one NAT gateway and EKS cluster… but my big doubt is related to the LoadBalancers…
Which one to use? But most important… how?
I started thinking to use ALB, but it does not implements url rewrite and it requires to change all the .yml ingress files…
Then i saw that maybe NLB can be a solution (put in front of EKS to “proxy” the traffict to nginx that will dispatch it), but i want to use my own DNS (not AWS) and my own SSL certificates… and i wasn’t able to find any real and practical example of terraform scripts that implement this scenario… (the scripts that I found around the net do things a little different from each other and in any case not what I need …)
Does anyone know about terraform scripts that implement this scenario? it seams to me that it is really basic… i was astonished to not found any literature for it…
Thanks in advance to all of you.
You can continue to use the nginx ingress if you want. For that you’d stick an NLB in front pointing at all of the worker nodes (to send the 443/80 traffic to nginx).
Equally you could use the AWS Loadbalancer Controller which creates ALBs. You could use that to point to your nginx ingress or directly to your services. Configuration is via ingress annotations, including setting the certificate details (stored in ACM - you can use Terraform to upload your own certificate if you don’t want to use the automatic Amazon ones).
You could use External DNS to control the DNS records (depending on the provider/system you are using) or Terraform.
One of the great things about both Kubernetes & AWS is there is so much choice and different ways to do things. One of the worst things is also there being so much choice!
As a result you might not find a guide/blog post/stackoverflow describing things the way you want to do it (or working to your requirements/limitations).
For us we provision the EKS cluster via Terraform and also use it to deploy Helm charts for both the applications and support systems (for us that includes External DNS, Cluster Autoscaler, AWS Loadbalancer Controller). We then let Kubernetes manage the ALBs (we don’t need any of the more advanced functionality that nginx might give us) and the DNS (we use Route53 which is supported by External DNS). But that is only one way of many you could set things up…
Hi Stuart and thanks for your reply.
Thanks for also for the confirmations you give me about the tools that can be used (and that it is possible to use them in the way i intended); i agree with you specially when you write:
"
One of the great things about both Kubernetes & AWS is there is so much choice and different ways to do things. One of the worst things is also there being so much choice!
"
What i hoped was to be able to find a recipe about how to do this configuration (that, given the requests, i found it very basic); Terraform is a great tool and if it were supported by something like a “cookbook” it would be perfect.
I will work hard on it, thanks again.