Creating a security group ingress rules from an api with Ips lists

Terraform
1

Github publishes a list of IPs in this endpoint https://api.github.com/meta.

I want to create a security group that only allows the ips specified in this endpoint. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group

Is it reasonable to simply use a http provider and jsonsencode to dynamically create these ingress rules in the cidr range?

If not, what is a better approach i.e. pull the ips into a tfvars or hcl file periodically?

2

Hi @gridcellcoder

Welcome to Terraform forums.

If you know how to build the list of IPs, you can use an external data source.

See an example for getting you public Internet IP and using it in a firewall rule in AWS:

3

Hi @javierruizjimenez

Thanks for the reply, would you consider getting this data programatically bad practice or against the declarative nature of Terraform?

Should’nt we be getting these IPs manually/programatically outside of terraform and using that as variables?

Many thanks for your input.

4

Hi @gridcellcoder

Thanks for the reply, would you consider getting this data programmatically bad practice or against the declarative nature of Terraform?

Yes, I try to avoid getting external data. It has serious security implications, and it slows the process. The example that I shared about getting the public IP is for testing, a proper infrastructure will have a fixed IP range and will probably use a VPN to connect with AWS.

If GitHub doesn’t change the IPs often, I would hard code the values as Terraform variables, and monitor api.github.com/meta for changes, trigger a human alert (if it happens rarely it doesn’t need to be fully automated).

If Github changes IPs often, Terraform will not know by itself, you will need to apply the changes manually (terraform apply), so I will try a different solution, maybe:

  • Monitoring the API for changes and then running a hook (Adapt this AWS lambda) that modifies the rule (Terraform will probably see a change later on but the end result will be the same).

  • Don’t use AWS Security rules for that, maybe use a NGINX firewall module or other software firewall that can use FQDN as sources, make sure you trust your DNS server!. - Haven’t tried, don’t know if possible -

Lets ask the expert, @apparentlymart what do you think?

5

Thanks @javierruizjimenez

I appreciate the advice on not getting external data, though it is a bit confusing that this functionality exists and that it is not marked for testing use only.

We are happy to automate the terraform apply in say github actions /cron job though yes would appreciate thoughts from an expert @apparentlymart do you have any thoughts?

6

I think Terraform will have some supported, integrated and easy to use way to load that data as HashiCorp is now publishing a list of IPs used by its servers.

See:

https://www.terraform.io/docs/cloud/api/ip-ranges.html

And today’s announcement for Terraform Cloud Business with fixed IPs and agents can certainly make use of that info.

@apparentlymart sure has a solution.