How to update a Security Group deployed by Terraform and with a lambda function?

We have to update dynamically a Security Group thru a Lambda Function ; the IP to be allowed are extracted from the CloudWatch events.

This Security Group and the owner are deployed by Terraform into a VPC, within a lot of components and with a lot of updates.

These components, included the Security Group, are under control of Terraform, meaning it could be redeployed if Terraform identifies some differences from the source. Then, the differences dynamically made by the Lambda are lost.

How can we control / avoid this behavior, please ?