Output variables in state are not removed by refresh

With any configuration management tool (such as Terraform) users should not modify the managed resources via any other mechanism other than updating the code/tfvars. If changes are made outside of Terraform they will be reverted the next time Terraform apply is run, which is exactly what is supposed to happen.

While it can be necessary to make emergency changes they should always be reflected in the Terraform code ASAP to prevent them being undone.

The possible changes you mention sound like “normal changes” rather than things needed to fix an incident, so should always be being done via Terraform code changes and then an apply - Terraform code is the source of truth and should be maintained accordingly.