I believe there was an undocumented, breaking change in the AWS V4 Provider which results in a failure when it looks for a profile that doesn’t exist.
The V3 provider family didn’t fail: it would fall back in the same manner as AWS-CLI, and use the default (which works fine with configure-aws-credentials and other aws tools).
The easiest solution is to create a variable (bool, default false) which indicates whether the operation is being run in a pipeline. In your pipeline, set this variable to true. Then in your terraform provider config, dynamically set the profile using the pipeline variable as a flag.
provider = var.pipeline ? “” : $YourLocalProfileNameHere
This will allow you to continue to work locally using profile names, while the empty string will cause TF to grab the default profile (generated by configure-aws-credentials or some similar library).