@maxb, the fact that data resources were, in earlier releases, not read when refresh was disabled was perceived (correctly, in my estimation) as a bug; people trying to use data blocks as intended and documented found that they then couldn’t use -refresh=false because it would prevent Terraform from noticing that an external dependency had changed, and yet noticing such changes is exactly what data blocks are for.
I understand that you wanted -refresh=false to mean “do fewer API requests”, but that is not what it is intended to mean. It instead means “I promise I’ve not changed any managed resources outside of Terraform since the last run; don’t waste time checking that”. In that situation the refreshing requests are unnecessary, but the data source reads are still necessary because data resources are expected to change outside of Terraform.
The fact that the protocol is designed around this difference is a reason why it is technically complex to change this now, but even if it were easy to change I still don’t think it would be good to change this because it would prevent folks from using -refresh=false when they are using data sources for their intended purpose.
It’s possible in principle that there could be some other option that means “use stale data source results too” but I personally don’t think I’ve seen this feedback enough to justify the additional complexity of supporting such an option. I do acknowledge your feedback here, but unfortunately it seems like your position is not shared by most folks who learned Terraform after we fixed this bug, suggesting that the current behavior is the more broadly-useful behavior.
Therefore I don’t expect this position to change, though I do hope that we can one day reach the point of supporting a batch request optimization when a provider is able to satisfy many data source reads and/or many managed resource refreshes in a single request to the remote API. That would shrink the number of requests without compromising the correct behavior.