Access Denied When Using VPC Enpoint with S3 Access Points

I am experimenting with S3 access points for VPC’s (Creating access points restricted to a virtual private cloud - Amazon Simple Storage Service), but I am unable to get them to work properly.

I create a S3 bucket, an S3 access point, and the following bucket policy:

data "aws_iam_policy_document" "example" {
   statement {
        actions   = ["s3:GetObject"]
        resources = ["${aws_s3_bucket.example.arn}/*"]
        principals {
            type = "*"
            identifiers = ["*"]
        }
        effect = "Allow"
        condition {
            test = "IpAddress"
            variable = "aws:SourceIp"
            values = [ 
                "<Public IP of EC2 instance>",
            ]
        }
    }
}

resource "aws_s3_bucket" "example" {
    bucket = "example"
}

resource "aws_s3_bucket_policy" "example" {
    bucket = aws_s3_bucket.example.id
    policy = data.aws_iam_policy_document.example.json
}

resource "aws_s3_access_point" "example" {
    bucket = aws_s3_bucket.example.id
    name   = "access-example"

    vpc_configuration {
        vpc_id = xxxxxx
    }
}

I can successfully access any items I put in the bucket from the EC2 instance.

# curl -I https://<url of bucket>/item_in_bucket.bin
HTTP/1.1 200 OK
Date: Fri, 18 Feb 2022 05:40:46 GMT
Last-Modified: Mon, 20 Sep 2021 06:56:54 GMT
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Server: AmazonS3
Content-Length: 485313

The moment I create the VPC endpoint, I am no longer able to access

resource "aws_vpc_endpoint" "s3" {
  vpc_id       =  xxxxxx
  service_name = "com.amazonaws.us-west-2.s3"
  route_table_ids = [xxxxxxx]
}
HTTP/1.1 403 Forbidden
Content-Type: application/xml
Date: Fri, 18 Feb 2022 05:20:17 GMT
Server: AmazonS3