AWS' SageMaker runtime integration

Terraform Providers AWS
1

Hi.

I’m pretty new with terraform. So far, so good. But I’ve been trying to integrate SageMaker runtime endpoints with an api endpoint. The way is described here: Creating a machine learning-powered REST API with Amazon API Gateway mapping templates and Amazon SageMaker | AWS Machine Learning Blog

These kind of integrations are easy with lambdas, but I’d prefer to skip them because they are not necessary.

Do you know if Sagemaker-runtime endpoints integrations are supported?

I’d appreciate any help on this. Thanks in advance.

2

Oh! I found the way.

It works like any other AWS service integration. With a few twists: we need to properly form the “uri” parameter. And create a role that the api gateway can assume at the moment of the request.

More in detail, if we have a Sage Maker endpoint, we need to use its NAME not the ARN; and we create a role that allows invocations on this endpoint at a certain region… The integration config would look like this:

resource "aws_api_gateway_integration" "YOUR-RESOURCE-INTEGRATION-NAME" {
  rest_api_id = aws_api_gateway_rest_api.YOUR-API-NAME.id
  resource_id = aws_api_gateway_resource.YOUR-RESOURCE-NAME.id
  http_method = aws_api_gateway_method.YOUR-METHOD-NAME.http_method

  type                    = "AWS"
  credentials             = "ARN ROLE THAT ALLOWS THE API TO INVOKE YOUR SM ENDPOINT"
  integration_http_method = "POST"
  uri                     = "arn:aws:apigateway:[REGION]:runtime.sagemaker:path/endpoints/[SAGEMAKER-ENDPOINT-NAME]/invocations"
  passthrough_behavior    = "WHEN_NO_MATCH"
}

Once we have that, still doesn’t work. It is necessary to add an integration response. Otherwise our requests could fail, because the API is not able to directly consume Sage Maker responses and fallbacks to a 500 internal server error. Despite you can see the correct outcome from your model in the logs.

To fix that, add an integration response. Something simple like this should work:

resource "aws_api_gateway_integration_response" "DEMO_INTEGRATION_RESPONSE" {
  rest_api_id = aws_api_gateway_rest_api.YOUR API NAME.id
  resource_id = aws_api_gateway_resource.YOUR RESOURCE NAME.id
  http_method = aws_api_gateway_method.YOUR METHOD NAME.http_method
  status_code = aws_api_gateway_method_response.response_200.status_code
}

Obviously this could me more solid by adding templates to format the output, etc… But at least is good enough for a poc or as a starting point.

I hope it helps.

1 Like