What is the grpcurl request to access the server?

cmoulliard · October 28, 2020, 4:49pm

Hi,

In order to validate my ingress route to access the grpc server deployed on the server, I’m doing a grpcurl request but it fails with the following error

grpcurl -insecure waypoint-grpc.127.0.0.1.nip.io:443 service/List
Error invoking method “service/List”: failed to query for service descriptor “service”: rpc error: code = Unavailable desc = Bad Gateway: HTTP status code 502; transport: received the unexpected content-type “text/html”

What is the command to be used and service/Method to call ?
Why the service is returning a HTML response ?

Cheers

Charles

nic · October 28, 2020, 5:42pm

Hi,

An example of Listing Workspaces would be:

grpcurl --insecure -H "client-api-protocol: 1,1" -H "authorization: your_authtoken" localhost:9701 hashicorp.waypoint.Waypoint.ListWorkspaces
{
  "workspaces": [
    {
      "name": "default",
      "applications": [
        {
          "application": {
            "application": "wpmini",
            "project": "wpmini"
          },
          "activeTime": "2020-10-28T17:34:36.864888900Z"
        }
      ],
      "activeTime": "2020-10-28T17:34:36.864888900Z"
    }
  ]
}

You can list all the services with

➜ grpcurl --insecure localhost:9701 describe hashicorp.waypoint.Waypoint                                                                                                                     hashicorp.waypoint.Waypoint is a service:
service Waypoint {
  rpc BootstrapToken ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.NewTokenResponse );
  rpc CancelJob ( .hashicorp.waypoint.CancelJobRequest ) returns ( .google.protobuf.Empty );
  rpc ConvertInviteToken ( .hashicorp.waypoint.ConvertInviteTokenRequest ) returns ( .hashicorp.waypoint.NewTokenResponse );
  rpc CreateHostname ( .hashicorp.waypoint.CreateHostnameRequest ) returns ( .hashicorp.waypoint.CreateHostnameResponse );
  rpc DeleteHostname ( .hashicorp.waypoint.DeleteHostnameRequest ) returns ( .google.protobuf.Empty );
  rpc EntrypointConfig ( .hashicorp.waypoint.EntrypointConfigRequest ) returns ( stream .hashicorp.waypoint.EntrypointConfigResponse );
  rpc EntrypointExecStream ( stream .hashicorp.waypoint.EntrypointExecRequest ) returns ( stream .hashicorp.waypoint.EntrypointExecResponse );
  rpc EntrypointLogStream ( stream .hashicorp.waypoint.EntrypointLogBatch ) returns ( .google.protobuf.Empty );
  rpc GenerateInviteToken ( .hashicorp.waypoint.InviteTokenRequest ) returns ( .hashicorp.waypoint.NewTokenResponse );
  rpc GenerateLoginToken ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.NewTokenResponse );
  rpc GetBuild ( .hashicorp.waypoint.GetBuildRequest ) returns ( .hashicorp.waypoint.Build );
  rpc GetConfig ( .hashicorp.waypoint.ConfigGetRequest ) returns ( .hashicorp.waypoint.ConfigGetResponse );
  rpc GetDeployment ( .hashicorp.waypoint.GetDeploymentRequest ) returns ( .hashicorp.waypoint.Deployment );
  rpc GetJob ( .hashicorp.waypoint.GetJobRequest ) returns ( .hashicorp.waypoint.Job );
  rpc GetJobStream ( .hashicorp.waypoint.GetJobStreamRequest ) returns ( stream .hashicorp.waypoint.GetJobStreamResponse );
  rpc GetLatestBuild ( .hashicorp.waypoint.GetLatestBuildRequest ) returns ( .hashicorp.waypoint.Build );
  rpc GetLatestPushedArtifact ( .hashicorp.waypoint.GetLatestPushedArtifactRequest ) returns ( .hashicorp.waypoint.PushedArtifact );
  rpc GetLatestRelease ( .hashicorp.waypoint.GetLatestReleaseRequest ) returns ( .hashicorp.waypoint.Release );
  rpc GetLogStream ( .hashicorp.waypoint.GetLogStreamRequest ) returns ( stream .hashicorp.waypoint.LogBatch );
  rpc GetProject ( .hashicorp.waypoint.GetProjectRequest ) returns ( .hashicorp.waypoint.GetProjectResponse );
  rpc GetPushedArtifact ( .hashicorp.waypoint.GetPushedArtifactRequest ) returns ( .hashicorp.waypoint.PushedArtifact );
  rpc GetRelease ( .hashicorp.waypoint.GetReleaseRequest ) returns ( .hashicorp.waypoint.Release );
  rpc GetRunner ( .hashicorp.waypoint.GetRunnerRequest ) returns ( .hashicorp.waypoint.Runner );
  rpc GetServerConfig ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.GetServerConfigResponse );
  rpc GetVersionInfo ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.GetVersionInfoResponse );
  rpc GetWorkspace ( .hashicorp.waypoint.GetWorkspaceRequest ) returns ( .hashicorp.waypoint.GetWorkspaceResponse );
  rpc ListBuilds ( .hashicorp.waypoint.ListBuildsRequest ) returns ( .hashicorp.waypoint.ListBuildsResponse );
  rpc ListDeployments ( .hashicorp.waypoint.ListDeploymentsRequest ) returns ( .hashicorp.waypoint.ListDeploymentsResponse );
  rpc ListHostnames ( .hashicorp.waypoint.ListHostnamesRequest ) returns ( .hashicorp.waypoint.ListHostnamesResponse );
  rpc ListInstances ( .hashicorp.waypoint.ListInstancesRequest ) returns ( .hashicorp.waypoint.ListInstancesResponse );
  rpc ListProjects ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.ListProjectsResponse );
  rpc ListPushedArtifacts ( .hashicorp.waypoint.ListPushedArtifactsRequest ) returns ( .hashicorp.waypoint.ListPushedArtifactsResponse );
  rpc ListReleases ( .hashicorp.waypoint.ListReleasesRequest ) returns ( .hashicorp.waypoint.ListReleasesResponse );
  rpc ListWorkspaces ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.ListWorkspacesResponse );
  rpc QueueJob ( .hashicorp.waypoint.QueueJobRequest ) returns ( .hashicorp.waypoint.QueueJobResponse );
  rpc RunnerConfig ( stream .hashicorp.waypoint.RunnerConfigRequest ) returns ( stream .hashicorp.waypoint.RunnerConfigResponse );
  rpc RunnerGetDeploymentConfig ( .hashicorp.waypoint.RunnerGetDeploymentConfigRequest ) returns ( .hashicorp.waypoint.RunnerGetDeploymentConfigResponse );
  rpc RunnerJobStream ( stream .hashicorp.waypoint.RunnerJobStreamRequest ) returns ( stream .hashicorp.waypoint.RunnerJobStreamResponse );
  rpc SetConfig ( .hashicorp.waypoint.ConfigSetRequest ) returns ( .hashicorp.waypoint.ConfigSetResponse );
  rpc SetServerConfig ( .hashicorp.waypoint.SetServerConfigRequest ) returns ( .google.protobuf.Empty );
  rpc StartExecStream ( stream .hashicorp.waypoint.ExecStreamRequest ) returns ( stream .hashicorp.waypoint.ExecStreamResponse );
  rpc UpsertApplication ( .hashicorp.waypoint.UpsertApplicationRequest ) returns ( .hashicorp.waypoint.UpsertApplicationResponse );
  rpc UpsertBuild ( .hashicorp.waypoint.UpsertBuildRequest ) returns ( .hashicorp.waypoint.UpsertBuildResponse );
  rpc UpsertDeployment ( .hashicorp.waypoint.UpsertDeploymentRequest ) returns ( .hashicorp.waypoint.UpsertDeploymentResponse );
  rpc UpsertProject ( .hashicorp.waypoint.UpsertProjectRequest ) returns ( .hashicorp.waypoint.UpsertProjectResponse );
  rpc UpsertPushedArtifact ( .hashicorp.waypoint.UpsertPushedArtifactRequest ) returns ( .hashicorp.waypoint.UpsertPushedArtifactResponse );
  rpc UpsertRelease ( .hashicorp.waypoint.UpsertReleaseRequest ) returns ( .hashicorp.waypoint.UpsertReleaseResponse );
  rpc ValidateJob ( .hashicorp.waypoint.ValidateJobRequest ) returns ( .hashicorp.waypoint.ValidateJobResponse );
  rpc _ListJobs ( .hashicorp.waypoint.ListJobsRequest ) returns ( .hashicorp.waypoint.ListJobsResponse );
}

And individual parameters using something like:

➜ grpcurl --insecure --msg-template localhost:9701 describe hashicorp.waypoint.ListReleasesRequest
hashicorp.waypoint.ListReleasesRequest is a message:
message ListReleasesRequest {
  repeated .hashicorp.waypoint.StatusFilter status = 1;
  .hashicorp.waypoint.OperationOrder order = 2;
  .hashicorp.waypoint.Ref.Application application = 3;
  .hashicorp.waypoint.Ref.Workspace workspace = 4;
  .hashicorp.waypoint.Operation.PhysicalState physical_state = 5;
  .hashicorp.waypoint.Release.LoadDetails load_details = 6;
}

Message template:
{
  "status": [
    {
      "filters": [
        {

        }
      ]
    }
  ],
  "order": {
    "order": "UNSET",
    "desc": false,
    "limit": 0
  },
  "application": {
    "application": "",
    "project": ""
  },
  "workspace": {
    "workspace": ""
  },
  "physicalState": "UNKNOWN",
  "loadDetails": "NONE"
}

cmoulliard · October 28, 2020, 6:07pm

Many thanks. There is still a HTTP Error 502 even If I pass the headers and token

grpcurl -insecure -H "client-api-protocol: 1,1" -H "authorization: $TOKEN" waypoint-grpc.127.0.0.1.nip.io:443 hashicorp.waypoint.Waypoint.ListWorkspaces
Error invoking method "hashicorp.waypoint.Waypoint.ListWorkspaces": failed to query for service descriptor "hashicorp.waypoint.Waypoint": rpc error: code = Unavailable desc = Bad Gateway: HTTP status code 502; transport: received the unexpected content-type "text/html"

The log of the ingress reports such an error when the grpcurl request is issued

2020/10/28 18:12:47 [error] 259#259: *10230 upstream prematurely closed connection while reading response header from upstream, client: 172.18.0.1, server: wayp │
│ 172.18.0.1 - - [28/Oct/2020:18:12:47 +0000] "POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0" 502 150 "-" "grpc-go/1.30.0" 248 0.00 │
│ 2020/10/28 18:12:47 [error] 259#259: *10230 upstream prematurely closed connection while reading response header from upstream, client: 172.18.0.1, server: wayp │
│ 172.18.0.1 - - [28/Oct/2020:18:12:47 +0000] "POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0" 502 150 "-" "grpc-go/1.30.0" 43 0.000 │

nic · October 28, 2020, 7:22pm

Hi,

I am wondering if there is a problem with your ingress to the Waypoint API, what are using to front the gateway, is it just a Kubernetes Loadbalancer or something like that?

Kind regards,

Nic

cmoulliard · October 28, 2020, 7:23pm

I did some modifications to my ingress resource and it looks like the grpc server is doing something (even if no log appears within the pod of the waypoint server )

 grpcurl -insecure -H "client-api-protocol: 1,1" -H "authorization: $TOKEN" waypoint-grpc.127.0.0.1.nip.io:443 hashicorp.waypoint.Waypoint.ListWorkspaces
Error invoking method "hashicorp.waypoint.Waypoint.ListWorkspaces": failed to query for service descriptor "hashicorp.waypoint.Waypoint": rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: PROTOCOL_ERROR

ingress log

2020/10/28 19:03:56 [warn] 1319#1319: *48172 a client request body is buffered to a temporary file /tmp/client-body/0000000009, client: 172.18.0.1, server: waypoint-grpc.127.0.0.1.nip.io, request: "POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0", host: "waypoint-grpc.127.0.0.1.nip.io:443"
172.18.0.1 - - [28/Oct/2020:19:04:56 +0000] "POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0" 408 0 "-" "grpc-go/1.30.0" 43 59.933 [default-waypoint-grpc] [] - - - - 647ca6a21793e904567e7608732e7b63

cmoulliard · October 28, 2020, 8:41pm

it is a kind - kubernetes cluster where ingress is used a gateway between the local host and the cluster. No loadbalancer is deployed as they cannot work with docker on macos

FYI: I did a test using this example https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/grpc successfully

nic · October 28, 2020, 9:37pm

Awesome thanks, I will give it a test in the morning. I was running my tests on K3D which I really really rate. Rancher delivered a great product there.

cmoulliard · October 29, 2020, 11:29am

I was able to fix the ingress issue using this configuration

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: GRPCS
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  labels:
    app: waypoint-grpc
  name: waypoint-grpc
    secretName: waypoint-grpc
  rules:
  - host: waypoint-grpc.127.0.0.1.nip.io
    http:
      paths:
      - backend:
          serviceName: waypoint
          servicePort: grpc
        path: /

I can now query the gRPC server exposed by nginx

grpcurl -insecure -H "client-api-protocol: 1,1" -H "Authorization: $TOKEN" waypoint-grpc.127.0.0.1.nip.io:443 describe hashicorp.waypoint.Waypoint 
hashicorp.waypoint.Waypoint is a service:
service Waypoint {
  rpc BootstrapToken ( .google.protobuf.Empty ) returns ( .hashicorp.waypoint.NewTokenResponse );
  rpc CancelJob ( .hashicorp.waypoint.CancelJobRequest ) returns ( .google.protobuf.Empty );

cmoulliard · October 29, 2020, 12:13pm

Here are the instructions that I followed to install the waypoint server (using kubernetes resources) on a kind kubernetes cluster where we can access it using the ingress controller.

I still have some authentication issue like this one like also a deadline exceeded which is really annoying

 waypoint context verify my-k8s-server
✓ Context "my-k8s-server" connected successfully.
waypoint token new                   
! Authorization token is not supplied

waypoint token new
! failed to create client: context deadline exceeded

See: https://github.com/ch007m/kind-config/blob/master/README-waypoint.md#waypoint-grpc--http-server

nic · October 29, 2020, 3:43pm

Thanks, @cmoulliard, we have had a couple of strange issues surrounding Kind which we are addressing. We should have an answer on this today, your link is super useful for setting up Kind info too.

Kind regards,

Nic

cmoulliard · October 29, 2020, 3:47pm

Yep. I discovered some other problems that I reported as issues on the github waypoint project