How to Deploy a gRPC Service

Deploying a gRPC service is the point where a lot of people discover that 'this platform supports HTTP' and 'this platform supports gRPC' are very different claims. The service runs fine locally. You deploy it, point a client at the URL, and get a connection error that mentions neither HTTP/2 nor the proxy that broke it. The debugging session that follows is usually an hour long and ends in the same realisation.

I'm Ajay, I build PandaStack. This post is the thing I wish someone had written before my first gRPC deploy: what the actual requirement is, how to test whether a platform meets it in thirty seconds, and the three deployment shapes that work — including the one where you keep gRPC internally and speak something else at the edge.

The requirement, precisely

gRPC is not 'a protocol that uses HTTP/2 as a transport detail'. It requires HTTP/2 features that have no HTTP/1.1 equivalent, which is why there's no automatic downgrade:

• Bidirectional streaming over a single connection. HTTP/1.1 has no multiplexing, so a streaming RPC has nothing to map onto.
• Trailers. gRPC sends its status code in HTTP trailers after the body. HTTP/1.1 technically permits trailers with chunked encoding, but almost nothing in the proxy world handles them.
• Long-lived connections carrying many concurrent streams, which is where the performance comes from in the first place.

The consequence is a single hard rule: HTTP/2 must survive every hop between the client and your process. One proxy in the middle that terminates HTTP/2 and speaks HTTP/1.1 upstream breaks gRPC completely, even though it will happily serve your REST endpoints from the same binary. That is why the symptom is so confusing — half your service works.

Step 1: Test the path before you build on it

Thirty seconds of testing saves the hour. Deploy a trivial gRPC server with reflection enabled and probe it with grpcurl:

# Does the endpoint speak gRPC end-to-end?
grpcurl your-app.example.com:443 list

# Success: a list of services.
# "server does not support the reflection API" -> gRPC works, reflection is off.
# "connection closed" / "unexpected HTTP status 400" -> a hop is HTTP/1.1.

You can also check the negotiation directly. If ALPN doesn't advertise h2, the connection was never going to carry gRPC:

# ALPN should negotiate h2. If it says http/1.1, stop here.
openssl s_client -alpn h2 -connect your-app.example.com:443 </dev/null 2>&1 \
  | grep -i "ALPN protocol"

Do this against your actual deployed URL, not localhost, and do it before you write the client. The whole class of confusion comes from testing the server and assuming the path.

Shape 1: End-to-end HTTP/2, if your platform gives you it

The clean case. You need a platform that either hands you a raw TCP listener or explicitly documents gRPC support with h2c to your container. Google Cloud Run, AWS ALB with a gRPC target group, Fly.io with a TCP handler, and any VM or Kubernetes cluster where you own the ingress all qualify. On Kubernetes this means an ingress controller configured for gRPC backends — nginx needs the grpc_pass annotation, and it is not the default.

If you have this, deploy normally and stop reading. If your platform's router is an HTTP reverse proxy you don't control — and most application platforms' routers are — you need one of the next two shapes.

Shape 2: gRPC internally, a gateway at the edge

This is the shape I'd recommend to most teams, and not just as a workaround — it's a better architecture for a service with external consumers regardless.

You keep your protobuf definitions and your gRPC implementation. You put a transcoding gateway in front that accepts ordinary JSON over HTTP/1.1 and translates to gRPC on localhost. External clients get a REST API generated from the same proto files; internal service-to-service calls use native gRPC and never leave the machine, so the HTTP/2 requirement is satisfied trivially.

// gateway.go -- grpc-gateway in front of a local gRPC server.
// The proxy hop is HTTP/1.1 JSON; the internal hop is native gRPC.
package main

import (
	"context"
	"net/http"
	"os"

	"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"

	pb "example.com/api/gen"
)

func main() {
	ctx := context.Background()

	// Dial the gRPC server running in this same VM. No network, no TLS,
	// no proxy in between -- HTTP/2 is trivially available on loopback.
	conn, err := grpc.NewClient("127.0.0.1:9000",
		grpc.WithTransportCredentials(insecure.NewCredentials()))
	if err != nil {
		panic(err)
	}

	mux := runtime.NewServeMux()
	if err := pb.RegisterOrdersHandler(ctx, mux, conn); err != nil {
		panic(err)
	}

	// This is what the platform's HTTP/1.1 router talks to.
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}
	if err := http.ListenAndServe(":"+port, mux); err != nil {
		panic(err)
	}
}

The mapping between RPCs and HTTP routes comes from annotations in the proto file, so there's one source of truth and no hand-written REST layer to drift:

service Orders {
  rpc GetOrder(GetOrderRequest) returns (Order) {
    option (google.api.http) = { get: "/v1/orders/{order_id}" };
  }
  rpc CreateOrder(CreateOrderRequest) returns (Order) {
    option (google.api.http) = { post: "/v1/orders", body: "*" };
  }
}

One caveat that trips people up: streaming RPCs don't transcode cleanly. Server-streaming becomes a chunked JSON stream, and client- or bidirectional-streaming has no sensible REST mapping at all. If a streaming method must be reachable externally, use gRPC-Web or a WebSocket for that specific method rather than trying to force it through the gateway.

Shape 3: gRPC-Web for browsers

gRPC-Web exists because browsers cannot do raw gRPC either — the fetch API gives no access to HTTP/2 frames or trailers. It's a wire format that works over HTTP/1.1, with a proxy translating to real gRPC on the backend. Envoy has a built-in filter; the standalone grpcwebproxy does the same job in one binary.

The limitation is the same one: server-streaming works, client-streaming and bidirectional do not. If your API design leans on bidirectional streams, gRPC-Web won't carry it and you want a WebSocket for that path.

How this plays out on PandaStack

Being straightforward about our own constraint, since it's the same one most application platforms have. Our app and per-port URLs are HTTP reverse proxies — the request goes through the control plane and then to the agent hosting your microVM, and those hops are HTTP/1.1. So native gRPC from an external client to a PandaStack app does not work today, and you'd hit exactly the confusing error described above.

What does work, and works well, is shape 2. Because an app runs in a full Firecracker microVM rather than a constrained function runtime, you can run your gRPC server and a gateway as two processes in the same VM. Internal gRPC on loopback is native and fast; the outside world sees JSON over HTTP.

#!/bin/sh
# start.sh -- gRPC server on loopback, gateway on the public port.
set -e

# Native gRPC, bound to loopback only. Nothing off-VM reaches it.
./bin/orders-server --listen 127.0.0.1:9000 &

# Wait for it before the gateway starts accepting traffic.
for i in $(seq 1 40); do
  grpc_health_probe -addr=127.0.0.1:9000 >/dev/null 2>&1 && break
  sleep 0.25
done

# The gateway is what the platform's HTTP router talks to.
exec ./bin/orders-gateway --port "${PORT:-8080}"

The checklist

1. Probe the deployed URL with grpcurl before writing any client code. Confirm ALPN negotiates h2.
2. Enumerate every hop: CDN, platform router, load balancer, ingress controller, sidecar. Any one of them terminating HTTP/2 ends the discussion.
3. If the path is clean, deploy normally. If it isn't, run gRPC on loopback and put a transcoding gateway on the public port.
4. Check which of your methods stream. Streaming is what survives transcoding least well and it's the thing you find out about last.
5. Add a health check the platform can actually use — an HTTP endpoint on the gateway, not a gRPC health RPC the router can't speak.

The summary

gRPC needs HTTP/2 unbroken from client to process, and most application platforms' routers quietly terminate it. Test with grpcurl against the real URL before you build anything. If the path is clean, deploy normally. If it isn't — and it usually isn't — keep gRPC on loopback between your own processes and put a grpc-gateway or gRPC-Web proxy on the public port. You keep your protos as the single source of truth, internal calls stay native gRPC, and external clients get an API that works over the HTTP/1.1 path every platform actually gives you.