gRPC at Lyft

gRPC Meetup - SF

Chris Roche

Lyft, Software Engineer - Core Libraries

Howdy!

Background

Infrastructure @ Lyft

What problems are we trying to solve?

Solution: gRPC and Protocol Buffers!

Tier-Zero Core Services

Tier-Zero Core Services

gRPC Unary Interceptor

func RequestID(
    ctx context.Context,
    req interface{},
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler) (interface{}, error) {

    if meta, ok := metadata.FromContext(ctx); ok {
        if hdrs, ok := meta[RequestIDHeader]; ok && len(hdrs) > 0 {
            ctx = context.WithValue(ctx, RequestIDHeader, hdrs[0])
        }
    }

    return handler(ctx, req)
}

Chaining Interceptors

func Chain(wrappers ...grpc.UnaryServerInterceptor) grpc.UnaryServerInterceptor {
    return func(
        ctx context.Context,
        req interface{},
        info *grpc.UnaryServerInfo,
        handler grpc.UnaryHandler) (interface{}, error) {

        for i := len(wrappers) - 1; i >= 0; i-- {
            handler = wrapHandler(wrappers[i], info, handler)
        }

        return handler(ctx, req)
    }
}

func wrapHandler(
    wrapper grpc.UnaryServerInterceptor,
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler) grpc.UnaryHandler {

    return func(ctx context.Context, req interface{}) (interface{}, error) {
        return wrapper(ctx, req, info, handler)
    }
}

Chaining Interceptors

    grpc.NewServer(
        grpc.UnaryInterceptor(
            Chain(
                RequestID,
                Metrics,
                // etc...,
            ),
        ),
        // other server options...
    )

Future: protoc-gen-go plugin

Python Services

Python Services

Gevent + gRPC =

Envoy

Envoy

Goal

"The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem."

Design

Features

Topology

So what about gRPC?

Code Generation via protoc

Proto Extensions + Options

service HelloWorld {
  option (http_server_options).isHttpServer = true;

  rpc GetHttpHello (SayHelloRequest) returns (SayHelloResponse) {
    option (http_options).path = "/api/gethello";
    option (http_options).method = "get";
    option (http_options).impl = "test_http.handle_hello_world_get";
  }
}

Generated Server

@blueprint.route('/api/gethello', methods=['GET'])
def get_hello():
    if request.headers.get('Content-Type') == 'application/proto':
        try:
            input = SayHelloRequest()
            input.ParseFromString(request.data)

            # Call the actual implementation method
            resp = handle_hello_world_get(input)
            return resp.SerializeToString()
        except Exception as e:
            logger.warning(
                'Exception calling handle_hello_world_get on get_hello: {}'.format(repr(e))
            )
            raise e
    else:
        # Non proto application code goes here
        return handle_hello_world_get(request)

Generated Client

def get_hello(self, input):
    try:
        assert isinstance(input, SayHelloRequest)
        headers = {
            'Content-Type': 'application/proto'
        }
        response = self.get(
            '/api/gethello',
            data=input.SerializeToString(),
            headers=headers,
            raw_request=True,
            raw_response=True)
        op = SayHelloResponse()
        op.ParseFromString(response.content)

        return op
    except Exception as e:
        logger.warning(
            'Exception calling get_hello : {}'.format(repr(e))
        )
        raise e

Envoy filter to upgrade/downgrade gRPC

gRPC + Envoy + gEvent =

Organization & Process

Difficulties

protoc and plugins:
- Gnarly installation and API
- Differences in behavior between language plugins

Previously...
- Each project rolled their own build
- Divergent versions of protoc and plugins

Dockerized protoc + plugins

docker pull lyft_idl     # pull the builder image
git checkout -b updates  # create a new branch
vim proto/hello.proto    # modify IDL
make build               # builds generated code
git add * && git commit  # Commit IDL + generated

Lyft IDL Repository

Live Coding: gRPC KV Store + Envoy + Flask Python Client

Links

Thank you

Chris Roche

Lyft, Software Engineer - Core Libraries

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)