A Comprehensive Guide to gRPC in Go

内容目录

Go GRPC Introduction

Go GRPC Introduction
GRPC

1. Installation Packages

grpc

The golang-grpc package provides a code library related to gRPC, through which we can create gRPC services or clients. First, we need to install it.

go get -u google.golang.org/grpc

Protocol Plugins

To work with gRPC, we naturally need proto files, and we need to install two packages for processing protobuf files.

go get -u github.com/golang/protobuf  
go get -u github.com/golang/protobuf/protoc-gen-go

Note: There is a protoc-gen-go.exe file under GOPATH/bin, but this is just a plugin for protoc; it is not the protoc tool itself...

Protocol Buffers

Protocol Buffers is a language-agnostic, platform-agnostic extensible mechanism for serializing structured data, which is a data exchange format. gRPC uses protoc as the protocol processing tool.

When learning gRPC in Go, there's a pitfall not mentioned in many articles that you need to install this; otherwise, you'll receive an error message stating that the protoc command does not exist.

First, go to https://github.com/protocolbuffers/protobuf/releases to download the appropriate package; for example, I downloaded protoc-3.15.6-win64.zip.

After decompressing, copy the bin\protoc.exe file into the GOPATH\bin directory, alongside protoc-gen-go.exe.

Testing

Once everything is set up, create a test.proto file in a new directory, with the following content:

Note: There are many examples in the protoc-3.15.6-win64\include\google\protobuf directory.

syntax = "proto3";

// Package name
package test;

// Specify the output Go language source directory and file name
// The test.pb.go will be generated in the test.proto directory
// You can also simply use "./"
option go_package = "./;test";

// If you need to output other languages
// option csharp_package="MyTest";

service Tester {
  rpc MyTest(Request) returns (Response) {}
}

// Function parameters
message Request {
  string jsonStr = 1;
}

// Function return values
message Response {
  string backJson = 1;
}

Then, in the directory where the proto file is located, execute the command to convert the proto into the corresponding programming language file.

protoc --go_out=plugins=grpc:. *.proto

You will find that a test.pb.go file has been output to the current directory.

2. gRPC Server

Create a Go program, copying test.pb.go to the same directory as main.go, and import grpc in main.go:

import (
	"context"
	"fmt"
	"google.golang.org/grpc"
	// The default package name of test.pb.go is package main, no need to import here
	"google.golang.org/grpc/reflection"
	"log"
	"net"
)

In test.pb.go, two interfaces of Tester are generated. Let's take a look at the definitions of these two interfaces:

type TesterServer interface {
	MyTest(context.Context, *Request) (*Response, error)
}

type TesterClient interface {
	MyTest(ctx context.Context, in *Request, opts ...grpc.CallOption) (*Response, error)
}

To implement the service defined in the proto file, we need to implement the TesterServer interface. To implement the client, we need to implement the TesterClient interface.

Here, we will first implement the Server.

// Used to implement Tester service
type MyGrpcServer struct{}

func (myserver *MyGrpcServer) MyTest(context context.Context, request *Request) (*Response, error) {
	fmt.Println("Received a grpc request, request parameters:", request)
	response := Response{BackJson: `{"Code":666}`}
	return &response, nil
}

Next, we create the gRPC service.

func main() {
	// Create Tcp connection
	listener, err := net.Listen("tcp", ":8028")
	if err != nil {
		log.Fatalf("Listening failed: %v", err)
	}

	// Create gRPC service
	grpcServer := grpc.NewServer()

	// Tester register service implementer
	// This function is automatically generated in test.pb.go
	RegisterTesterServer(grpcServer, &MyGrpcServer{})

	// Register reflection service on gRPC
	// func Register(s *grpc.Server)
	reflection.Register(grpcServer)

	err = grpcServer.Serve(listener)
	if err != nil {
		log.Fatalf("Failed to serve: %v", err)
	}
}

3. gRPC Client

Create a new Go project and copy test.pb.go to the same directory as main.go. The code for main.go will be:

package main

import (
	"bufio"
	"context"
	"google.golang.org/grpc"
	"log"
	"os"
)

func main() {
	conn, err := grpc.Dial("127.0.0.1:8028", grpc.WithInsecure())
	if err != nil {
		log.Fatal("Failed to connect to gRPC service,", err)
	}

	defer conn.Close()

	// Create gRPC Client
	grpcClient := NewTesterClient(conn)

	// Create request parameters
	request := Request{
		JsonStr: `{"Code":666}`,
	}

	reader := bufio.NewReader(os.Stdin)

	for {
		// Send request, call MyTest interface
		response, err := grpcClient.MyTest(context.Background(), &request)
		if err != nil {
			log.Fatal("Request failed, reason:", err)
		}
		log.Println(response)

		reader.ReadLine()
	}
}

4. Compile and Run

Since the package name used in creating test.pb.go is main, we need to compile multiple Go files together:

go build .\main.go .\test.pb.go

Then start the server and client separately. Each time you press the Enter key on the client, a gRPC message will be sent.

gRPC Requests and Responses

Here, we have learned the complete process of creating gRPC, from defining the protocol to creating the service and client. Next, we will learn some related knowledge and understand some details.

5. Others

proto.Marshal can serialize the request parameters, for example:

	// Create request parameters
	request := Request{
		JsonStr: `{"Code":666}`,
	}

	out, err := proto.Marshal(&request)
	if err != nil {
		log.Fatalln("Failed to encode address book:", err)
	}
	if err := ioutil.WriteFile("E:/log.txt", out, 0644); err != nil {
		log.Fatalln("Failed to write address book:", err)
	}

While proto.Unmarshal can deserialize.

We can also customize how to serialize and deserialize messages, here is an example:

b, err := MarshalOptions{Deterministic: true}.Marshal(m)

Interested readers can visit https://pkg.go.dev/google.golang.org/protobuf/proto#MarshalOptions

GRPC

Protobuf Buffer

Protobuf buffer is a data format, while Protobuf is the gRPC protocol; here, it needs to be differentiated.

Protobuf buffer is an open-source mechanism used by Google for serializing structured data. To define a protobuf buffer, you need to use the message definition.

message Person {
  string name = 1;
  int32 id = 2;
  bool has_ponycopter = 3;
}

In the open-source definition, each field has a number; = 1 is not an assignment but an identifier. Each field in a message has a unique identifier, and these numbers are used to identify fields in binary format (which may be compressed during data transmission). When the identifier ranges from 1-15, it requires one byte to store the identifier. Therefore, it is preferable to limit the number of fields in a message to no more than 15; identifiers 1-15 are used to define frequently occurring message elements. Of course, you can also use numbers between 16-2047 as identifiers, where it will require two bytes for storage.

For more detailed information, you can refer to the official documentation:

https://developers.google.com/protocol-buffers/docs/overview

Field Types

Field types are not listed in detail here; readers can refer to the official documentation. Here are a few common data types:

double, float, int32, int64, bool, string, bytes, enums.

Since gRPC needs to consider compatibility with C, C#, Java, Go, and other languages, the types in gRPC are not equivalent to the related types in the programming languages. These types are defined within gRPC, and converting them to types in programming languages may require some conversion mechanisms that can sometimes be quite cumbersome.

Field Rules

Each field can specify a rule indicated at the beginning of the field type definition.

There are three rules available:

required: A correctly formatted message must have exactly one of these fields; that is, a required field.
optional: A correctly formatted message may contain zero or one of this field (but not more than one, meaning the value is optional).
repeated: In a correctly formatted message, this field can occur multiple times (including zero times), and the order of repeated values will be preserved, indicating that this field can contain 0 to N elements.

Due to historical reasons, the encoding efficiency of repeated scalar numeric type fields is not high. New code should use the special option [packed=true] to achieve more efficient encoding. For example:

repeated int32 samples = 4 [packed=true];

In optional fields, we can set a default value for it. If this field is not filled when passing messages, the default value will be used:

optional int32 result_per_page = 3 [default = 10];

Protobuf

Next, I will introduce the gRPC protocol format (protobuf), and here is an example from the official documentation:

syntax = "proto3";
package tutorial;

import "google/protobuf/timestamp.proto";

The syntax indicates the version of the protocol;

the package indicates the name of this .proto file;

the import keyword allows you to include other .proto files in the current .proto file. Since gRPC's basic data types do not include a time format, you can introduce timestamp.proto.

Different programming languages have different ways of importing packages/libraries. C++ and C# use namespaces to differentiate code locations; Java distinguishes package names with directories and public classes; while Go allows arbitrary package name settings in a .go file.

As mentioned before, protoc can convert protocol files into specific code.

To maintain compatibility across various programming languages, we set the _package option, which supports generating differing code names for different languages.

For example:

option go_package = "Test";					// ...
option csharp_package = "MyGrpc.Protos";	// Generate namespace MyGrpc.Protos{}
option java_package = "MyJava.Protos";		// ...

Four Types of gRPC Service Methods

In addition to defining messages, protobuf can also define streaming interfaces.

gRPC allows you to define four types of service methods:

Unary RPC, where the client sends a single request to the server and receives a single response, just like a regular function call. What we discussed previously are all unary gRPC.
```
rpc SayHello(HelloRequest) returns (HelloResponse);
```
Server streaming RPC, where the client sends a request to the server and receives a stream to read back a series of messages. The client reads from the returned stream until no more messages are available. gRPC guarantees that messages are ordered within a single RPC call.

Client -> Server -> Returns Stream -> Client -> Receives Stream
```
rpc LotsOfReplies(HelloRequest) returns (stream HelloResponse);
```
Client streaming RPC, where the client writes a sequence of messages and sends them to the server via the provided stream. After the client has finished writing messages, it waits for the server to read the messages and return its response. gRPC again guarantees the order of messages within a single RPC call.

Client -> Sends Stream -> Server -> Receives Stream ->
```
rpc LotsOfGreetings(stream HelloRequest) returns (HelloResponse);
```
Bidirectional streaming RPC, where both parties use read-write streams to send a series of messages. These two streams operate independently, so both client and server can read and write in whatever order they prefer: for example, the server could wait for all client messages to arrive before writing a response, or it could read messages first before writing, or some other combination of read and write. The order of messages in each stream will be preserved.
```
rpc BidiHello(stream HelloRequest) returns (stream HelloResponse);
```

Compile proto

Earlier, we used protoc to compile .proto files into Go language, and to support compilation for Go, we need to install the protoc-gen-go plugin. For C#, the protoc-gen-csharp plugin can be installed.

It’s important to note that it is not mandatory to install protoc to convert .proto into programming language code.

For instance, in C#, you only need to place the .proto file in the project and install a library through the package manager, which automatically converts it into the corresponding code.

Back to the main point, let’s talk about commands for the protoc compiling .proto files.

Common parameters for protoc include:

 --proto_path=.		#Specify the proto file path; using . means the current directory
 --go_out=.			#Indicates the storage path for the compiled files; if compiling for C#, use --csharp_out
 --go_opt={xxx.proto}={path to xxx.proto}	
 # Example: --go_opt=Mprotos/bar.proto=example.com/project/protos/foo

The simplest compile command is:

protoc --go_out=.  *.proto

The --{xxx}_out directive is necessary, as it indicates that specific programming language code needs to be outputted.

The output file path is determined by the path where the command was executed. If we do not execute the command in the directory of the .proto file, the generated code will not be in the same location. To solve this problem, we can use: