protocol buffer

Web 评论
Developer Guide

.proto 文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

Once you’ve defined your messages, you run the protocol buffer compiler for your application’s language on your .proto file to generate data access classes. These provide simple accessors for each field (like name() and set_name()) as well as methods to serialize/parse the whole structure to/from raw bytes

You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code.

Language Guide
Defining A Message Type
1
2
3
4
5
6
7
syntax = "proto3"; // First non-empty; first non-comment line

message SearchRequest {
  string query = 1; // unique numbered tag
  int32 page_number = 2;
  int32 result_per_page = 3;
}
Specifying Field Types
Assigning Tags

1-15 one byte 16-2047 two bytes you should reserve the tags 1 through 15 for very frequently occurring message elements. Remember to leave some room for frequently occurring elements that might be added in the future.

range: 1 to 536,870,911 You also cannot use the numbers 19000 through 19999 (FieldDescriptor::kFirstReservedNumber through FieldDescriptor::kLastReservedNumber)

Specifying Field Rules
  • singular zero or one of this field
  • repeated any number of times
Adding More Message Types
Reserved Fields
1
2
3
4
message Foo {
  reserved 2, 15, 9 to 11;
  reserved "foo", "bar";
}

Note that you can’t mix field names and tag numbers in the same reserved statement.

What’s Generated From Your .proto?
Default Values

sigular: - string - byte - bool - numeric type - enum - message field

repeated: - repeated filed

Enumerations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
  enum Corpus {
    UNIVERSAL = 0;
    WEB = 1;
    IMAGES = 2;
    LOCAL = 3;
    NEWS = 4;
    PRODUCTS = 5;
    VIDEO = 6;
  }
  Corpus corpus = 4;
}

You can define aliases by assigning the same value to different enum constants

1
2
3
4
5
6
7
8
9
10
11
12
enum EnumAllowingAlias {
  option allow_alias = true;
  UNKNOWN = 0;
  STARTED = 1;
  RUNNING = 1;
}

enum EnumNotAllowingAlias {
  UNKNOWN = 0;
  STARTED = 1;
  // RUNNING = 1;  // Uncommenting this line will cause a compile error inside Google and a warning message outside.
}
Reserved Values
1
2
3
4
enum Foo {
  reserved 2, 15, 9 to 11, 40 to max;
  reserved "FOO", "BAR";
}

Note that you can’t mix field names and numeric values in the same reserved statement.

Using Other Message Types

Define a message in the same .proto file.

1
2
3
4
5
6
7
8
9
message SearchResponse {
  repeated Result results = 1;
}

message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
}
Importing Definitions

By default you can only use definitions from directly imported .proto files. import "myproject/other_protos.proto";

1
2
3
4
5
6
7
8
9
10
11
// new.proto
// All definitions are moved here
======================================================
// old.proto
// This is the proto that all clients are importing.
import public "new.proto";
import "other.proto";
======================================================
// client.proto
import "old.proto";
// You use definitions from old.proto and new.proto, but not other.proto

The protocol compiler searches for imported files in a set of directories specified on the protocol compiler command line using the -I/–proto_path flag. If no flag was given, it looks in the directory in which the compiler was invoked. In general you should set the –proto_path flag to the root of your project and use fully qualified names for all imports.

Using proto2 Message Types

It’s possible to import proto2 message types and use them in your proto3 messages, and vice versa. However, proto2 enums cannot be used directly in proto3 syntax (it’s okay if an imported proto2 message uses them).

Nested Types
1
2
3
4
5
6
7
8
message SearchResponse {
  message Result {
    string url = 1;
    string title = 2;
    repeated string snippets = 3;
  }
  repeated Result results = 1;
}

If you want to reuse this message type outside its parent message type, you refer to it as Parent.Type:

1
2
3
message SomeOtherMessage {
  SearchResponse.Result result = 1;
}

You can nest messages as deeply as you like

1
2
3
4
5
6
7
8
9
10
11
12
13
14
message Outer {                  // Level 0
  message MiddleAA {  // Level 1
    message Inner {   // Level 2
      int64 ival = 1;
      bool  booly = 2;
    }
  }
  message MiddleBB {  // Level 1
    message Inner {   // Level 2
      int32 ival = 1;
      bool  booly = 2;
    }
  }
}
Updating A Message Type
  • Don’t change the numeric tags for any existing fields
  • If you add new fields, any messages serialized by code using your “old” message format can still be parsed by your new generated code
  • Fields can be removed, as long as the tag number is not used again in your updated message type
    • You may want to rename the field instead, perhaps adding the prefix “OBSOLETE_”, or make the tag reserved, so that future users of your .proto can’t accidentally reuse the number.
  • Compatibility
    • int32, uint32, int64, uint64, and bool are all compatible
    • sint32 and sint64 are compatible with each other but are not compatible with the other integer types
    • string and bytes are compatible as long as the bytes are valid UTF-8
    • Embedded messages are compatible with bytes if the bytes contain an encoded version of the message
    • fixed32 is compatible with sfixed32, and fixed64 with sfixed64
    • enum is compatible with int32, uint32, int64, and uint64 in terms of wire format (note that values will be truncated if they don’t fit)
  • Moving any fields into an existing oneof is not safe
Any
1
2
3
4
5
6
import "google/protobuf/any.proto";

message ErrorStatus {
  string message = 1;
  repeated google.protobuf.Any details = 2;
}
Oneof

You can add fields of any type, but cannot use repeated fields

Features: - Setting a oneof field will automatically clear all other members of the oneof - If the parser encounters multiple members of the same oneof on the wire, only the last member seen is used in the parsed message - If you’re using C++, make sure your code doesn’t cause memory crashes - Again in C++, if you Swap() two messages with oneofs, each message will end up with the other’s oneof case

Maps

map<key_type, value_type> map_field = N The key_type can be any integral or string type. The value_type can be any type except another map.

  • Map fields cannot be repeated
  • Wire format ordering and map iteration ordering of map values is undefined
  • When generating text format for a .proto, maps are sorted by key
  • When parsing from the wire or when merging, if there are duplicate map keys the last key seen is used. When parsing a map from text format, parsing may fail if there are duplicate keys

backwords compatibility:

1
2
3
4
5
6
message MapFieldEntry {
  key_type key = 1;
  value_type value = 2;
}

repeated MapFieldEntry map_field = N;
Packages
JSON Mapping