Protocol Buffers for the Rest of Us
Motivation
protoletariat has one goal: fixing the broken imports for the Python code generated by protoc.
Usage
Here's an example of how to use the tool, called protol:
- Create a few protobuf files
// thing1.proto
syntax = "proto3";
import "thing2.proto";
package things;
message Thing1 {
Thing2 thing2 = 1;
}
// thing2.proto
syntax = "proto3";
package things;
message Thing2 {
string data = 1;
}
- Run
protocon those files
$ mkdir out
$ protoc --python_out=out --proto_path=directory/containing/protos thing1.proto thing2.proto
- Run
protolon the generated code
$ protol --create-init --overwrite -g out --proto-path=directory/containing/protos thing1.proto thing2.proto
The out/thing1_pb2.py file should show a diff containing at least these lines:
-import thing2_pb2 as thing2__pb2
-
+from . import thing2_pb2 as thing2__pb2
How it works
At a high level, protoletariat converts absolute imports to relative imports.
However, it doesn't just convert any absolute import to a relative import.
The protol tool will only convert imports that were generated from .proto files. It does this by inspecting FileDescriptorProtos from the protobuf files.
The core mechanism is implemented using a simplified form of pattern matching, that looks at the Python AST, and if an import pattern is matched and corresponding rewrite rule is fired.
Help
$ protol --help
Usage: protol [OPTIONS] PROTO_FILES...
Rewrite protoc-generated imports for use by the proletariat.
Options:
-g, --generated-python-dir DIRECTORY
Directory containing generated Python code
[required]
-p, --proto-path DIRECTORY Protobuf file search path(s). Accepts
multiple values. [required]
--overwrite / --no-overwrite Overwrite all generated Python files with
modified imports
--create-init / --dont-create-init
Create an __init__.py file under the
`generated-python-dir` directory
--help Show this message and exit.
156 Oct 21, 2022
104 Dec 07, 2022
45 Dec 20, 2021
9 Nov 27, 2022
1 Dec 14, 2021
4 Feb 23, 2022
34 Jun 15, 2022
37 Nov 22, 2022
1 Nov 26, 2021
52 Nov 18, 2022
9 Jul 25, 2022
12 Oct 25, 2022
7 Dec 14, 2022
7k Dec 26, 2022
12 Oct 28, 2022
10 Dec 02, 2022
2 Nov 14, 2021
69 Dec 23, 2022
184 Jan 03, 2023
14 Dec 25, 2022