What Is Protobuf And Why Should You Use It?

What is protobuf?

If you’re a software engineer, you’ve likely heard of protobuf or protocol buffers — but what is it, exactly? Probobuf is a data serialization format that is efficient and easy to use. It’s ideal for applications that need to exchange data quickly and easily.

Let's start with a simple protobuf example.

Imagine having a Go struct (we’re a go shop at Streamdal) and you want to send an object to your frontend application, written in React or Javascript. Every language varies in its object implementation. A go struct is not the same thing as a javascript object. We can think of this like having two native speakers communicating without a translator. It just doesn't work.

How do we make this communication consistent?

In order to make the communication consistent, we should create a common language: a platform-neutral data format! This is where protobuf shines, let's take a closer look at protobuf and its benefits.

What are protobuf’s benefits?

Protobuf was designed with event-driven systems and microservices in mind. It is much more efficient than JSON and XML, making it ideal for transferring data over the network. Protoc, protobuf’s compilation tool, supports a wide range of programming languages, making it easy to integrate into your existing system.

Generally, proto schemas are:

Thanks to these advantages, protobuf is quickly becoming the standard for data exchange in event-driven systems and microservices.

Unsolved problems

While protobuf has a good number of benefits as mentioned above, there are still some open issues with protobuf validation. The current validation framework does a decent job of validating most data types, but there are some edge cases where it falls short.

For example, it’s not currently possible to validate that all fields in a key-value object have the correct data type.

Why We Still Think protobuf is the Future

Despite the many challenges protobuf faces, it is still widely considered to be the future of data exchange. Today protobuf remains one of the most popular data serialization formats, used by millions of developers around the world. Here are four reasons why we believe protobuf is still the future of data serialization:

Each of these points is important in its own right, but together they make protobuf the clear choice for data serialization. As long as efficiency, extensibility, and backward compatibility remain important, protobuf will continue to be the standard for data serialization.