Maintaining Backwards Compatibility

Protocol negotiation

Protocol negotiation is another technique for maintaining backward compatibility through the use of explicitly versioned software.

Let’s consider an example mentioned previously, where the client of an application is a mobile application. Every version of the application needs to be backward compatible with all the client application versions running on user phones currently. This means that the staged approach described previously cannot be used when making backward incompatible changes since end users cannot be forced to upgrade to a newer version. Instead, the application can identify the version of the client and adjust its behavior accordingly.

For example, consider a feature introduced on version 4.1.2 that is backward incompatible with versions < 4.x.x. If the application receives a request from a 3.0.1 client, it can disable that feature to maintain compatibility. If it receives a request from a 4.0.3 client, it can enable the feature.

Patch version: This version is incremented when a backwards compatible bug fix is made to the software.

Minor version: This version is incremented when new functionality is added in a backward compatible way.

Major version: This version is incremented when a backwards incompatible change is made.

As a result, the clients of the software can easily understand the compatibility characteristics of new software and the associated implications.

When providing software as a binary artifact, the version is usually embedded in the artifact. The consumers of the artifact then need to take the necessary actions if it includes a backwards incompatible change, e.g., adjusting their application’s code.

Applying semantic versioning to live applications

Semantic versioning is implemented slightly differently in live applications. The major version needs to be embedded in the address of the application’s endpoint, while the minor and patch versions can be included in the application’s responses. For an example, see this. This is needed so that clients can automatically upgrade to newer versions of the software if desired, but only if these are backward compatible.

Application unaware of other applications consuming its data

In some cases, an application might not be aware of the other applications consuming its data. An example of this is the publish-subscribe model.

The publish-subscribe model

In the publish-subscribe model, the publisher does not necessarily need to know all the subscribers. However, it’s still very important to ensure that the consumers can deserialize and process any produced data successfully as its format changes.

One pattern used here is defining a schema for the data used by both producers and consumers. This schema can be embedded in the message itself. Otherwise, to avoid duplication of the schema data, a reference to the schema can be put inside the message, and the schema can be stored in a separate store. For example, this pattern is commonly used in Kafka via the Schema Registry.

However, it’s important to remember that producers and consumers are evolving independently, even in this case. Hence, consumers are not necessarily using the latest version of the schema used by the producer. So, producers need to preserve compatibility either by ensuring consumers can read data of the new schema using an older schema or by ensuring all consumers have started using the new schema before producing messages with it.

Note that similar considerations need to be made for the compatibility of the new schema with old data. For example, if consumers cannot read old data with the new schema, the producers might have to make sure everyone has consumed all the messages with the previous schema first. Interestingly, the Schema Registry defines different categories of compatibility along these dimensions, which determine what changes are allowed in each category and the upgrade process, e.g., if producers or consumers need to upgrade first. It can also check two different versions of a schema and confirm that they are compatible under one of these categories to prevent errors later on. See this, for example.

Two-phase deployment

Note that it’s not only changes in data that can break backward compatibility. Slight changes in the behavior or semantics of an API can also have serious consequences in a distributed system. For example, let’s consider a failure detector that uses heartbeats to identify failed nodes. Every node sends a heartbeat every one second, and the failure detector considers a node failed if it hasn’t received a single heartbeat in the last three seconds. This causes a lot of network traffic that affects the performance of the application, so we will increase the interval of a heartbeat from one to five seconds and the threshold of the failure detector from three to fifteen seconds.

Suppose we perform a rolling deployment of this change. In that case all the servers with the old version of the software will start thinking all the servers with the new version have failed. This is due to the fact that their failure detectors will still have the old deadline of three seconds, while the new servers will send a heartbeat every five seconds.

One way to make this change backward compatible would be to perform an initial change that increases the failure detector threshold from three to fifteen seconds. And then, follow this with a subsequent change that increases the heartbeat interval to five seconds only after the first change has been deployed to all the nodes. This technique of splitting a change into two parts to make it backward compatible is commonly used and it’s also known as two-phase deployment.