Preventing Safety Risks in Leases

Explore how fencing prevents issues with distributed leases.

We'll cover the following

Fencing

Fencing is a simple technique used to prevent safety risks in leases.

Purpose

The main idea of fencing is to allow the system to block some nodes from performing some operations when these nodes are malfunctioning.

Note: In our previous example, nodes are malfunctioning in the sense that they think they hold a lease, while they don’t.

How it works

The locking subsystem can associate every lease with a monotonically increasing number. All the other systems can then use this number to keep track of the node that has performed an operation with the most recent lease. If a node with an older lease attempts to perform an operation, the system can detect that and reject it while also notifying the node that it’s not the lease owner anymore. The following illustration shows how fencing would work in practice:

This means lock management cannot be performed by a single part of the system in a distributed system, but it has to be done collectively by all the parts of the system that are protected by this lock. For this to be possible, the various components of the system need to provide the following basic capabilities:

  • The locking subsystem needs to provide a monotonically increasing identifier for every lock acquisition. Some examples of systems that provide this are Zookeeper via the zxid or the znode version number and Hazelcast as part of the fenced token provided via the FencedLock API.
  • Any external systems protected by the locks need to provide conditional updates with linearizability guarantees.

Get hands-on with 1400+ tech skills courses.