Transfer of Data

Learn about the ways data is transferred over the network and its associated trade-offs.

We'll cover the following

In the previous lesson we studied the options for creating and parsing the data sent through the network. However, the main ways this data is sent and received and the associated trade-offs remain unanswered.

The above question does not refer to the underlying transfer protocols that are used, such as Ethernet, IP, TCP, UDP, etc. Instead, it refers to the two basic, high-level forms of communication, synchronous and asynchronous communication.

Synchronous and asynchronous communication

If node A is communicating synchronously with node B, node A will wait until it has received a response from node B before proceeding with subsequent tasks.

If node A is communicating asynchronously instead, this means that it does not have to wait until that request is complete; it can proceed and perform other tasks at the same time.

The Following illustration shows the difference between synchronous and asynchronous communication:

Example

Synchronous communication is typically used in cases where node A needs to retrieve some data to execute the next task, or it needs to be sure a side-effect has been performed successfully on node B’s system before proceeding.

Asynchronous communication is preferred in cases where the operation performed in node B can be executed independently without blocking other operations from making progress.

For example, an e-commerce system might need to know that a payment was performed successfully before dispatching an item, thus using synchronous communication. However, after dispatching the order, the process that sends the appropriate notification e-mail to the customer can be triggered asynchronously.

Implementing synchronous and asynchronous communication

Synchronous communication is usually implemented on top of the existing protocols, such as using HTTP to transmit data in JSON or some other serialization formats (e.g., protocol buffers)…

In the case of asynchronous communication between two nodes, there is usually a need to store incoming requests until they are processed persistently. This is needed so that requests are not lost even if the recipient node fails for some reason.

Note: This is not necessarily needed in synchronous communication because if the recipient crashes before processing a request, the sender will notice at some point, and it can retry the request. Even when using asynchronous communication, there can be cases where losing requests is not a big problem. For example, the autocomplete functionality of a search engine can be done asynchronously in the background, and losing some requests will lead to fewer suggestions to the user so that it might be acceptable.

These requests can be stored in an intermediate datastore to prevent loss in case of failures. The datastores that are typically used for this purpose belong in two main categories:

  • Message queues
  • Event logs

We will discuss these categories in detail in the next lesson.

Get hands-on with 1400+ tech skills courses.