Task Distribution Tasks with ZeroMQ

We’re now starting to understand how ZeroMQ is different from traditional web services and why it’s a perfect tool for building a distributed messaging system.

Building a distributed hashsum cracker with ZeroMQ

Now it’s time to build a sample application to see the properties of the PUSH/PULL sockets we just described in action.

A simple and fascinating application to work with would be a hashsum cracker: A system that uses a brute-force approach to try to match a given hashsum (such as MD5 or SHA1) to the hashsum of every possible variation of characters of a given alphabet, therefore discovering the original string that the given hashsum was created from.

This is an embarrassingly parallel workload, which is perfect for building an example demonstrating the power of parallel pipelines.

Note: Never use plain hashsums to encrypt passwords as they’re very easy to crack. Instead, use ia purpose-built algorithm such as bcrypt, scrypt, PBKDF2, or Argon2.

For our application, we want to implement a typical parallel pipeline where we have the following:

• A node to create and distribute tasks across multiple workers

• Multiple worker nodes (where the actual computation happens)

• A node to collect all the results

The system we just described can be implemented in ZeroMQ using the following architecture: