Limited Parallel Execution

To implement a limited parallel execution pattern with async/await, we can simply reuse the TaskQueue class that we created previously. We can either use it as it is or convert its internals to async/await. Converting the TaskQueue class to async/await is a trivial operation and we’ll leave this to you as an exercise. Either way, the TaskQueue external interface shouldn’t change; both implementations will have a runTask() method that returns a Promise that settles when the task has been run by the queue.

Starting with this assumption, converting the web spider v4 from promises to async/await is also a trivial task and we won’t show all the steps here as we won’t be learning anything new. Instead, what we’ll do in this section is examine a third variation of the TaskQueue class that uses async/await and a producer-consumer approach.

The producer-consumer approach

The general idea to apply this approach to our problem goes as follows:

• On one side, we have an unknown set of producers adding tasks to a queue.

• On the other side, we have a predefined set of consumers, responsible for extracting and executing the tasks from the queue, one at a time.

The following illustration should help us understand the setup: