Block Layer

Stack multiple blocks together to create a block layer in ResNet.

We'll cover the following

Chapter Goals:

  • Learn how a layer of blocks is organized
  • Understand the function for creating a layer of blocks

A. Block layers

A ResNet model is made up of four block layers. Each block layer will contain a different number of blocks, depending on the total number of weight layers in the ResNet model. For example, an 18 layer ResNet model has 2 blocks in each block layer.

The blocks within each block layer are connected, so the output of block i\small i is the input of block i+1\small i + 1 in the same layer. Furthermore, the four block layers themselves are connected, so the output of block layer j\small j becomes the input of block layer j+1\small j + 1.

First block layer of a ResNet model with 50 weight layers. The first bottleneck block uses a projection shortcut, so it is colored differently. The output of the first block layer becomes the input of the second block layer.
First block layer of a ResNet model with 50 weight layers. The first bottleneck block uses a projection shortcut, so it is colored differently. The output of the first block layer becomes the input of the second block layer.

The block layers account for a large majority of the weight layers in a ResNet model. However, they don't make up the entire structure of a ResNet model. The block layer is implemented in the ResNetModel class as the block_layer function:

Press + to interact
import tensorflow as tf
class ResNetModel(object):
    # __init__ and other functions omitted
    # Creates a layer of blocks
    def block_layer(self, inputs, filters, strides, num_blocks, is_training, index):
        with tf.compat.v1.variable_scope('block_layer{}'.format(index)):
            shortcut_filters = 4 * filters if self.bottleneck else filters
            block_fn = self.bottleneck_block if self.bottleneck else self.regular_block
            block_output = block_fn(inputs, filters, strides, is_training, 0, 
                shortcut_filters=shortcut_filters)
            # stack the blocks in this layer
            for i in range(1, num_blocks):
                block_output = block_fn(block_output, filters, 1, is_training, i)
            return block_output

As you can see, the function stacks multiple building blocks. The particular block to use (block_fn) depends on whether or not the model uses bottlenecks.

Get hands-on with 1300+ tech skills courses.