In this lesson, we’ll also explore performing tasks that go beyond the basic capabilities of CloudFormation templates, specifically when we want to install and configure services on resources, like EC2 instances, provisioned within the stack.

For this purpose, we’ll be using specialized scripts provided by AWS CloudFormation that we can execute inside the runtime of an EC2 instance to initialize, configure, and manage any packages and services on the EC2 instance.

Using CloudFormation helper scripts

The CloudFormation helper scripts help us initialize and configure any runtime-based AWS resources, like EC2 instances, that are provisioned using AWS CloudFormation templates. These helper scripts also allow us to manage the runtime even after it’s initialized. The following diagram lists all the helper scripts provided by CloudFormation:

Press + to interact
CloudFormation helper scripts
CloudFormation helper scripts

Let’s briefly explore all the CloudFormation helper scripts together with an example:

cfn-init helper script

The cfn-init helper script facilitates us in configuring an EC2 instance by executing the required set of initialization commands defined in the AWS::CloudFormation::Init section of the metadata. We can leverage the cfn-init helper script to install packages, create files, configure services, and perform other actions. Here are some of the common parameters for the cfn-init helper script:

  • -s|--stack: We provide the ID or name of the stack that contains the resource.

  • -r|--resource: We provide the logical ID of the resource whose metadata we want to execute to initialize the resource.

  • --region: We provide the region in which the stack is running.

Example: Installing the httpd package with cfn-init

Here’s a CloudFormation template to provision an EC2 instance and a security group. We use the cfn-init helper script to execute and install the httpd package, which we’ll use in the actual EC2 instance runtime to launch a simple httpd server.

Resources:
  Instance:
    Type: AWS::EC2::Instance
    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages: 
            yum:
              httpd: []
    Properties:
      ImageId: ami-0e731c8a588258d0d
      InstanceType: t2.micro
      SecurityGroupIds:
        - !GetAtt SecurityGroup.GroupId
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          # cfn-init helper script to install the httpd package
          /opt/aws/bin/cfn-init -s ${AWS::StackName} --region ${AWS::Region} -r Instance     
          # Launching a simple httpd server 
          systemctl start httpd.service
          systemctl enable httpd.service
          # Writing HTML file to serve on the httpd server
          echo "Hello World from Educative" > /var/www/html/index.html
  SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: 'Internal Security group'
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
cfn-init helper script example template

Here’s a brief explanation of the relevant cfn-init code above:

  • Lines 5–10: In the CloudFormation template above, we define the commands in the AWS::CloudFormation::Init metadata section to install the httpd.x86_64 package.

  • Lines 21–22: We define the cfn-init script command itself in the EC2 instance’s user data. We reference and provide references for the stack, region, and resource for which we want to execute the AWS::CloudFormation::Init metadata.

cfn-signal helper script

The cfn-signal helper script facilitates us in signaling to CloudFormation when a specific set of commands has been successfully executed. This way, the stack creation is only marked as complete when all of the commands in an EC2 instance have been executed. The cfn-signal helper script can be especially useful when an AWS resource is dependent on the complete initialization of an EC2 instance.

Example: Signaling successful script execution with cfn-signal

Here’s a template to signal CloudFormation when the httpd server is finally deployed:

Resources:
  Instance:
    Type: AWS::EC2::Instance
    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages: 
            yum:
              httpd: []
    CreationPolicy: 
      ResourceSignal:
        Timeout: PT5M
        Count: 1
    Properties:
      ImageId: ami-0e731c8a588258d0d
      InstanceType: t2.micro
      SecurityGroupIds:
        - !GetAtt SecurityGroup.GroupId
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          # cfn-init helper script to install the httpd package
          /opt/aws/bin/cfn-init -s ${AWS::StackName} --region ${AWS::Region} -r Instance     
          # Launching a simple httpd server 
          systemctl start httpd.service
          systemctl enable httpd.service
          # Writing HTML file to serve on the httpd server
          echo "Hello World from Educative" > /var/www/html/index.html
          # cfn-signal helper script to signal initialization of the EC2 instance
          /opt/aws/bin/cfn-signal --stack ${AWS::StackId} --resource Instance --region ${AWS::Region}
  SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: 'Internal Security group'
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
cfn-signal helper script example template

Here’s a brief explanation of the relevant cfn-signal code above:

  • Lines 10–13: In the CloudFormation template above, we define the CreationPolicy resource attribute for the EC2 instance. The creation policy requires CloudFormation to wait at least five minutes for one signal from the EC2 instance before marking the EC2 instance and, hence, the stack creations as complete.

  • Lines 33–34: We define the cfn-signal script command itself in the EC2 instance’s user data after the commands to launch the httpd server.

cfn-hup helper script

The cfn-hup helper script is essentially a daemon. A daemon is a background process that runs continuously, often providing specific services. The cfn-hup helper script is used for detecting any changes in the metadata of any resource. It also executes a specific set of commands whenever it detects a change.

The cfn-hup helper script is useful when it comes to updating stacks where there are running EC2 instances, and we want a way to re-execute the existing initialization commands or any new commands on the existing runtime of these EC2 instances. If we tried updating the EC2 instance without using this helper script, then we either just manually run commands directly on the EC2 runtime or have the stack modify and replace the old EC2 instance with a new one.

It must be noted that any such change in the resource metadata must be made through the AWS CloudFormation service.

Disclaimer: If we just update the metadata of a running EC2 instance in the stack template but don’t utilize the cfn-hup helper script and don’t modify any property of the EC2 instance, then nothing will happen. The cfn-init helper script is not re-executed, and the EC2 instance will keep running the existing runtime without any changes.

Example: Starting a daemon with cfn-hup

Here’s a template to start a daemon using the CloudFormation cfn-hup helper script:

Parameters:
  WebsiteMessage:
    Type: String
    Default: 'Hello World from Educative'
Resources:
  Instance:
    Type: AWS::EC2::Instance
    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages: 
            yum:
              httpd: []
          files:
            "/var/www/html/index.html":
              content:
                Fn::Sub:
                  - |
                    <h1>${WebsiteMessage}</h1>
                  - WebsiteMessage: !Ref WebsiteMessage
              mode: "000644"
              owner: "root"
              group: "root"
            "/etc/cfn/cfn-hup.conf":
              content: !Sub |
                [main]
                stack=${AWS::StackId}
                region=${AWS::Region}
                verbose=true
                interval=1
              mode: "000400"
              owner: "root"
              group: "root"
            "/etc/cfn/hooks.d/1":
              content: !Sub |
                [cfn-auto-reloader-hook]
                triggers=post.update
                runas=root
                path=Resources.Instance.Metadata.AWS::CloudFormation::Init
                action=/opt/aws/bin/cfn-init --stack ${AWS::StackName} --resource Instance --region ${AWS::Region}
              mode: "000400"
              owner: "root"
              group: "root"
          services:
            sysvinit:
              httpd:
                enabled: 'true'
                ensureRunning: 'true'
    CreationPolicy: 
      ResourceSignal:
        Timeout: PT5M
        Count: 1
    Properties:
      ImageId: ami-0e731c8a588258d0d
      InstanceType: t2.micro
      SecurityGroupIds:
        - !GetAtt SecurityGroup.GroupId
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          # cfn-init helper script to install the httpd package
          /opt/aws/bin/cfn-init -s ${AWS::StackName} --region ${AWS::Region} -r Instance     
          # cfn-signal helper script to signal initialization of the EC2 instance
          /opt/aws/bin/cfn-signal --stack ${AWS::StackId} --resource Instance --region ${AWS::Region}
          # cfn-signal helper script is initialized as daemon
          /opt/aws/bin/cfn-hup || error_exit 'Failed to start cfn-hup'    
  SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: 'Internal Security group'
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
cfn-hup helper script example template

Here’s a brief explanation of the relevant cfn-hup code above:

  • Lines 1–4: We also define a CloudFormation parameter, which we reference in the EC2 instance metadata to retrieve the input value for the message we want to display on the website.

  • Line 25–44: In the CloudFormation template above, we migrated all UserData commands to launch the httpd server to the MetaData section of the EC2 instance. The idea is that we can change the message when updating the stack and have the server restart and display the new custom message.

  • Lines 69–70: We define the cfn-hup helper script command that launches a daemon that follows the configurations we define in the cfn-hup configuration files.

cfn-get-metadata helper script

The cfn-get-metadata helper script is an AWS CloudFormation-provided tool for retrieving the metadata of any AWS resource provisioned within the stack. When we execute the cfn-get-metadata helper script in the instance runtime, the entire metadata of the requested resource will be printed in the terminal.

With the cfn-get-metadata helper script, we can also use it to retrieve a sub-tree of the metadata block if we specify a top-level key.

Best practices

Here are some best practices that we can use when using CloudFormation helper scripts:

  • Update CloudFormation helper scripts before using them: AWS periodically pushes updates to CloudFormation helper scripts. For example, we can ensure to include any update commands in the user data property of an EC2 instance to make sure the latest helper script is executed on the initialized EC2 instance.

  • Always initialize EC2 with the cfn-init helper script in the template: We can utilize the cfn-init helper script to execute any commands provided in the EC2 instance’s metadata on its runtime.

This lesson taught us how to manage the runtimes on EC2 instances with the help of all the helper scripts provided by AWS CloudFormation.

Get hands-on with 1300+ tech skills courses.