AWS CloudTrail

Here are some use cases of CloudTrail:

• Audit the AWS Infrastructure: We can use CloudTrail to help audit our AWS infrastructure to affirm that it’s compliant with any industry and security regulations.

• Maintain AWS Infrastructure security: We can use CloudTrail to improve the infrastructure security to log all user activity and have an automated service like EventBridge to set up workflows to identify any unusual activity within the collected logs and trigger the appropriate actions.

• Operational statistics: By logging all operations that take place on AWS with CloudTrail we can extrapolate some insightful statistics from the logs. These statistics can be used to speed up debugging operational issues, investigate operational anomalies, or provide explanations to any operational queries. Some CloudTrail services, like CloudTrail Lake, also offer dashboards that display any visual trends.

CloudTrail events

A CloudTrail event is any action taken by an AWS user, role, resource, or service within our account. These actions are recorded as events within the logs by CloudTrail. CloudTrail events are of the following types:

• Management events: Management events represent actions that are performed on resources within our AWS account. For example, launching an EC2 instance (RunInstances) is considered a management event.

• Data events: Data events represent object-level actions that are performed within specific AWS services, like Amazon S3 or AWS Lambda. Data events provide detailed information about the resource operations performed on or within the resource itself. For example, uploading an object (PutObject) or deleting an object (DeleteObject) in an S3 bucket are considered data events, as these are actions related to the S3 object. In retrospect, creating an S3 bucket (CreateBucket) itself is a management event, as we’re not performing any action within a resource, but actually, the action is to create a resource.

• CloudTrail Insight events: CloudTrail Insight events represent any actions that are considered by the CloudTrail Insight service as an unusual activity within our AWS account. For example, let’s say our daily activity includes launching and deleting ~5 EC2 instances in our AWS account, which CloudTrail Insight sets as the baseline. Launching of ~50 EC2 instances would be considered a CloudTrail insight event as that’s considered an unusual activity.

A CloudTrail event can comprise any of the following key details:

• Identity of the user, role, or service that performed the action.

• Time of when the request to perform the action was made.

• Source IP address of the action request.

• Parameters passed in the action request.

• Response JSON returned by the AWS service.

CloudTrail services

CloudTrail is an expansive service that we can break into the following sub-services:

CloudTrail Event History

CloudTrail Event History is a service within CloudTrail that allows us to view, search, and download our AWS account’s immutable historical activity data in the most recent 90 days. CloudTrail Event History is useful for ad-hoc investigations and quick lookups into recent account activities. By default, CloudTrail Event History only logs management events.

CloudTrail Lake

CloudTrail Lake is a managed service within CloudTrail that offers features like security lake and auditing support to provide a unified view of all our AWS CloudTrail event data across a specific region, all regions, or even all accounts within an organization account.

CloudTrail Lake allows us to aggregate, immutably store, and query activity logs for operational and security insights of our AWS cloud. Compared to CloudTrail Event History, CloudTrail Lake is designed for more complex queries, long-term event data storage, and analysis of compliance and security incidents over extended periods.

CloudTrail trails

CloudTrail trails allow us to configure and specify which events we need to record and where we want to store them.

Unlike CloudTrail Event History, which logs events for 90 days, CloudTrail trails log data events and stores them in more long-term storage, like an S3 bucket. We can also customize if we want to set up a trail within specific regions, all regions, or even all accounts within an organization account.

We can filter events logged within a trail for specific actions. The recorded events are usually stored in an Amazon S3 bucket in the form of JSON objects or optionally logged within CloudWatch Logs for real-time monitoring.

CloudTrail Insights

CloudTrail Insights is an advanced service within CloudTrail that leverages AI and ML models to detect and notify us of any unusual activity within our AWS account. It monitors our activity regularly and constructs a baseline based on it. Using this baseline, it can identify unusual patterns or activity in our AWS account, like spikes in the usual activity, rarely or not at all used services being utilized, or provisioning of more costly resources than normal.

Hence, CloudTrail Insights can be helpful in identifying security breaches like compromised access credentials or insider threats. It also provides timely notifications to investigate the threat and mitigate the damage already done.

How CloudTrail works?

All CloudTrail events are maintained within CloudTrail Event History for 90 days, after which they are migrated and stored in an S3 bucket if we create a CloudTrail trail to capture data in it. Only tracking of management events is enabled by default, and we explicitly need to enable logging of data events but at an additional cost.

Here’s an example illustration diagram of how CloudTrail would normally log data:

In the diagram, we can notice how event history is used for short-term storage of management event history and how we can alternatively create trails for long-term trails from both management and data events.

Integration with other services

AWS CloudTrail is a versatile service that we can integrate with a multitude of AWS services. Doing so allows for a more comprehensive monitoring, logging, and analysis of our AWS account. Moreover, we can use CloudTrail with several services, like EventBridge, to automate triggering the appropriate action based on the logged events.

Here’s an illustration of how we can integrate and use CloudTrail with some of the most commonly used services:

Here’s a breakdown of the services we’ve shown, an example of CloudTrail integration in the diagram above, after CloudTrail collects and logs events.

• AWS EventBridge: By configuring an EventBridge, we can detect the relevant API calls we want to monitor via CloudTrail and, upon detection, trigger a notification with SNS.

• Amazon CloudWatch: We previously discussed that we can store a CloudTrail trail in CloudWatch logs. However, we can also set up a CloudWatch to monitor the CloudWatch logs for relevant events and, upon detection, trigger an alarm to send a notification via SNS.

• AWS SNS: We can set up an SNS topic and subscription to send an email to a mailing list of cloud administrators regarding any unusual activity.

• AWS S3 & S3 Glacier: We previously discussed that we can store a CloudTrail trail in an S3 bucket for long-term storage. Considering that log files can exponentially increase over time, and the basic S3 storage can be expensive, we can alternatively back up and store old irrelevant data within the S3 Glacier service.

• AWS Athena & ElasticSearch: We can use AWS querying services like AWS Athena and ElasticSearch to query and filter relevant events’ data from the JSON-formatted CloudTrail log files stored in the S3 bucket. With filtered and specific results, we can better analyze the data.

• Other tools: Alternatively, we can use third-party or AWS partner tools to analyze the JSON-formatted CloudTrail log files stored in the S3 bucket.

Benefits

Here are some of the benefits of the CloudTrail service:

• Collect and compile events from multiple sources, including any other cloud service providers or from any on-premises resources.

• By analyzing the activity logs, we can identify any unauthorized access or unintended actions. A query-based service, Amazon Athena, would be useful in querying only the relevant data from the huge collection of CloudTrail logs.

• With CloudTrail Lake, we can identify and store any critical events that would be especially important to share in an audit. The stored data is also immutable, meaning it cannot be altered or changed.

Understanding costs

Now, let’s look at the cost of different sub-services of the CloudTrail:

• CloudTrail Event History: CloudTrail Event History is free to use and allows us to view and fetch the event history of our AWS account for the most recent 90 days at no additional cost.

• CloudTrail Lake: The pricing for CloudTrail Lake is based on the amount of data ingested, stored, and analyzed. CloudTrail Lake is relatively more expensive than Cloud Event History and CloudTrail Trail, depending on our usage of the services.

• CloudTrail Trails: The pricing for CloudTrail trails is based on whether we’ve enabled any and set up any management or data events or integrated it with S3 and SNS services, which have their own separate costs. Depending on our usage, the cost for CloudTrail Trails is variable and can be cheap to expensive.

• CloudTrail Insights: CloudTrail Insights is an additional feature that incurs charges based on the number of write management events analyzed. Based on usage, CloudTrail Insights are relatively the most expensive and only available on the paid tier of AWS.

Understanding that AWS CloudTrail pricing is usage-based and varies by the type and amount of data monitored. Keep in mind that exact costs can vary, and we can check them on the AWS pricing page for accuracy.

This lesson taught us about the Amazon CloudTrail service, its potential benefits, and how it helps keep logs with which we can track activity within our AWS account.