Amazon AppFlow

A flow transfers data bidirectionally between a source and a destination. The following table lists some of the AWS services and SaaS applications AppFlow supports as a source and destination.

AppFlow gets access to different services and applications by creating connections. These connections include all the necessary credentials, such as API tokens, usernames, and passwords, required to enable data flow between our applications.

Triggering flows

We need to create triggers in order to run a flow. These triggers determine how and when AppFlow transfers data between source and destination components. Following are the types of triggers AppFlow supports:

• On-demand flows: These flows are run manually by the users whenever they need to transfer data.

• Event-triggered flows: These flows run whenever a specific event is generated. For example, an event is generated when an insert operation is executed on the source side.

• Schedule-triggered flows: Users can specify a schedule to run flows. Schedule-triggered flows support two types of data transfers:

  • Full transfer: In this method, all available records, at the time of flow run in the source, are sent to the destination.

  • Incremental transfer: In this method, only the records created after the last successful flow run are sent to the destination.

Private flows in AppFlow

Amazon PrivateLink allows us to create private connections between AWS services without exposing them to the public internet. We can integrate AppFlow with PrivateLink to create private flows between AWS services and SaaS applications.

PrivateLink uses Amazon VPC endpoints to transfer data between our applications. These endpoints act as an entry point to privately access AppFlow. The following diagram illustrates how the source and destination applications are linked with AppFlow over a PrivateLink.

While using SaaS applications with PrivateLink, we must ensure that PrivateLink access is enabled in our account. Following are the SaaS applications that support PrivateLink:

• Salesforce

• Singular

• Snowflake

• Trend Micro

Use case: Detecting inappropriate messages sent in a Slack channel

Let’s assume we want to detect inappropriate messages sent in a Slack channel by using the ML services provided by Amazon, such as AWS Comprehend. We can achieve this goal easily by following the steps given below:

• Create an AppFlow flow using Slack as a source and S3 as the destination service. Here, we can specify the Slack channels we want to monitor.

• Create a Lambda function that uses AWS Comprehend to detect inappropriate content in a file available in the S3 bucket.

• Run the AppFlow flow, which will transfer data from Slack into an S3 bucket.

• Once our data file is uploaded to our S3 bucket, it will invoke our Lambda function, which will process our file.

Pricing in AppFlow

Amazon AppFlow follows the pay-as-you-go approach; there are no minimum fees or required upfront commitments. We are charged for every successful flow run, where AppFlow tries to fetch data from the source application. We are charged even if the source application sends an empty response.

In addition to this, the volume of data per GB is charged every month. The prices may vary depending if the destination application is hosted on AWS or not and whether they’re using PrivateLink.