Amazon Neptune

The main power of Amazon Neptune is its high-performance graph database engine that can store billions of relationships, and querying can be done in milliseconds latency. It supports popular graph models property-graph and W3C’s RDF and their respective query languages. Amazon Neptune uses the power of Neptune ML to improve the prediction accuracy by over 50% compared to non-graph databases.

Amazon Neptune is a highly available, durable, and fault-tolerant database. It supports low latency read replicas across three Availability Zones making it 99.99% available, improves the read capacity, and allows it to execute 100k queries per second. A Neptune cluster has three main components: primary DB instance, replica, and cluster volume. This primary cluster can have a maximum of 15 reader instances in a cluster.

Neptune global database

Amazon Neptune supports global databases to introduce read performance, high availability, and fault tolerance. Global database for Neptune:

• Spans multiple AWS Regions.

• Provides low latency reader instances for global reads.

• Improves fast recovery in case of whole AWS Region goes down.

• Have a primary DB cluster in one AWS Region and can have up to 5 secondary DB clusters in different regions.

Note: Only primary DB cluster can entertain write operations and replicates the data to the secondary DB clusters, and each secondary DB cluster can have up to 16 reader instances.

Global database in failover

A global database is more reliable than a standalone Neptune cluster in a failover event. It can recover quickly from a disaster with a downtime (RTO) of minutes. Two techniques can be adopted for failover: Detach-and-promote and Managed planned failover.

Detach-and-promote (manual unplanned recovery)

This method is used for unplanned outages in the primary DB cluster, where we remove one of the secondary DB clusters (with a small lag time) from the global database and then promote it to the primary DB cluster. The below diagram shows the steps for the manual failover process. Downtime depends on how quickly we perform the failover process, and data loss is typically in seconds but can be longer because of storage replication lag.

Managed planned failover

This approach is used for planned maintenance, upgrades, or scheduled operations. In this method, we relocate the primary DB cluster with one of the secondary DB clusters. This method has no data loss because of the synchronization between the primary and secondary clusters before failover. 

Performance and scaling 

Amazon Neptune enhances the performance by allowing us to scale the resources per our requirements. Scaling is available for the following three resources:

• Storage scaling: Neptune automatically scales the cluster’s storage and can expand to 128 TiB in all supported AWS Regions. The storage is checked on an hourly basis for cost calculation. 

Note: The maximum storage limit for China and GovCloud regions is 64 TiB. Neptune storage is billed according to the usage based on per GB-monthly increments. For I/Os, we are charged per million requests increments.

• Instance scaling: We can manually scale our DB cluster as required by modifying the existing DB instance class for each instance or adding new instances to the cluster. Each instance can have a different instance class, depending on the workload.

• Read scaling: Because Neptune works in a cluster manner, we can add multiple read replicas in a single DB cluster and connect to them directly. All the replicas share the same data as they are attached to the cluster storage. 

• Auto-scaling: Amazon Neptune also supports auto-scaling. Once enabled, it monitors the workload, scales out, and scales in automatically to meet our requirements. To enable auto-scaling on a Neptune DB cluster, there must be one primary writer instance and at least one read replica, and no read replica should be in any other state than “Available.”

Backup and restore

Two types of backups are available in Amazon Neptune: Automated and Manual. 

• Automated backups: These are continuous backups that are created automatically for a specified retention period. The default value for the retention period is 1 day and can extend to 35 days. We can’t turn off automated backups on Neptune; all the automated backups are deleted when a DB cluster is deleted. 

• Manual backups: These are the snapshots of the whole database and are persistent beyond the retention period. These are useful when we want to keep our database for a longer period. Manual backups can be shared across AWS Regions and accounts. 

We can restore the database by creating a new instance from automated or manual backup and modifying the parameters and settings according to the requirements. Amazon Neptune also supports point-in-time recovery from the DB cluster, allowing us to create a new DB instance from any point with 1-second granularity.