Setup Steps
DLH.io Setup Guide for Amazon DynamoDB
Prerequisites
This connector replicates every Amazon DynamoDB table in one AWS region. Before you begin, make sure that you have:
- The AWS region of your tables, for example us-east-1. Tables in other regions need a separate connection.
- An IAM user with an access key (Access Key ID and Secret Access Key) that has the read-only policy below. Optionally, an IAM role that this user may assume, with an External ID.
- Recommended: DynamoDB Streams turned on for each table you sync (any view type). Tables without a stream are still synced, by reading the whole table on every run.
Instructions (Basic Connection Setup)
DLH.io connects to DynamoDB over HTTPS with AWS signed requests. The connection is read only: DLH.io never writes to, creates or changes your tables, and the policy below grants no write action.
-
In the AWS console open IAM, create a user (no console access), and add this inline policy:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["dynamodb:ListTables", "dynamodb:ListStreams"], "Resource": "*" }, { "Effect": "Allow", "Action": ["dynamodb:DescribeTable", "dynamodb:Scan", "dynamodb:BatchGetItem", "dynamodb:DescribeStream", "dynamodb:GetShardIterator", "dynamodb:GetRecords"], "Resource": ["arn:aws:dynamodb:*:*:table/*", "arn:aws:dynamodb:*:*:table/*/stream/*"] } ] }Narrow the table resources to the tables you want to sync if you prefer.
-
Open the user, choose Security credentials, then Create access key (use case Third-party service). Copy the Access Key ID and the Secret Access Key; AWS shows the secret only once.
-
Optional: to use a role instead, create a role whose trust policy allows this user to call sts:AssumeRole (with an sts:ExternalId condition if you want one), attach the same policy to the role, and allow the user to assume it.
-
Recommended: for each table, open Exports and streams and turn on the DynamoDB stream. View type New and old images or New image is the most efficient; Keys only also works.
-
In DLH.io enter a Name or Alias for this connection, in the Name/Alias field, that is unique from other connectors.
-
Enter in the Target Schema Prefix field, a name that will be unique in your destination where you wish to land the data.
-
Select the AWS Region and enter the Access Key ID and Secret Access Key. If you use a role, also enter the IAM Role ARN and, if the role's trust policy requires one, the External ID. The region cannot be changed after the connection is saved; create another connection for tables in a different region.
-
Choose the Sync Format: Packed (default) or Unpacked. You can change it later in the connection settings; after a change, reload the schema and run a historical re-sync.
-
Click Save & Test to validate the connection, then pick the tables to sync.
Save & Test signs in to AWS with the access key and lists the DynamoDB tables in the selected region, which confirms the region, the access key and the role (if any) in one step. If this step does not pass, check the access key, secret and region, and that the key (or the role it assumes) is allowed dynamodb:ListTables.
See How data is synchronized for how first and ongoing syncs work and for the Packed and Unpacked sync formats.
If any issues persist please contact our support team via the DLH.io Support Portal.