Amazon S3 is a simple storage service that allows for easy and secure file sharing to Data Core Service (DCS) and retrieves requested exports from DCS. To use Amazon S3, you need the bucket (location) you wish to access and the access key/secret for authentication.
Configure S3 access
To integrate DCS with Amazon S3:
- Go to Account Settings > Integrations > AWS.
- If you do not have access to S3 yet, you need to generate a key and secret. Click Generate Access Keys to create your key and secret:
You can now access Amazon S3 using command line or a third-party application.
Connect to Amazon S3
Use the command line
Install the AWS CLI tools and us the S3 command line reference guide, as needed.
Import files
Copy a local file to S3
aws s3 cp zaius_customers.csv s3://zaius-incoming/<your tracker ID>/ --sse
Copy a complete directory of files to S3
aws s3 sync /tmp/yourlocaldir/ s3://zaius-incoming-temp/<your tracker ID>/ --sse
Once you place a file in your Zaius-incoming bucket, DCS will automatically import it with no additional steps required. This allows you to fully automate the import process if you regularly create data for imports and can schedule that data to be added to your S3 bucket.
Export files
Retrieve an S3 file (retrieve all files under an export id)
aws s3 cp s3://zaius-outgoing/<your tracker ID>/data-exports/<id> . --recursive --sse
The above command outputs the contents of your requested ID to your current directory. To specify a location, replace the . with the directory path. For example, to output the directory to your desktop, specify /Users/<name>/Desktop (OSX) or C:\Users\<name>\Desktop (Windows).
DCS exports are a set of files in a directory folder identified by the export ID provided. Be sure you transfer the entire directory to get the entirety of the export.
Use an application
Download and configuration
- Download a free tool like Cyberduck.
- Launch Cyberduck and create a New Bookmark.
- Select Amazon S3 from the services dropdown menu.
- Give your connection a nickname (optional).
- Enter the AWS Access Key ID that you received from DCS.
- Enter the AWS Secret Access Key that you received from DCS.
- Open More Options and enter the path seen on the AWS card in Integrations. Do not copy
S3://, but preface the directory with a single forward-slash. Each import and export has a separate directory. - Close the dialog to save as a new bookmark.
You can find additional information on using Cyberduck to import files in their documentation.