The Amazon S3 connector exports your data from the app to an Amazon S3 bucket as JSON files, so you can use it in your own reporting and business intelligence (BI) tools. The connector only writes to the bucket and never reads data from it back into the app. You can also point it at S3-compatible storage from MinIO or Cloudflare R2.
With this connector, you can:
- Export orders, shipping requests, fulfillments, products, inventory, locations, purchases, transfers, arrivals, receipts, and exceptions to your bucket every 5 minutes.
- Choose which record types to export, and export only records created after a date you set.
- Control the folder path and file names of the exported files.
- Mask personal information in exported orders, shipping requests, and fulfillments.
- Export to Amazon S3, MinIO, or Cloudflare R2.
Setup
Before you begin, make sure you have:
- An Amazon S3 bucket, or a bucket in MinIO or Cloudflare R2.
- The bucket URL in the format
s3://bucketname/path/to/root, for examples3://my-bucket/exports. The bucket name always goes in this URL, for every provider. - The bucket region, for example
us-west-2. For Cloudflare R2, the region isauto. For MinIO, any value works, for exampleus-east-1. - An access key and secret key that can list the bucket and write objects to it.
- For MinIO or Cloudflare R2 only: your S3-compatible endpoint URL, for example
https://minio.example.com:9000orhttps://<account-id>.r2.cloudflarestorage.com.
To connect Amazon S3:
- Navigate to Integrations in the app.
- Click Add Integration, and then select Amazon S3.
- Enter your bucket URL in the Specify Amazon S3 bucket URL field, for example
s3://my-bucket/exports. - Enter the bucket region in the Specify AWS Region field.
- Enter your access key in the Specify AWS API Access Key field.
- Enter your secret key in the Specify AWS API Access Secret field.
- Under Storage provider, select Amazon S3, MinIO, or Cloudflare R2.
- If you selected MinIO or Cloudflare R2, enter your endpoint URL in the S3-compatible endpoint URL field, and leave Use path-style bucket addressing on. For Amazon S3, leave the endpoint field empty.
- Click Connect.
Settings
Each record type has its own group of settings. Record types: locations, orders, shipping requests, fulfillments, transfers, purchases, arrivals, receipts, products, inventory, and exceptions.
| Setting | Description | Options/Default |
|---|---|---|
| Separator for file name components | The character between the parts of each exported file name. | Colon (:), Underscore (_), Dash (-), Dot (.). Default: Colon (:) |
| Generate POSIX compatible file names | Removes characters other than letters, numbers, periods, dashes, and underscores from file names. | On/Off. Default: Off |
| Use path template when posting files to Amazon S3 | The folder path inside your bucket root. You can use {orgKey}, {entityType}, and {entityDate} (the UTC date the record was last updated). |
Default: pipe17/{orgKey}/{entityType}
|
| Mask Personal Identifiable Information (PII) in objects reported to Amazon S3 | Replaces personal information, such as names and addresses, with * in exported orders, shipping requests, and fulfillments. |
On/Off. Default: Off |
| Push <record type> to Amazon S3 | Exports that record type to your bucket, for example Push orders to Amazon S3. | On/Off. Default: Off |
| Only push <record type> created after | Exports only records of that type created after the date and time you set, for example Only push orders created after. | Optional date and time |
| Force <record type> resync | Exports records of that type again from the start, for example Force orders resync. | On/Off |
Flows
| Flow | Direction | Notes |
|---|---|---|
| Orders, shipping requests, and fulfillments | Pipe17 → Amazon S3 | Runs every 5 minutes when the matching Push … to Amazon S3 setting is on. Each new or updated record is written as a separate JSON file. If Mask Personal Identifiable Information (PII) in objects reported to Amazon S3 is on, personal information is masked. |
| Products, inventory, and locations | Pipe17 → Amazon S3 | Runs every 5 minutes when the matching Push … to Amazon S3 setting is on. Each new or updated record is written as a separate JSON file. |
| Purchases, transfers, arrivals, and receipts | Pipe17 → Amazon S3 | Runs every 5 minutes when the matching Push … to Amazon S3 setting is on. Each new or updated record is written as a separate JSON file. |
| Exceptions | Pipe17 → Amazon S3 | Runs every 5 minutes when Push exceptions to Amazon S3 is on. Each new or updated exception is written as a separate JSON file. |
| Data from the bucket | Not supported | The connector never reads files from your bucket into the app. |
Files are saved at <bucket root>/<path template>/<file name>.json. Each file name contains the record ID and the date and time the record was last updated, joined by the separator you choose. Because the update time is part of the name, every update creates a new file instead of replacing the previous one.
Troubleshooting
-
The connection fails with a bucket URL error
The Specify Amazon S3 bucket URL field must use the formats3://bucketname/path. If you use MinIO or Cloudflare R2, enter the server URL in S3-compatible endpoint URL, not in the bucket URL field. -
The connection fails with a region error
A region is always required. For Cloudflare R2, enterauto. For MinIO, enter any value, such asus-east-1. -
The connection fails with an endpoint error for MinIO or Cloudflare R2
Both providers need an endpoint URL that starts withhttp://orhttps://. MinIO also requires Use path-style bucket addressing to be on. The connector doesn’t accept self-signed TLS certificates, so use a trusted certificate. -
The connection fails with an access denied error
The access key or secret key is wrong, or the key can’t list the bucket. When you connect, the app lists the bucket to test access, so the key needs permission to list and write objects. Update the keys or the bucket permissions, and then connect again. -
No files appear in the bucket
Every Push … to Amazon S3 setting is off by default. Turn on each record type you want to export, and check that Only push … created after isn’t set to a future date. Files are written every 5 minutes.
Need Help?
If you need additional assistance:
- Use Ask Pippen, our AI agent, located at the top of the platform page.
- Submit a support request with as much relevant detail as possible. Learn how to submit a request.
We’re here to help you succeed with your operations.
Comments
0 comments