The GCP Cloud Storage connector lets you export operational data from Pipe17 directly to a Google Cloud Storage bucket. Use it to push entity data, including orders, inventory, fulfillments, and more, as structured files for downstream reporting, analytics, or archival.
Key capabilities:
- Push locations, orders, shipments, fulfillments, transfers, purchases, arrivals, receipts, products, inventory, and exceptions to a GCS bucket
- Configure a custom file path template to organize exported files by org, entity type, or date
- Mask personally identifiable information (PII) before data leaves Pipe17
- Generate POSIX-compatible file names for compatibility with Linux-based processing pipelines
- Force a full resync for any entity type on demand
Setup
Prerequisites:
- A Google Cloud project with Cloud Storage enabled
- A GCS bucket created and ready to receive data
- A GCP service account with the Storage Object Admin role on the target bucket
- A service account key file (JSON format) downloaded from the GCP Console
- Navigate to Connectors in the app and select Add connector.
- Select GCP Cloud Storage from the connector list.
- Enter the GCS bucket URL in the format
gs://bucketname/path/to/root. - Paste the contents of your service account key JSON into the Service Account Key field.
- Select a separator for file name components from the dropdown (Colon, Underscore, Dash, or Dot).
- Toggle Generate POSIX compatible file names on or off depending on your pipeline requirements.
- Enter a path template for how files should be organized in the bucket, or accept the default (
pipe17/{orgKey}/{entityType}). - Click Save to establish the connection.
Settings
Connection settings
| Setting | Description | Default |
|---|---|---|
| GCS bucket URL | The target bucket and root path. Format: gs://bucketname/path/to/root
|
— |
| Service account key | JSON credentials for the GCP service account used to write to the bucket | — |
| Separator for file name components | Character used between file name parts |
: (Colon) |
| Generate POSIX compatible file names | When on, file names use only POSIX-safe characters | Off |
| Use path template when posting files to GCP Cloud Storage | Controls folder structure inside the bucket. Allowed variables: {orgKey}, {entityType}, {entityDate}
|
pipe17/{orgKey}/{entityType} |
General settings
Personal identifiable information
| Setting | Description | Default |
|---|---|---|
| Mask PII in objects reported to GCP Cloud Storage | When on, PII such as names and addresses is masked with * in orders, shipments, and fulfillments |
Off |
Entity push settings
Each entity type below has three configurable fields:
- Push [entity] to GCP Cloud Storage - toggle on to enable outbound sync for that entity type
- Only push [entity] created after - date/time filter; only records created after this timestamp are pushed
- Force [entity] resync - toggle on to trigger a full resync of all records for that entity type
Entities with these settings: Locations, Orders, Shipping requests, Fulfillments, Transfers, Purchases, Arrivals, Receipts, Products, Inventory, Exceptions.
Flows
| Flow | Direction | Notes |
|---|---|---|
| Locations | Pipe17 → GCP Cloud Storage | Exports location records as files to the configured bucket |
| Orders | Pipe17 → GCP Cloud Storage | Exports order records; supports date filter and resync |
| Shipping requests | Pipe17 → GCP Cloud Storage | Exports shipment records; supports date filter and resync |
| Fulfillments | Pipe17 → GCP Cloud Storage | Exports fulfillment records; PII masking applies when enabled |
| Transfers | Pipe17 → GCP Cloud Storage | Exports transfer records; supports date filter and resync |
| Purchases | Pipe17 → GCP Cloud Storage | Exports purchase records; supports date filter and resync |
| Arrivals | Pipe17 → GCP Cloud Storage | Exports arrival records; supports date filter and resync |
| Receipts | Pipe17 → GCP Cloud Storage | Exports receipt records; supports date filter and resync |
| Products | Pipe17 → GCP Cloud Storage | Exports product catalog records; supports date filter and resync |
| Inventory | Pipe17 → GCP Cloud Storage | Exports inventory records; supports date filter and resync |
| Exceptions | Pipe17 → GCP Cloud Storage | Exports exception records; supports date filter and resync |
Troubleshooting
- Files are not appearing in the GCS bucket
The service account may lack write permissions on the bucket. Verify the service account has the Storage Object Admin role assigned in GCP IAM, then save the connector settings again to re-test the connection.
- The connector fails to authenticate
The service account key JSON may be malformed or expired. Download a fresh key from the GCP Console, paste the full JSON contents into the Service Account Key field, and save.
- Files are written to the wrong folder path
The path template may contain an unsupported variable or typo. Confirm the template uses only the allowed variables: {orgKey}, {entityType}, and {entityDate}, and that the bucket URL ends without a trailing slash.
- PII is visible in exported files despite masking being enabled
PII masking applies only to orders, shipments, and fulfillments. Records for other entity types are exported without masking by design. If masking is not taking effect on the expected entities, toggle the setting off, save, toggle it back on, and save again.
- A resync is not picking up older records
The Only push [entity] created after date filter takes precedence over a forced resync. If the date filter is set, records created before that date are excluded. Clear the date field to allow all records to sync, then enable Force [entity] resync.
Need Help?
If you need additional assistance:
- Use Ask Pippen, our AI agent, located at the top of the app page.
- Submit a support request with as much relevant detail as possible. Learn how to submit a request.
- For urgent issues, email us directly at support@pipe17.com.
We're here to help you succeed with your operations.
Comments
0 comments