Common Data Patterns in Pipe17

Canonical Data Model

Pipe17 uses a canonical data model approach, which is one of its most fundamental patterns:

Key Characteristics:

• Standardized Schema: Pipe17 maintains a consistent internal data structure (canonical schema) for all entities regardless of their source.

• Schema Mapping Layer: Data from external systems is transformed to and from this canonical model using schema mappings.

• Consistent Processing: This approach allows Pipe17's internal engines (Automation Engine, Order Routing Engine) to work with standardized data regardless of the source or destination systems.

Order Lifecycle Pattern

Orders follow a well-defined state transition pattern through the system:

Primary Order States

new → onHold → readyForFulfillment → sentToFulfillment → fulfilled

Additional States

• draft: Incomplete orders requiring additional data

• reviewRequired: Orders needing manual review

• partialFulfillment: Some items fulfilled, others pending

• canceled: Order processing stopped

• archived: Order removed from active view

Routing Status Pattern

Routing status operates alongside order status to track fulfillment routing progress.

Values:
pending, ready, acknowledged, failed, disabled, excluded, pendingInventory

Order Disposition Pattern

Pipe17 uses dispositions to track the status of each line item within an order.

Common Disposition Values:
routed, sent, fulfilled, canceled, failed, rejected, unrouted, unsent, unfulfilled, and partial variants such as partiallySent and partiallyFulfilled.

Entity Relationship Patterns

Pipe17 maintains consistent relationships across entities for multi-tenant and order-centric processing.

Order-Centric Flow:
Orders → Shipping Requests → Fulfillments

• Each order can generate multiple shipping requests (for split shipments).

• Each shipping request can generate multiple fulfillments (for partial shipments).

Multi-Tenant Isolation:
All entities include orgKey, integration, userId, and apikeyId fields for tenant and access isolation.

External References:
Entities include extOrderId, extProductId, and similar fields for mapping to external systems.

Soft Delete Pattern:
Entities use status: 'deleted' instead of physical deletion for audit consistency.

Disposition Relationships:
Order line-item dispositions link to shipments and fulfillments, supporting granular tracking.

Integration Data Flow Patterns

Data flows through Pipe17 in consistent patterns:

Inbound Flow:

1. External system data is pulled via connector

2. Schema mapping transforms external data to canonical format

3. Data is stored in Pipe17's database

4. Automation Engine processes the data based on rules

5. Order Routing Engine determines fulfillment location

Outbound Flow:

1. Canonical data is retrieved from Pipe17's database

2. Schema mapping transforms canonical data to external system format

3. Data is pushed to external system via connector

Schema Mapping Patterns

Pipe17 uses several common patterns for transforming data between systems:

Direct Field Mapping:

{
  "name": "Map Order Number",
  "enabled": true,
  "source": "external.order_number",
  "target": "p17.order.orderId"
}

Default Value Pattern:

{
  "name": "Set Default Sales Channel",
  "enabled": true,
  "source": "external.sales_channel",
  "target": "p17.order.channel",
  "default": "Web"
}

Status Translation Pattern:

{
  "name": "Map Fulfillment Status",
  "enabled": true,
  "source": "external.fulfillment_status",
  "target": "p17.order.fulfillmentStatus",
  "lookup": {
    "pending": "unfulfilled",
    "partial": "partially_fulfilled",
    "fulfilled": "fulfilled"
  }
}

Event-Driven Architecture

Pipe17 operates on an event-driven model to manage real-time updates.

Event Fields:

• direction: in | out

• source: system origin

• timestamp: event occurrence time

• entityType, entityId: context references

• keywords: searchable tags

• requestId: correlation key for traceability

• event_compression_enabled: supports event compression

Event Status Flow:
completed → acknowledged → failed

Webhook Integration:
Events can publish to subscribed webhook topics for downstream processing.

Exception Handling Pattern

Pipe17 uses structured exception tracking to flag issues requiring manual or automated resolution.

Key Points:

• Exceptions link directly to specific entities (orders, fulfillments, etc.).

• Categorized by type (e.g., validation, communication).

• Can be resolved manually or through automation rules.

Product Type Patterns

Products in Pipe17 support several standardized types.

Product Types:
simple, bundle, bundleItem, child, parent

Relationships:

• parentExtProductId defines parent-child linkage.

• Bundles: Defined by a bundleItems array containing sku and quantity for each component.

• Status Enum: Products can be active or inactive.

Inventory Event Patterns

Pipe17 uses a ledger-based inventory system with granular tracking.

Inventory Events:
ingest, adjust, ship, fulfill, xferout, xferfulfill, xferin, futureShip, receive, release

Inventory States:
onHand, committed, available, incoming, commitShip, commitXfer, unavailable, future, committedFuture, damaged, hold

Ledger Pattern:
Inventory updates are recorded immutably in a ledger to ensure traceability.

Inventory Type Field:
ptype: Indicates inventory origin (wms, ims, oms, portal).

Shipment Status Pattern

Shipments follow a defined status model:

new → pendingInventory → pendingShippingLabel → reviewRequired → readyForFulfillment → sentToFulfillment → fulfilled → partialFulfillment → canceled → canceledRestock → failed → onHold

Pick Status Pattern

Tracks warehouse pick operations:

onHold → readyToPick → picking → picked → packing → packed → readyToLoad → partiallyShipped → shipped → canceled

Fulfillment Type Pattern

Fulfillment actions use:
accepted, rejected
Mapped to corresponding disposition outcomes.

Purchase Order Pattern

Purchase orders follow these states:
new → draft → pending → paid → reported

Data Transformation Patterns

Pipe17 supports transformation functions for data consistency and logic.

Helper Functions and Examples:

• $N(): Convert to number.
  Example: "price": "$N(source.price)"

• $S(): Convert to string.
  Example: "sku": "$S(source.sku)"

• $D(): Provide default value.
  Example: "channel": "$D(source.channel, 'Web')"

• $M(): Handle missing values gracefully.
  Example: "description": "$M(source.desc)"

• _: Use Lodash for advanced array or object transformations.

Conditional and Array Mapping:

{
  "name": "lineItems[].sku",
  "enabled": true,
  "source": "line_items[$0].sku",
  "target": "lineItems[$0].sku"
}

Use conditional expressions to apply logic during transformation (e.g., conditional price formatting or field selection).

Automation Engine Integration

The autoEngUpdateStatus field enables the Automation Engine to track rule-based workflow execution.

API Versioning Pattern

Every document includes:

• apiVersion: Version identifier for compatibility.

• documentType: Defines the data category for parsing and validation.

Practical Applications

Understanding these patterns helps in several ways:

1. Integration Configuration: Knowing how data flows through the system helps when setting up new integrations

2. Troubleshooting: Recognizing normal patterns makes it easier to identify when something is wrong

3. Custom Mapping: Understanding mapping patterns enables effective customization for specific business needs

4. Order Management: Knowledge of the order lifecycle helps track orders through the system

5. Inventory Management: Understanding inventory patterns helps maintain accurate stock levels

Need Help?

If you need additional assistance:

• Use Ask Pippen, our AI agent, located at the top of the app page.

• Submit a support request with as much relevant detail as possible. Learn how to submit a request.

• For urgent issues, email us directly at support@pipe17.com.

We're here to help you succeed with your operations.