Qeasy Custom Data Processing Factory: Five Hook Templates and Practical Guide

Feature Overview

We designed this capability with one goal in mind: to provide a programmable, hot-pluggable extension point beyond the standardized integration pipeline. The Qeasy Data Integration Platform (DataHub) exposes "factory" hooks at every critical stage of the data flow. Developers only need to write a lightweight PHP class to rewrite, supplement, or split response data and request parameters.

The platform currently ships with five built-in hooks that cover the full lifecycle from source to target:

• BeforeSourceJobInsert: Pre-processing before the source platform API call, used to transform request parameters before they are persisted.
• AfterSourceInvoke: Runs after the source API returns, used to clean, merge, and reshape the raw response.
• AfterTargetGenerate: Runs after the target queue is generated, used to enrich the request payload with aggregated fields.
• BeforeTargetInvoke: Runs before the target API call, used to trigger reverse auditing, side-effects, or logging.
• AfterTargetInvoke: Runs after the target API returns, used to write receipt IDs back to the original queue document.

Every hook follows a fixed shape of "constructor + run()". The response or request is passed by reference, so any modification inside the factory takes effect immediately—no service restart required.

Usage Scenarios

The typical use case is: when standard mapping cannot satisfy business-specific field enrichment or state linkage rules. Here are several common real-world requirements we have seen:

1. Shipping fee converted into a ledger entry: The source outbound order contains a post_fee field, but the target ERP does not recognize free-floating fields. You need to append a ledger entry with spec_code=2222, goods_count=1, and price equal to the shipping fee. BeforeSourceJobInsert is the right hook for this.
2. Splitting an assembly order: The source assembly order's stockAssembleDetailsDto contains both finished products and raw materials, but the business needs them split into two arrays, product and material. This reshuffle belongs in AfterSourceInvoke.
3. Aggregating order paid amount: The target queue carries only line items; you need to compute price * num and write the sum to a paid field. Use AfterTargetGenerate to finish the calculation before the queue is committed.
4. Reverse auditing before submission: Some ERP platforms require a reverse-audit call before pushing a new order, to avoid document occupation. BeforeTargetInvoke can trigger this prerequisite action via the adapter SDK.
5. Receipt ID write-back: After the target platform returns a result, you need to write the receipt ID back to the original queue document for reconciliation. AfterTargetInvoke accomplishes this via DataStorage.

Configuration

Factories are deployed as standalone PHP class files in the platform-defined extension directory, and the file name must exactly match the class name. Once uploaded, developers attach the factory to the corresponding stage of an integration strategy through the "Factory" panel.

A standard factory template consists of three elements:

• Constructor: Receives reference parameters (&$response or &$request), the adapter instance $adapter, and—in some hooks—$job (queue task object) or $ids (list of MongoDB ObjectIds). The reference semantics are key: modifications made inside the factory are reflected in the upstream and downstream arrays.
• run() method: The entry point of the factory. The platform calls it automatically when the corresponding stage is reached.
• Early-exit mechanism: Most run() implementations begin with a check such as if ($response['code'] != 200), then return directly on failure to avoid polluting downstream data.

For example, BeforeSourceJobInsert iterates through the source response's order array and appends a ledger entry to details_list based on business conditions. AfterSourceInvoke leverages the reference nature of &$response to reshape every element in result.data. BeforeTargetInvoke can additionally call other action interfaces of the target platform through $this->adapter->SDK->invoke(...) and log key events via $this->adapter->getLogStorage()->insertOne(...) for troubleshooting.

Once attached, the platform will automatically instantiate the class, inject the parameters, and execute run() whenever a task reaches the corresponding stage—no manual intervention needed.

Caveats

1. Preserve reference semantics: All parameters in the constructor must use the & reference symbol; otherwise, modifications inside the factory will not propagate.
2. Class name must match file name: The platform loads factories via its autoloader. A mismatch will result in a Class Not Found error.
3. Avoid long-running operations: run() executes synchronously within the dispatch thread. External HTTP calls or blocking operations will slow down the entire integration pipeline. Move heavy work to a separate queue when necessary.
4. Fail fast to avoid dirty data: Always validate the response code or success flag at the beginning of run() and return immediately on failure.
5. Be precise with write-back targets: When using _id for write-back in AfterTargetInvoke, always reference IDs from the task, such as $this->job->ids[0]. Hard-coded IDs risk corrupting other records.
6. Logging and observability: For complex factories, we recommend using $adapter->getLogStorage() to record key checkpoints for auditing and troubleshooting.
7. PHP version compatibility: Factories run inside the platform's managed PHP runtime. Avoid extensions that are not enabled in the host environment.

With the Custom Data Processing Factory, business teams can sink logic that "standard mapping tables cannot accommodate" into native platform hooks. This preserves the stability of the Qeasy Data Integration Platform (DataHub) main pipeline while delivering the flexibility of in-house scripts.