Best Practices

Pipelines and Pipeline files

  1. Pipelines should have the same name as the pipeline file.
    ❌Dont:
    • pipline name:

      • product_abstract

    • file name:

      • product.xml

    ✅Do:

    • pipline name: product_abstract

    • file name:

      • product_abstract.xml

  2. Pipeline files with multiple pipelines should be named by the common prefix of the pipeline names.
    ❌Dont:
    • pipelines:

      • product_abstract_DE

      • product_abstract_EN

      • product_abstract_FR

    • file name:

      • product_by_stores.xml

    ✅Do:
    • pipelines:

      • product_abstract_DE

      • product_abstract_EN

      • product_abstract_FR

    • file name:

      • product_abstract.xml

  3. There should not be multiple pipelines in a single pipeline file that are not related to each other.
    ❌Dont:
    • pipelines:

      • product_abstract_DE

      • product_abstract_EN

      • product_abstract_FR

      • product_detail_DE

      • product_detail_EN

      • product_detail_FR

    • file name:

      • product.xml

    ✅Do:
    • pipelines:

      • product_abstract_DE

      • product_abstract_EN

      • product_abstract_FR

    • file name:

      • product_abstract.xml

    • pipelines:

      • product_detail_DE

      • product_detail_EN

      • product_detail_FR

    • file name:

      • product_detail.xml

Transformer naming conventions

Transformers are sometimes used to load data from a database or other sources and transform it into a format that can be used by the pipeline. The naming of these transformers should reflect the source of the data and the type of data that is being transformed.

  1. Transformers that load data from a database should be prefixed by load-.
    ❌Dont:

    • <product />

    ✅Do:

    • <load-product />

  2. Transformers that read data from the configuration or similar sources should be prefixed by read-.
    ❌Dont:

    • <config-key-reader />

    ✅Do:

    • <read-config />

  3. Transformers that have a source field name attribute should name the source field attribute from
    ❌Dont:

    • <load-product sourceFieldName=”product_id” />

    • <load-product source=”product_id” />

    ✅Do:

    • <load-product from=”product_id” />

  4. Transformers that have a target field name attribute should name the target field attribute to
    ❌Dont:

    • <load-product targetFieldName=”product” />

    • <load-product target=”product” />

    ✅Do:

    • <load-product to=”product” />