How to write a new loader plugin

In this tutorial, we will show you how to write a new loader plugin for the Springbok framework.

The loader plugin is responsible for writing data to a file, database, or any other storage medium.

We will create a new loader that writes a “special” file format. The file format will be a plain text file, but formatted in “pizza” style.

The loader will write the file in the following format:

Pizza: Margherita Pizza
Description: A classic Neapolitan pizza with tomato sauce, mozzarella cheese, and fresh basil.

Pizza: Pepperoni Pizza
Description: A classic American pizza topped with tomato sauce, mozzarella cheese, and pepperoni slices.

Pizza: Cheese Pizza
Description: A simple pizza with just tomato sauce and mozzarella cheese.

Pizza: Chicago Deep Dish Pizza
Description: A deep-dish pizza with a thick, buttery crust, tomato sauce, and various toppings.

The loader will be used in the pipeline as follows:

<pizza-style
    outputFileName="data/output/pizza.txt"
    pizzaNameField="name"
    pizzaDescriptionField="description"
/>

Step 1: Create a new loader

First we need to create a new loader that will write the “pizza” file format.

Create a new class called “PizzaLoader” that extends the “AbstractLoader” class.

<?php

namespace Pyz\Zed\TextPizza\Business\Loader;

use Antiloop\Zed\SpringbokCommon\Business\Loader\AbstractLoader;
use Flow\ETL\Loader;
use Flow\ETL\Rows;

class PizzaStyleLoader extends AbstractLoader implements Loader
{
    /**
     * @var string
     */
    private $outputFileName;

    /**
     * @var mixed
     */
    protected mixed $outputFile;

    /**
     * @var bool
     */
    protected bool $fileNotOpened = true;

    /**
     * @var string
     */
    private $pizzaNameField;

    /**
     * @var string
     */
    private $pizzaDescriptionField;

    /**
     * @param string $outputFileName
     * @param string $pizzaNameField
     * @param string $pizzaDescriptionField
     */
    public function __construct(
        string $outputFileName,
        string $pizzaNameField,
        string $pizzaDescriptionField
    ) {
        $this->outputFileName = $outputFileName;
        $this->pizzaNameField = $pizzaNameField;
        $this->pizzaDescriptionField = $pizzaDescriptionField;
    }

    /**
     * @param \Flow\ETL\Rows $rows
     *
     * @return void
     */
    public function load(Rows $rows): void
    {
        if ($this->fileNotOpened) {
            $this->outputFile = fopen($this->outputFileName, 'w');
            $this->fileNotOpened = false;
        }

        foreach ($rows as $row) {
            $name = $row->valueOf($this->pizzaNameField);
            $description = $row->valueOf($this->pizzaDescriptionField);

            fwrite($this->outputFile, sprintf('Pizza: %s', $name) . PHP_EOL);
            fwrite($this->outputFile, sprintf('Description: %s', $description) . PHP_EOL);
            fwrite($this->outputFile, PHP_EOL);
        }
    }
}

Step 2: Add the loader to the factory

Next, we need to add the loader to the factory so that we can instantiate it.

<?php

namespace Pyz\Zed\TextPizza\Communication;

use Flow\ETL\Loader;
use Pyz\Zed\TextPizza\Business\Loader\PizzaStyleLoader;
use Spryker\Zed\Kernel\Communication\AbstractCommunicationFactory;

class TextPizzaCommunicationFactory extends AbstractCommunicationFactory
{
    /**
     * @param string $outputFileName
     * @param string $pizzaNameField
     * @param string $pizzaDescriptionField
     *
     * @return \Flow\ETL\Loader
     */
    public function createPizzaStyleLoader(
        string $outputFileName,
        string $pizzaNameField,
        string $pizzaDescriptionField
    ): Loader {
        return new PizzaStyleLoader(
            $outputFileName,
            $pizzaNameField,
            $pizzaDescriptionField,
        );
    }
}

Step 3: Create a new loader plugin

Next, we need to create a new loader plugin that will allow us to use the loader in a pipeline.

Create a new class called “PizzaStyleLoaderPlugin” that extends the “AbstractLoaderPlugin” class.

<?php

namespace Pyz\Zed\TextPizza\Communication\Plugins;

use Antiloop\Zed\SpringbokCommon\Communication\Plugins\Loaders\AbstractLoaderPlugin;
use Flow\ETL\Loader;

/**
* @method \Pyz\Zed\TextPizza\Communication\TextPizzaCommunicationFactory getFactory()
*/
class PizzaStyleLoaderPlugin extends AbstractLoaderPlugin
{
    /**
     * @var string
     */
    private const NAME = 'pizza-style';

    /**
     * @return string
     */
    public function getName(): string
    {
        return self::NAME;
    }

    /**
     * @param array $xml
     *
     * @return \Flow\ETL\Loader
     */
    public function build(array $xml): Loader
    {
        $outputFileName = $this->getAttribute($xml, 'outputFileName');
        $pizzaNameField = $this->getAttribute($xml, 'pizzaNameField');
        $pizzaDescriptionField = $this->getAttribute($xml, 'pizzaDescriptionField');

        return $this->getFactory()
            ->createPizzaStyleLoader(
                $outputFileName,
                $pizzaNameField,
                $pizzaDescriptionField,
            );
    }
}

Step 4: Add the plugin to the dependency provider

Next, we need to add the plugin to the dependency provider so that Springbok can load it.

<?php

namespace Pyz\Zed\Springbok;

use Antiloop\Zed\Springbok\SpringbokDependencyProvider as SpringbokSpringbokDependencyProvider;
use Pyz\Zed\TextPizza\Communication\Plugins\PizzaStyleLoaderPlugin;

class SpringbokDependencyProvider extends SpringbokSpringbokDependencyProvider

    /**
     * @return array<\Antiloop\Zed\Springbok\Dependency\Plugin\LoaderPluginInterface>
     */
    protected function getLoaderPlugins(): array
    {
        return [
            new PizzaStyleLoaderPlugin(),
        ];
    }
}

Step 5: Use the loader in a pipeline

Now you can use the plugin in your pipeline configuration

<pizza-style
    outputFileName="data/output/pizza.txt"
    pizzaNameField="name"
    pizzaDescriptionField="description"
/>

What we have done in this tutorial?

  • We created a new loader that writes a “pizza” file format.

  • We added the loader to the factory so that we can instantiate.

  • We created a new loader plugin that will allow us to use the transformer in a pipeline.

  • We added the plugin to the dependency provider so that it can be used in the pipeline.

Now you can use the plugin in your pipeline configuration to extract lines from a plain text file. And this concludes the tutorial on how to write a new loader plugin for the Springbok framework.

Further reading: