Propel Extractor¶
Extracts structured data from a Propel ORM query and transforms it into rows for ETL processing. Supports bulk processing and integration with pipeline configurations.
Configuration¶
Name |
Type |
Description |
|---|---|---|
|
Attribute |
Fully qualified class name of the Propel query to execute |
|
Attribute |
Number of rows extracted at once (default: 200) |
Example¶
Input data¶
N/A (Data is dynamically retrieved from the database.)
Pipeline configuration¶
<propel-extractor propelQueryName="Orm\Zed\Glossary\Persistence\SpyGlossaryKeyQuery" bulkSize="200" />
Output data¶
id |
name |
price |
|---|---|---|
1 |
Product A |
10.99 |
2 |
Product B |
15.50 |
Error Handling¶
Error |
Description |
|---|---|
Invalid Propel Query Class |
Throws an exception: “Invalid propel query” if the query class is incorrect. |
Empty Result Set |
Returns an empty set of rows when no records are found. |
Bulk Processing Issue |
Ensures correct batch sizes based on |
Performance Considerations¶
Consideration |
Description |
|---|---|
Use an appropriate ``bulkSize`` |
Small sizes improve memory efficiency but increase processing time. |
Ensure Propel Query is Correct |
The specified class must be a valid |
Optimize Database Query |
Indexes and query optimizations can improve extraction performance. |
Key Features¶
Feature |
Description |
|---|---|
Supports Bulk Extraction |
Processes data in chunks. |
Handles Propel Validation |
Ensures the query is valid before execution. |
Optimized for Large Datasets |
Uses batch processing for efficiency. |
Pipeline Integration |
Compatible with pipeline configurations and console commands. |