loafer is an AI agent designed to automate the ETL (Extract, Transform, Load) process, enabling data engineers to efficiently transform and load data into their target systems.

Loafer

A CLI-first, declarative ETL/ELT pipeline tool driven by YAML.

Overview

Modern data pipelines often become bogged down by repetitive boilerplate and fragile scripts. Loafer solves this by treating data movement and transformation as configuration.

Loafer is a CLI-first tool that allows you to extract data from databases, files, and APIs, apply powerful transformations (both custom and AI-generated), and load the results into target systems — all configured within a single, highly readable YAML file.

Key Idea: Describe your pipeline in YAML, run it from the CLI, and let Loafer handle the execution.

Features

• Declarative Configuration: Define your inputs, outputs, and processing logic in simple YAML. No complex workflow orchestrators required.
• ETL & ELT Support: Choose between Extract-Transform-Load (in-memory/streaming Python transformations) or Extract-Load-Transform (in-database SQL executions).
• AI-Powered Transformations: Optionally use natural language to auto-generate complex Python transforms or target-database SQL using modern LLMs (Gemini, OpenAI, Gemini).
• Streaming & Batch Processing: Automatically switches to memory-efficient streaming for large datasets based on configurable thresholds.
• Data Quality Guards: Built-in validation steps to catch malformed data or high null-rates before they hit your target.
• Developer-Friendly & Extensible: Use custom Python files for advanced transformations when declarative logic isn't enough.

Installation

Via pip (PyPI)

The easiest way to install Loafer is via pip:

pip install loafer-etl

Via Docker (GHCR)

Loafer is officially published to the GitHub Container Registry. Available tags include specific versions (e.g. 0.2.0, 0.2) and latest.

docker pull ghcr.io/lupppig/loafer:latest

To run Loafer using Docker, mount your current working directory so Loafer can access your configuration and local files:

docker ru