How to Get Started with Kestra

What Is Kestra and Why Teams Are Switching in 2026

Kestra is a declarative workflow orchestration platform. Where traditional tools like Apache Airflow require you to write Python DAGs and manage a complex Python environment, Kestra lets you define workflows in YAML — a format that operations managers, analysts, and even product teams can read and edit without engineering support.

In March 2026, Kestra published data showing that over 80% of organizations using AI-driven data platforms were moving toward event-driven, YAML-first pipeline tools. The shift reflects a broader trend: data work is no longer the exclusive domain of data engineers. Analysts and ops teams now own pipelines that were previously delegated.

Kestra has over 1,200 built-in plugins covering databases, cloud storage, REST APIs, dbt, Airbyte, and more — which means most standard data pipeline tasks require no custom code at all.

Step 1: Install Kestra with Docker

You need Docker installed. If you don't have it, download Docker Desktop from docker.com and follow the standard installation for your operating system.

Once Docker is running, open a terminal and run:

docker run --pull=always --rm -it -p 8080:8080 --user=root \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp:/tmp kestra/kestra:latest server local

This pulls the latest Kestra image and starts the server locally. After about 30 seconds, open a browser and navigate to http://localhost:8080. You will see the Kestra UI with the flow editor, execution history, and plugin library.

For production deployments, Kestra offers a Docker Compose configuration and Kubernetes Helm chart. The local single-container setup is sufficient to build and test any workflow before moving to production.

Step 2: Create Your First Flow

In the Kestra UI, click Flows in the left sidebar, then + Create. You'll see a YAML editor. A flow has three required parts: a namespace, an id, and at least one task.

Here is a minimal working example that fetches data from a public API and logs the result:

id: fetch-daily-report
namespace: ops.data

tasks:
  - id: get_data
    type: io.kestra.plugin.core.http.Request
    uri: https://api.example.com/daily-summary
    method: GET

  - id: log_result
    type: io.kestra.plugin.core.log.Log
    message: "{{ outputs.get_data.body }}"

Click Save and then Execute to run it immediately. The execution panel shows real-time logs for each task, including inputs, outputs, duration, and any errors.

You can reference the output of one task in the next task using the {{ outputs.task_id.field }} syntax. Kestra handles the dependency chain automatically.

Step 3: Connect to a Database

Most pipelines read from or write to a database. Kestra has native plugins for PostgreSQL, MySQL, BigQuery, Snowflake, Redshift, and more.

To query a PostgreSQL database:

id: daily-sales-extract
namespace: ops.data

tasks:
  - id: query_sales
    type: io.kestra.plugin.jdbc.postgresql.Query
    url: jdbc:postgresql://your-host:5432/your-database
    username: "{{ secret('DB_USER') }}"
    password: "{{ secret('DB_PASS') }}"
    sql: |
      SELECT date, SUM(revenue) as total_revenue
      FROM sales
      WHERE date = CURRENT_DATE - 1
      GROUP BY date
    store: true

Secrets like database passwords are stored in Kestra's secrets manager (or an external vault), not hardcoded in the YAML. The store: true flag saves the query output as an internal storage file that downstream tasks can reference.

Step 4: Add a Schedule

A flow without a trigger runs only when you execute it manually. To run on a schedule, add a triggers block:

triggers:
  - id: daily_at_8am
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 8 * * *"

This runs the flow every day at 8 AM UTC. Cron expressions follow standard syntax. You can also trigger on S3 file uploads, webhook calls, another flow completing, or Kafka messages.

Kestra stores the execution history for every triggered run, so you can go back and inspect what ran, what the inputs were, and where it failed — without digging through log files.

Step 5: Handle Errors and Retries

Production pipelines fail. A database goes down, an API rate-limits you, a file arrives late. Kestra handles this with task-level retry configuration:

tasks:
  - id: fetch_from_api
    type: io.kestra.plugin.core.http.Request
    uri: https://api.example.com/data
    retry:
      type: exponential
      maxAttempts: 3
      initialDelay: PT30S
      maxDelay: PT5M

This retries up to 3 times with exponential backoff, starting at 30 seconds and capping at 5 minutes. You can also define errors tasks that run only when a pipeline fails — useful for sending a Slack alert or writing a failure record to a database.

Step 6: Use Namespaces to Organize Workflows

As you add more flows, namespaces keep things organized. Think of them like folders. A common structure:

ops.data       — data extraction and loading flows
ops.reports    — scheduled reporting flows
ops.alerts     — monitoring and alerting flows
marketing.etl  — marketing data pipelines

Kestra lets you set namespace-level variables and secrets, so flows within the same namespace share configuration without duplicating it.

Analyzing Pipeline Output

Once your flows are running and producing data, the next step is making sense of that output. If your pipeline exports CSV files or writes to a database, tools like VSLZ let you upload the output and query it in plain English — useful for non-technical stakeholders who need to see the results without writing SQL.

What to Build First

For teams moving off manual spreadsheet workflows or replacing ad-hoc scripts, a good starting point is a daily extract-load flow: query your operational database at a set time, write the output to a data warehouse or a shared CSV, and trigger a Slack notification when it completes. That single flow typically replaces several hours of weekly manual work and gives you a foundation to build more complex pipelines on top of.

Kestra's blueprint library, available in the UI under Blueprints, includes pre-built templates for dbt runs, Airbyte syncs, BigQuery loads, and API polling patterns. Start from a blueprint rather than writing from scratch.