Course contentsShow
← Data Science
Lesson 1835 of 2,14543. Data Pipelines and OrchestrationFree lesson

Airflow Operators and Tasks

Common operators (PythonOperator, BashOperator, SQLOperator) and how to define executable tasks.

Airflow Operators and Tasks

What you'll learn: How to transform abstract DAG nodes into concrete, executable work using Airflow's built-in operators.

The Building Blocks of Work

In Airflow, operators are templates that define what work gets done, while tasks are specific instances of operators that actually execute that work in your DAG. Think of operators as cookie cutters (reusable shapes) and tasks as the actual cookies (unique instances you create).

When you define a DAG, you instantiate operators to create tasks. Each task represents a single unit of work: run a Python function, execute a bash command, query a database, etc.

Common Operators

PythonOperator

Executes any Python callable (function). Use this when your logic is written in Python.

from airflow.operators.python import PythonOperator

def my_function():
    print("Processing data...")
    return "Success"

task_python = PythonOperator(
    task_id='run_python_function',
    python_callable=my_function,
    dag=my_dag
)

BashOperator

Runs shell commands or scripts. Perfect for system operations or calling external tools.

from airflow.operators.bash import BashOperator

task_bash = BashOperator(
    task_id='run_bash_command',
    bash_command='echo "Pipeline started"',
    dag=my_dag
)

SQLExecuteQueryOperator

Executes SQL queries against databases (formerly SQLOperator in older versions).

from airflow.providers.common.sql.operators.sql import SQLExecuteQueryOperator

task_sql = SQLExecuteQueryOperator(
    task_id='run_query',
    sql='SELECT COUNT(*) FROM users',
    conn_id='my_postgres_conn',
    dag=my_dag
)

Each task gets a unique task_id that identifies it within the DAG. Operators handle retry logic, logging, and execution context automatically.

Key Takeaway: Operators are reusable templates that define task behavior; instantiating an operator creates a task that executes specific work—use PythonOperator for Python functions, BashOperator for shell commands, and SQLExecuteQueryOperator for database queries.