Airflow Operators and Tasks
What you'll learn: How to transform abstract DAG nodes into concrete, executable work using Airflow's built-in operators.
The Building Blocks of Work
In Airflow, operators are templates that define what work gets done, while tasks are specific instances of operators that actually execute that work in your DAG. Think of operators as cookie cutters (reusable shapes) and tasks as the actual cookies (unique instances you create).
When you define a DAG, you instantiate operators to create tasks. Each task represents a single unit of work: run a Python function, execute a bash command, query a database, etc.
Common Operators
PythonOperator
Executes any Python callable (function). Use this when your logic is written in Python.
from airflow.operators.python import PythonOperator
def my_function():
print("Processing data...")
return "Success"
task_python = PythonOperator(
task_id='run_python_function',
python_callable=my_function,
dag=my_dag
)
BashOperator
Runs shell commands or scripts. Perfect for system operations or calling external tools.
from airflow.operators.bash import BashOperator
task_bash = BashOperator(
task_id='run_bash_command',
bash_command='echo "Pipeline started"',
dag=my_dag
)
SQLExecuteQueryOperator
Executes SQL queries against databases (formerly SQLOperator in older versions).
from airflow.providers.common.sql.operators.sql import SQLExecuteQueryOperator
task_sql = SQLExecuteQueryOperator(
task_id='run_query',
sql='SELECT COUNT(*) FROM users',
conn_id='my_postgres_conn',
dag=my_dag
)
Each task gets a unique task_id that identifies it within the DAG. Operators handle retry logic, logging, and execution context automatically.
Key Takeaway: Operators are reusable templates that define task behavior; instantiating an operator creates a task that executes specific work—use PythonOperator for Python functions, BashOperator for shell commands, and SQLExecuteQueryOperator for database queries.