Reading and Writing Data Files
What You'll Learn
You'll learn how to load data from common file formats (CSV, JSON, Excel) into Pandas DataFrames and save your processed results—essential skills for starting any ML project.
Why This Matters for ML
Every machine learning workflow begins with data, and that data rarely lives directly in your Python code. Real-world datasets come as files: customer records in CSV files, sensor data in JSON format, or business metrics in Excel spreadsheets. Before you can train models, you need to get this data into a format NumPy and ML libraries can work with—typically a Pandas DataFrame.
Think of this step as opening a book before you can read it. The data exists, but it's locked in a file format until you load it into memory.
Loading Data into DataFrames
CSV Files (comma-separated values) are the most common format:
import pandas as pd
# Load a CSV file
df = pd.read_csv('customer_data.csv')
JSON Files contain structured data with nested fields:
df = pd.read_json('sensor_readings.json')
Excel Files may have multiple sheets:
df = pd.read_excel('sales_report.xlsx', sheet_name='Q1')
Saving Processed Data
After cleaning and transforming your data, save it for your ML pipeline:
# Save to CSV (most common for ML)
df.to_csv('cleaned_data.csv', index=False)
# Save to JSON
df.to_json('processed_data.json')
# Save to Excel
df.to_excel('final_dataset.xlsx', index=False)
The index=False parameter prevents Pandas from adding an extra index column to your output file.
Key Takeaway: Loading data with read_csv(), read_json(), and read_excel() transforms external files into DataFrames you can manipulate, while to_csv() and similar methods save your processed results for ML workflows.