Streaming Lines: Files.lines for Memory-Efficient Reading
What You'll Learn
You'll learn how to use Files.lines() to process large text files one line at a time without loading the entire file into memory.
The Memory Problem with readAllLines
When you use Files.readAllLines(), Java loads every line into a List<String> in memory before you can process any of them. For a 2 GB log file, that means consuming gigabytes of heap space. But what if you only need to process one line at a time?
Enter Files.lines: A Stream-Based Solution
Files.lines(Path path) returns a Stream<String> where each element is one line from the file. The magic? Lines are read lazily—only when you actually consume them from the stream. It's like reading a book one page at a time instead of photocopying the entire book first.
Think of it as a pipeline: the file stays on disk, and lines flow through your processing logic one at a time. Memory usage remains constant regardless of file size.
Basic Usage Pattern
Path logFile = Paths.get("application.log");
try (Stream<String> lines = Files.lines(logFile)) {
lines.filter(line -> line.contains("ERROR"))
.forEach(System.out::println);
}
Notice the try-with-resources: Files.lines() opens the file and returns a stream that must be closed to release the file handle. The stream is AutoCloseable, so it closes automatically.
Why This Matters
When processing a 10-million-line file, Files.lines() might use 100 MB of memory while readAllLines() could consume several gigabytes. You get the same results with drastically different resource requirements.
Key Takeaway: Use Files.lines() when working with large files—it returns a lazy Stream<String> that processes lines on-demand, keeping memory usage low and constant regardless of file size. Always close the stream with try-with-resources.