Understanding the TokenStream API

You will safely bridge between raw input streams and higher-level AST representations.