The constructor takes all tokens so that looking ahead for specific tokens can allow support for lambdas and other complex language features.
See Implementation
The constructor takes all tokens so that looking ahead for specific tokens can allow support for lambdas and other complex language features.