Text::TokenStream::Role::Stream - role for token-stream classes
# In some kind of parser class:
has stream => (
is => 'ro',
handles => 'Text::TokenStream::Role::Stream',
This role requires and/or implements methods that provide the machinery for scanning an input string into tokens. It exists as a role so that a parser class can easily delegate those methods to a stream instance.
Takes no arguments. Returns a list of all remaining tokens found in the input.
Takes a single argument indicating a token to match, as with Text::TokenStream::Token#matches. Scans through the input until it finds a token that matches the argument, and returns a list of all tokens before the matching one. If no remaining token in the input matches the argument, behaves as "collect_all".
Should take no arguments, and return the current input position.
Should take any number of arguments, and throw an exception that reports an error at the current position.
Should take a non-negative integer argument, and fill the internal buffer with that many tokens (or as many as are available), and return true iff that succeeded.
Should take any number of arguments for Text::TokenStream::Token#matches, fill the internal buffer with the right number of elements (returning false if there aren't enough), and return true if each token is matched by the corresponding argument.
Should take no elements, and return the next token (or undef if no more elements are available), advancing the current position.
Should take an argument for Text::TokenStream::Token#matches, ensure that there is at least one token remaining and that it matches that argument (reporting an error if not), and return that token, advancing the current position.
Should take no elements, and return the next token if one exists, without advancing the current position.
Should take an argument for Text::TokenStream::Token#matches, and if there's another token that matches it, advance past that token, and return true; otherwise, it should return false.
Should take a token, and any number of arguments, and throw an exception that reports an error at the position of the token.
Aaron Crane, <firstname.lastname@example.org>
Copyright 2021 Aaron Crane.
This library is free software and may be distributed under the same terms as perl itself. See http://dev.perl.org/licenses/.
To install Text::TokenStream, copy and paste the appropriate command in to your terminal.
perl -MCPAN -e shell
For more information on module installation, please visit the detailed CPAN module installation guide.