Text::TokenStream::Role::Stream - role for token-stream classes
# In some kind of parser class: has stream => ( is => 'ro', handles => 'Text::TokenStream::Role::Stream', );
This role requires and/or implements methods that provide the machinery for scanning an input string into tokens. It exists as a role so that a parser class can easily delegate those methods to a stream instance.
See Text::TokenStream.
collect_all
Takes no arguments. Returns a list of all remaining tokens found in the input.
collect_upto
Takes a single argument indicating a token to match, as with Text::TokenStream::Token#matches. Scans through the input until it finds a token that matches the argument, and returns a list of all tokens before the matching one. If no remaining token in the input matches the argument, behaves as "collect_all".
Text::TokenStream::Token#matches
"collect_all"
current_position
Should take no arguments, and return the current input position.
err
Should take any number of arguments, and throw an exception that reports an error at the current position.
fill
Should take a non-negative integer argument, and fill the internal buffer with that many tokens (or as many as are available), and return true iff that succeeded.
looking_at
Should take any number of arguments for Text::TokenStream::Token#matches, fill the internal buffer with the right number of elements (returning false if there aren't enough), and return true if each token is matched by the corresponding argument.
next
Should take no elements, and return the next token (or undef if no more elements are available), advancing the current position.
next_of
Should take an argument for Text::TokenStream::Token#matches, ensure that there is at least one token remaining and that it matches that argument (reporting an error if not), and return that token, advancing the current position.
peek
Should take no elements, and return the next token if one exists, without advancing the current position.
skip_optional
Should take an argument for Text::TokenStream::Token#matches, and if there's another token that matches it, advance past that token, and return true; otherwise, it should return false.
token_err
Should take a token, and any number of arguments, and throw an exception that reports an error at the position of the token.
Aaron Crane, <arc@cpan.org>
Copyright 2021 Aaron Crane.
This library is free software and may be distributed under the same terms as perl itself. See http://dev.perl.org/licenses/.
To install Text::TokenStream, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Text::TokenStream
CPAN shell
perl -MCPAN -e shell install Text::TokenStream
For more information on module installation, please visit the detailed CPAN module installation guide.