class documentation
This lexer takes one lexer as argument, the lexer for the language being formatted, and the left and right delimiters for escaped text. First everything is scanned using the language lexer to obtain strings and comments. All other consecutive tokens are merged and the resulting text is scanned for escaped segments, which are given the Token.Escape type. Finally text that is not escaped is scanned again with the language lexer.
Method | __init__ |
Undocumented |
Method | get |
Return an iterable of (index, tokentype, value) pairs where "index" is the starting position of the token within the input text. |
Instance Variable | lang |
Undocumented |
Instance Variable | left |
Undocumented |
Instance Variable | right |
Undocumented |
Method | _filter |
Keep only the tokens that match `pred`, merge the others together |
Method | _find |
Find escape tokens within text, give token=None otherwise |
Method | _find |
find escape tokens that are not in strings or comments |
Inherited from Lexer
:
Method | __repr__ |
Undocumented |
Method | add |
Add a new stream filter to this lexer. |
Method | analyse |
Has to return a float between ``0`` and ``1`` that indicates if a lexer wants to highlight this text. Used by ``guess_lexer``. If this method returns ``0`` it won't highlight it in any case, if it returns ``1`` highlighting with this lexer is guaranteed. |
Method | get |
Return an iterable of (tokentype, value) pairs generated from `text`. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined. |
Class Variable | alias |
Undocumented |
Class Variable | aliases |
Undocumented |
Class Variable | filenames |
Undocumented |
Class Variable | mimetypes |
Undocumented |
Class Variable | name |
Undocumented |
Class Variable | priority |
Undocumented |
Class Variable | url |
Undocumented |
Instance Variable | encoding |
Undocumented |
Instance Variable | ensurenl |
Undocumented |
Instance Variable | filters |
Undocumented |
Instance Variable | options |
Undocumented |
Instance Variable | stripall |
Undocumented |
Instance Variable | stripnl |
Undocumented |
Instance Variable | tabsize |
Undocumented |
Return an iterable of (index, tokentype, value) pairs where "index" is the starting position of the token within the input text. In subclasses, implement this method as a generator to maximize effectiveness.