class documentation

Undocumented

Method __init__ Undocumented
Method _tokenize Undocumented
Class Variable _import_settings Undocumented
Class Variable _keyword_settings Undocumented
Class Variable _other_settings Undocumented
Class Variable _tokens Undocumented
Instance Variable _custom_tokenizer Undocumented
Instance Variable _template_setter Undocumented

Inherited from Tokenizer:

Method tokenize Undocumented
Method _is_assign Undocumented
Instance Variable _index Undocumented
def __init__(self, template_setter=None): (source)
_import_settings: tuple[str, ...] = (source)
_keyword_settings: tuple[str, ...] = (source)
_other_settings: tuple[str, ...] = (source)
_custom_tokenizer = (source)

Undocumented

_template_setter = (source)

Undocumented