pytype.test_data.tokenize

module documentation
(source)

Test case for functions with complex cfg.

Function generate_tokens Undocumented
Constant ASYNC Undocumented
Constant COMMENT Undocumented
Constant DEDENT Undocumented
Constant ENDMARKER Undocumented
Constant ERRORTOKEN Undocumented
Constant INDENT Undocumented
Constant NAME Undocumented
Constant NEWLINE Undocumented
Constant NL Undocumented
Constant NUMBER Undocumented
Constant OP Undocumented
Constant STRING Undocumented
Variable endprogs Undocumented
Variable pseudoprog Undocumented
Variable single_quoted Undocumented
Variable tabsize Undocumented
Variable triple_quoted Undocumented
def generate_tokens(readline): (source)

Undocumented

ASYNC: str = (source)

Undocumented

Value
'...'
COMMENT: str = (source)

Undocumented

Value
'...'
DEDENT: str = (source)

Undocumented

Value
'...'
ENDMARKER: str = (source)

Undocumented

Value
'...'
ERRORTOKEN: str = (source)

Undocumented

Value
'...'
INDENT: str = (source)

Undocumented

Value
'...'
NAME: str = (source)

Undocumented

Value
'...'
NEWLINE: str = (source)

Undocumented

Value
'...'
NL: str = (source)

Undocumented

Value
'...'
NUMBER: str = (source)

Undocumented

Value
'...'
OP: str = (source)

Undocumented

Value
'...'
STRING: str = (source)

Undocumented

Value
'...'
endprogs = (source)

Undocumented

pseudoprog = (source)

Undocumented

single_quoted: str = (source)

Undocumented

tabsize: int = (source)

Undocumented

triple_quoted: str = (source)

Undocumented