pylint.checkers.unicode

module documentation
(source)

Unicode and some other ASCII characters can be used to create programs that run much different compared to what a human reader would expect from them. PEP 672 lists some examples. See: https://www.python.org/dev/peps/pep-0672/ The following checkers are intended to make users are aware of these issues.

Class UnicodeChecker Check characters that could be used to hide bad code to humans.
Function extract_codec_from_bom Try to extract the codec (unicode only) by checking for the BOM.
Function register Undocumented
Constant BAD_ASCII_SEARCH_DICT Undocumented
Constant BAD_CHARS Undocumented
Constant BIDI_UNICODE Undocumented
Constant BOM_SORTED_TO_CODEC Undocumented
Constant UNICODE_BOMS Undocumented
Constant UTF_NAME_REGEX_COMPILED Undocumented
Class _BadChar Representation of an ASCII char considered bad.
Function _byte_to_str_length Return how many byte are usually(!) a character point.
Function _cached_encode_search A cached version of encode used for search pattern.
Function _encode_without_bom Encode a string but remove the BOM.
Function _fix_utf16_32_line_stream Handle line ending for UTF16 and UTF32 correctly.
Function _line_length Get the length of a string like line as displayed in an editor.
Function _map_positions_to_result Get all occurrences of search dict keys within line.
Function _normalize_codec_name Make sure the codec name is always given as defined in the BOM dict.
Function _remove_bom Remove the bom if given from a line.
Type Variable _StrLike Undocumented
def extract_codec_from_bom(first_line: bytes) -> str: (source)

Try to extract the codec (unicode only) by checking for the BOM. For details about BOM see https://unicode.org/faq/utf_bom.html#BOM Args: first_line: the first line of a file Returns: a codec name Raises: ValueError: if no codec was found

def register(linter: pylint.lint.PyLinter): (source)

Undocumented

BAD_ASCII_SEARCH_DICT = (source)

Undocumented

Value
{char.unescaped: char for char in BAD_CHARS}
BAD_CHARS = (source)

Undocumented

Value
[_BadChar('backspace',
          '\x08',
          '\\b',
          'E2510',
          'Moves the cursor back, so the character after it will overwrite the c↵
haracter before.'),
 _BadChar('carriage-return', '\r', '\\r', 'E2511', 'Moves the cursor to the star↵
...
BIDI_UNICODE: list[str] = (source)

Undocumented

Value
['‪', '‫', '‬', '‭', '‮', '⁦', '⁧', '⁨', '⁩', '‏']
BOM_SORTED_TO_CODEC = (source)

Undocumented

Value
OrderedDict(((UNICODE_BOMS[codec], codec) for codec in ('utf-32le', 'utf-32be', ↵
'utf-8',
    'utf-16le', 'utf-16be')))
UNICODE_BOMS = (source)

Undocumented

Value
{'utf-8': codecs.BOM_UTF8,
 'utf-16': codecs.BOM_UTF16,
 'utf-32': codecs.BOM_UTF32,
 'utf-16le': codecs.BOM_UTF16_LE,
 'utf-16be': codecs.BOM_UTF16_BE,
 'utf-32le': codecs.BOM_UTF32_LE,
 'utf-32be': codecs.BOM_UTF32_BE}
UTF_NAME_REGEX_COMPILED = (source)

Undocumented

Value
re.compile(r'utf[ -]?(8|16|32)[ -]?(le|be|)?(sig)?',
           re.IGNORECASE)
def _byte_to_str_length(codec: str) -> int: (source)

Return how many byte are usually(!) a character point.

@lru_cache(maxsize=1000)
def _cached_encode_search(string: str, encoding: str) -> bytes: (source)

A cached version of encode used for search pattern.

def _encode_without_bom(string: str, encoding: str) -> bytes: (source)

Encode a string but remove the BOM.

def _fix_utf16_32_line_stream(steam: Iterable[bytes], codec: str) -> Iterable[bytes]: (source)

Handle line ending for UTF16 and UTF32 correctly. Currently, Python simply strips the required zeros after after the line ending. Leading to lines that can't be decoded properly

def _line_length(line: _StrLike, codec: str) -> int: (source)

Get the length of a string like line as displayed in an editor.

def _map_positions_to_result(line: _StrLike, search_dict: dict[_StrLike, _BadChar], new_line: _StrLike, byte_str_length: int = 1) -> dict[int, _BadChar]: (source)

Get all occurrences of search dict keys within line. Ignores Windows end of line and can handle bytes as well as string. Also takes care of encodings for which the length of an encoded code point does not default to 8 Bit.

def _normalize_codec_name(codec: str) -> str: (source)

Make sure the codec name is always given as defined in the BOM dict.

def _remove_bom(encoded: bytes, encoding: str) -> bytes: (source)

Remove the bom if given from a line.

_StrLike = (source)

Undocumented

Value
TypeVar('_StrLike', str, bytes)