Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

C

Cpython
- Activity
- Members
- Labels
- Issues
- Issue boards
- Milestones
- Wiki
- Environments
- Terraform modules
- Incidents

Snippets Groups Projects

8bc356a7

Unverified Commit 8bc356a7 authored 2 years ago by Michael Droettboom Committed by GitHub 2 years ago

Downloads
- Patches
- Plain Diff

gh-96268: Fix loading invalid UTF-8 (#96270)

This makes tokenizer.c:valid_utf8 match stringlib/codecs.h:decode_utf8.

It also fixes an off-by-one error introduced in 3.10 for the line number when the tokenizer reports bad UTF8.

parent 3e26de3c

Branches

Tags

No related merge requests found

Changes 3

Show whitespace changes

Inline Side-by-side

Showing

Lib/test/test_source_encoding.py 10 additions, 3 deletions

Lib/test/test_source_encoding.py
Misc/NEWS.d/next/Core and Builtins/2022-08-25-10-19-34.gh-issue-96268.AbYrLB.rst 2 additions, 0 deletions

...nd Builtins/2022-08-25-10-19-34.gh-issue-96268.AbYrLB.rst
Parser/tokenizer.c 45 additions, 13 deletions

Parser/tokenizer.c

with 57 additions and 16 deletions

Loading

0% Loading or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment