☑ What’s New in Python 3.11 - Exception Improvements

31 Dec 2022 at 3:04PM in Software
 |  | 

In this series looking at features introduced by every version of Python 3, we continue our look at the new Python 3.11 release, taking a look at new language features around exceptions and error handling.

This is the 25th of the 35 articles that currently make up the “Python 3 Releases” series.

python 311

After looking at the performance improvements in Python 3.11 in the last article, in this one I’m going to run through the first set of what I feel are the main new features in the release, some changes to exceptions and their handling.

As an aside, you’ll probably have noticed these articles on 3.11 are smaller than some of my previously monolithic beasts in this series — I’ve come to the conclusion that it’s easier for me and you both if I don’t try to cram so much into them, and hopefully keep them more accessible.

Here’s a headline summary of what I’ll be discussing in this article:

  • A group of exceptions can now be raised, and handled as such as well.
    • The grouping is implemented with a new exception type.
    • The handling is achieved through a minor syntax change to except.
  • Exception notes have been added.
    • Exceptions can have multiple string notes added to them.
    • These are reproduced when the backtrace is displayed.
  • Fine-grained error locations have been added to tracebacks.
    • Specific expression is pinpointed, not just the line number.

Exception Grouping

Exception grouping, in the words of PEP 654, allows programs to raise and handle multiple unrelated exceptions simultaneously. It defines a new exception type ExceptionGroup, which represents a group of exceptions, and a new except* syntax for handling these groups.

In this section I try to provide an overview of the important aspects, but there are some intricate subtleties that I can’t cover in full detail here, particularly around differences between raising and reraising exceptions withint exception group handlers, so do check out the PEP if you need a higher level of detail. If this section gets a little too mentally taxing, just skip to the next one!

Why?

Before we go into the technical details, it’s fair to ask what’s the motivation for such a change. The PEP has a fairly extensive Rationale section, but to summarise the specific examples it lists are:

  • Concurrent errors in async code, where exceptions can occur in multiple coroutines and have to be returned in bulk when the results are gathered.
  • Multiple failures during retries of an operation, for example if a connect operation tries multiple IP addresses before returning an error they might each yield different exceptions.
  • Multiple callbacks might be registered with some event mechanism, and if they each raise different exceptions it’s useful to return them as a group.
  • Multiple errors during calculations might occur and it may be useful to return them all.
  • Wrapper code which raises an exception, where replacing a user-raised exception might erroneously bypass the user’s specific exception handling — returning an exception group containing both is a more accurate solution.

Raising Grouped Exceptions

The exception groups are themselves exceptions, and there are two:

ExceptionGroup
Can contain any subclass of Exception, and it itself is a subclass of Exception.
BaseExceptionGroup
Can contain any subclass of BaseException, and it itself is a subclass of BaseException.

ExceptionGroup class hierarchy

Since these themselves are Exception and BaseException subclasses, existing except and other mechanisms work on them. Their constructors take a message string, much like Exception, but also a list of other exceptions to include in the group.

>>> from traceback import print_exception
>>>
>>> try:
...     raise ExceptionGroup("This is my exception group", [
...         IndexError(5),
...         KeyError("five"),
...         RuntimeError("cinq")
...     ])
... except Exception as exc:
...     print_exception(exc)
...
  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 2, in <module>
  | ExceptionGroup: This is my exception group (3 sub-exceptions)
  +-+---------------- 1 ----------------
    | IndexError: 5
    +---------------- 2 ----------------
    | KeyError: 'five'
    +---------------- 3 ----------------
    | RuntimeError: cinq
    +------------------------------------

The consequence of these groups being themselves exceptions is that they can be nested — there’s nothing to stop you raising an ExceptionGroup which itself contains one or more ExceptionGroup instances as its contained exceptions. This means you can think of these groups as less of a container, and potentially more as a tree — although I don’t know in practice how often this sort of nesting is likely to be useful.

This is quite flexible, but it might make writing generic code to deal with these exceptions quite challenging — fortunately I suspect that most people will just need some fairly simple behaviours to deal with them.

Handling Grouped Exceptions

There are two basic ways to handle these exceptions, which I’ll discuss below.

Catch ExceptionGroup

The easiest way to handle these exception groups is to specifically catch them with something like this:

try:
    some_function()
except ExceptionGroup as exc_group:
    ...

At this point, there’s probably one of a couple of things you want to do. One would be to simply log the exception and move on — you can do this without any special effort, as we saw in the interaction session above the traceback for one of these groups includes the details for each contained exception.

The other likely option would be to look for certain types of exception which can be specially handled, and extract them from the group. There are a couple of methods on the group which can be used for this purpose. Both of these methods take a filter function, which expects to be passed an Exception or BaseException instance as the sole argument, and should return a bool.

subgroup()
Returns a group with the same metadata and structure, but with the contained exceptions filtered to include only those for which the filter function returns True. Any exception groups which are empty after filtering are elided, and if the group as a whole contains no exceptions then None is returned instead of an empty group.
split()
Similar to subgroup(), except it returns a 2-tuple where the first value is the same as for subgroup(), and the second value is the negation — i.e. a group of all the exceptions not included in the first return value.

For completeness there is also an exceptions attribute which is a tuple containing all of the exceptions within the group, which can be used to iterate through them manually, should that be required.

As a potentially contrived example, let’s imagine code which wishes to make a TCP connection to any one of the set of IP addresses returned for a domain, but only wants to log if any of the errors were ConnectionRefusedError, and introduce a delay in this case to avoid triggering security warnings.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
import socket
import time
from typing import Optional

def make_connection(host: str, port: int, max_tries: int) -> Optional[socket.socket]:
    for i in range(max_tries):
        try:
            return socket.create_connection((host, port), all_errors=True)
        except ExceptionGroup as exc_group:
            conn_refused = exc_group.subgroup(lambda x: isinstance(x, ConnectionRefusedError))
            if conn_refused is not None:
                print("Connection refused error(s): waiting for 5 secs"))
                time.sleep(5)
    return None

Incidentally, socket.create_connection() is one of only two examples in the library which I can find which has been updated to raise an ExceptionGroup, although only if the argument all_errors=True is passed. The other one I’ve come across is in asyncio.TaskGroup, which uses an ExceptionGroup (or BaseExceptionGroup) to share exceptions raised by tasks within the group.

New except* Syntax

Whilst the approach of catching ExceptionGroup explicitly like this is simple enough, it’s also a bit of a pain in some ways. What you’ll probably want to handle are the individual exceptions within the group, and using this approach you’d need to iterate over them yourself and do your own exception dispatch. The subgroup() and split() methods can be used for this purpose, along with the exceptions attribute, but a new except* syntax provides a more convenient approach.

The new except* syntax allows subsets of exception types to be extracted from the exception group, as with the subgroup() method, and handled within the a nearly normal except block. It’s important to note with this syntax, however, that the extracted exception will still be an ExceptionGroup (or BaseExceptionGroup) instance — Python doesn’t iterate over the grouped exceptions individually for you.

You can see a simple example of it in action here:

>>> from traceback import print_exception
>>>
>>> try:
...     raise ExceptionGroup("Message", [
...         IndexError(1),
...         KeyError("two"),
...         ValueError(3),
...         KeyError("four"),
...         IndexError(5)
...     ])
... except* ValueError as exc_group:
...     print("ValueError:")
...     print_exception(exc_group)
... except* (KeyError, IndexError) as exc_group:
...     print("IndexError and ValueError:")
...     print_exception(exc_group)
...
ValueError:
  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 2, in <module>
  | ExceptionGroup: Message (1 sub-exception)
  +-+---------------- 1 ----------------
    | ValueError: 3
    +------------------------------------
IndexError and ValueError:
  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 2, in <module>
  | ExceptionGroup: Message (4 sub-exceptions)
  +-+---------------- 1 ----------------
    | IndexError: 1
    +---------------- 2 ----------------
    | KeyError: 'two'
    +---------------- 3 ----------------
    | KeyError: 'four'
    +---------------- 4 ----------------
    | IndexError: 5
    +------------------------------------

The listed exception type(s) are extracted from the group as if using subgroup() and the resultant filtered group is set as the exception argument, exc_group in the examples above. In this example I’m just printing them, but you could equally well use the exceptions attribute to iterate across them and perform some action on each one individually.

As with normal tryexcept blocks, each clause is processed in turn, and each exception is only handled by the first matching block, even if the specification of a later block overlaps with it. Also, any exceptions remaining in the group which weren’t handled by any block will be put into a new group and propagate upwards.

Another aspect that’s important to realise is that these blocks will also handle regular exceptions which aren’t contained within a group — the PEP refers to these as naked exceptions. Since the code expects groups, however, these are wrapped in a group so the code handling them can be consistent without the need for additional special cases.

>>> try:
...     raise ValueError("hello, world")
... except* ValueError as exc_group:
...     print_exception(exc_group)
...
  | ExceptionGroup:  (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "<stdin>", line 2, in <module>
    | ValueError: hello, world
    +------------------------------------

To further keep the clauses consistent in their coding style, this new except* syntax cannot be combined with the original syntax — each tryexcept block must be all of one or the other. The excerpt below demonstrates the SyntaxError that occurs if you do mix them.

>>> try:
...     pass
... except ValueError:
...     pass
... except* IndexError as e:
  File "<stdin>", line 5
    except* IndexError as e:
    ^^^^^^^
SyntaxError: cannot have both 'except' and 'except*' on the same 'try'

In addition to this limitation, bare catch-all except* blocks, with no exception type specified, are not allowed because the meaning would be quite confusing. Also, you can’t specify ExceptionGroup (or BaseExceptionGroup) as the exception type within an except* block, only an except block — this is because the meaning would be rather ambiguous, so it’s been made invalid to avoid confusion.

Finally, the flow control statements break, continue and return cause a SyntaxError if they occur within an except* block and would cause flow to exit the block, despite the fact that they’re all valid in a normal except block. This decision was taken because the exceptions within a group are assumed to be independent of each other, so the handling of one of them shouldn’t be allowed to affect the handling of the others. However, if one except* block was allowed to change the flow of the code, then this principle would be broken.

>>> for i in range(1):
...     try:
...         some_function()
...     except* ValueError:
...         break
...
  File "<stdin>", line 5
SyntaxError: 'break', 'continue' and 'return' cannot appear in an except* block

These statements do appear to be legal if they are internal to the block itself, however, and do not affect the flow through other exception handling blocks. This does mean it is still possible to write code that skips handling of other exceptions, as long as their type is handled in the same clause.

>>> try:
...     raise ExceptionGroup("Test", [
...         ValueError(1),
...         ValueError(2)
...     ])
... except* ValueError as exc_group:
...     for exc in exc_group.exceptions:
...         print(exc)
...         break
...
1

Tracebacks

One more aspect that is of interest is the tracebacks which are printed for each exception within the group.

A normal exception has a single traceback object, stored as the __traceback__ attribute of the exception. This is a series of execution stack frames which lead from the point at which the exception was raised to where it was caught.

For exception groups the situation is similar, except that __traceback__ of the group represents the path from where the group was raised to where it was caught, and the __traceback__ of each exception within the group represents the path from where that exception was raised to where it was caught before being included in the group. When the group is printed, the tracebacks of each grouped exception is also displayed.

To illustrate this, consider the code below which raises an ExceptionGroup based on several exceptions caught whilst processing a contrived operation.

exceptiongroups.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import statistics

def convert(arg):
    ret = float(arg)
    return ret

def double_and_mean(list_str):
    values = []
    exceptions = []
    try:
        for arg in (i.strip() for i in list_str.split(",")):
            try:
                values.append(convert(arg))
            except Exception as exc:
                exceptions.append(exc)
        ret = statistics.geometric_mean(values)
    except Exception as exc:
        exceptions.append(exc)

    if exceptions:
        raise ExceptionGroup("Failed to calculate mean", exceptions)
    return ret

def main():
    print(double_and_mean("1.2, 3.4, ∞, 0"))

if __name__ == "__main__":
    main()

When executed, the ExceptionGroup propagates to the outermost scope and is printed, producing the output shown below.

  + Exception Group Traceback (most recent call last):
  |   File "/tmp/exceptiongroups.py", line 28, in <module>
  |     main()
  |   File "/tmp/exceptiongroups.py", line 25, in main
  |     print(double_and_mean("1.2, 3.4, ∞, 0"))
  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/tmp/exceptiongroups.py", line 21, in double_and_mean
  |     raise ExceptionGroup("Failed to calculate mean", exceptions)
  | ExceptionGroup: Failed to calculate mean (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/tmp/exceptiongroups.py", line 13, in double_and_mean
    |     values.append(convert(arg))
    |                   ^^^^^^^^^^^^
    |   File "/tmp/exceptiongroups.py", line 4, in convert
    |     ret = float(arg)
    |           ^^^^^^^^^^
    | ValueError: could not convert string to float: '∞'
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "/tmp/exceptiongroups.py", line 16, in double_and_mean
    |     ret = statistics.geometric_mean(values)
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Users/andy/.pyenv/versions/3.11.1/lib/python3.11/statistics.py", line 489, in geometric_mean
    |     raise StatisticsError('geometric mean requires a non-empty dataset '
    | statistics.StatisticsError: geometric mean requires a non-empty dataset containing positive numbers
    +------------------------------------

You can see that the initial traceback for the group as a whole leads to line 21, where the group was raised. The tracebacks for the ValueError and StatisticsError contained within it, however, lead to line 4 and line 16 respectively, which are the lines which triggered them.

Challenges

This looks like it has the potential to be a very useful feature, although perhaps only applicable in a fairly restricted set of cases. There are also a few challenges to adoption that I can see, however, and I wanted to finish up talking about a few things to consider.

Firstly, developers using libraries or functions updated to take advantage of this feature are going to have to include some careful notes in their documentation to remind developers they may want to switch to using except*. The feature has been implemented fairly carefully, however, so existing exception handling stands a good chance of doing some sort of a decent job — in particular, except Exception should still do something useful. The differentiation between ExceptionGroup and BaseExceptionGroup should make sure that except Exception clauses don’t end up catching the few exceptions which aren’t derived from it (e.g. GeneratorExit and KeyboardInterrupt). Overall this doesn’t seem like a major issue, but worth checking the release notes for libraries you use over the next year or two if they provide the sort of functionality that might feasibly be updated to use this grouping. Also be aware this isn’t a guarantee, however, since they might have a dependency which is updated to use grouping, but not mention it in their own release notes at all.

Secondly, as with any new Python feature this is going to make libraries depend on newer Python versions. Unlike some others, however, it’s going to be rather fiddly to take advantage of this in an optional manner — in other words, allow those using Python 3.11 to benefit from exception grouping, but still leave the code able to run unmodified on earlier Python versions. To do so would mean a difference in external interface depending on what Python version you’re using, and that could make life tricky for clients of you library as they upgrade. This might mean that only standard library modules, implicitly tied to the Python release cycle, may use this feature for awhile.

Thirdly, and perhaps most fundamentally, this feature has some tricky complexities that might elude developers for awhile until they become accustomed to it. Without wanting to get too bogged down in details, you can get some tricky subtleties if you mess around with nesting exception groups within each other, and with combinations of bare raise statements along with explicitly re-raising exceptions. If you don’t believe me, take a look at the code below and, without running it, see if you can predict the structure of the exception group that will propagate to the outermost scope.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
def one(arg):
    if arg < 4:
        return arg
    elif arg < 5:
        raise ValueError(arg)
    elif arg < 6:
        raise IndexError(arg)
    else:
        raise Exception("Foo")

def two(limit):
    errors = []
    for i in range(limit):
        try:
            print(f"Arg: {one(i)}")
        except Exception as exc:
            errors.append(exc)
    if errors:
        raise ExceptionGroup("There were errors", errors)

def three():
    errors = []
    for i in range(3, 8, 2):
        try:
            two(i)
        except Exception as exc:
            errors.append(exc)
    if errors:
        raise ExceptionGroup("Lots of errors", errors)

def four():
    try:
        three()
    except* ValueError as exc_group:
        print(f"Got {len(exc_group.exceptions)} ValueErrors")
        raise exc_group
    except* IndexError as exc_group:
        print(f"Got {len(exc_group.exceptions)} IndexErrors")
        raise

if __name__ == "__main__":
    four()

If you think it’s easy, run it and see if you were right. If you get that right first time with only a few moments thought, you’re a much better Python programmer than I!

If you’re still a bit puzzled, however, the diagram below may (or may not!) help you understand the grouping of exceptions that results — the sharp rectangles all represent ExceptionGroup instances, the rounded rectangles represent naked exceptions, and they’re all laballed with the line of code in the example above which is in the most recent entry in their traceback.

Working out why the code above leads to this grouping is a great way to test your understanding of this feature. As a hint, the part I find most confusing is working out where the top-level ExceptionGroup comes from, since there doesn’t appear to be anything which would create this. To work this out, compare the difference between line 36 and line 39, and read the section Raising Exceptions in an except* Block in the PEP.

Complex exception example diagram

Exception Notes

If you read all the way through the previous section, you may be happy to hear that the other two features covered in this article have considerably less scope for detail! The first of these is the ability to add notes to exceptions.

Very simply, exceptions now have a list of string notes that can be added using a new add_note() method, which is provided by BaseException. These are stored in a list under the __notes__ attribute, and they are also included whenever the exceptions are printed. As an aside, the __notes__ attribute is only created on the first call to add_note(), so don’t rely on it in exception handling code. You can see this all in action in the simple example below.

>>> try:
...     e = Exception("Description")
...     e.add_note("First note")
...     e.add_note("Second note")
...     raise e
... except Exception as exc:
...     print("Notes:", repr(exc.__notes__))
...     exc.add_note("Third note")
...     raise
...
Notes: ['First note', 'Second note']
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
Exception: Description
First note
Second note
Third note

There are a few uses for this, but I’d imagine one of the primary ones is when exceptions are caught and then rethrown — you often want to add some contextual information, but you don’t want to lose the original exception information in the process. Exception chaining, added way back in Python 3.0, goes some way to achieving this but the ability to append notes is also a simple and convenient way to add more information.

PEP 678 recommends that chaining is still the preferred option for cases of re-raising exceptions, but notes should be used in cases where the original exception object should be preserved, or in the case of exception groups, as described above. Speaking of exception groups, the notes for each exception within the group are also printed, which might be useful at times — see the snippet below.

>>> e1 = Exception("one")
>>> e1.add_note("This is one")
>>> e2 = RuntimeError("two")
>>> e2.add_note("This is two")
>>> raise ExceptionGroup("This is the group", (e1, e2))
  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 1, in <module>
  | ExceptionGroup: This is the group (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Exception: one
    | This is one
    +---------------- 2 ----------------
    | RuntimeError: two
    | This is two
    +------------------------------------

Fine-Grained Error Locations

The last error-handling related change is an improvement to the accuracy of error reporting, defined by PEP 657, building on the better error reporting that we saw in Python 3.10. Instead of just showing the line numbers for an entry in the traceback, the specific expression within the line can also now be highlighted.

Example

To see the difference, let’s see the exception raised by the following code:

foo.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import logging
import sys

CONFIG = {
    "logging": {
        "enabled": True,
        "filename": "/var/log/foo.log",
        "level": "DEBUG",
    }
}

def set_up_logging(config_dict):
    if not config_dict["logging"]["logging"]:
        return True
    filename = config_dict["logging"]["filename"]
    level = config_dict["logging"]["level"]
    if os.path.exists(filename):
        return False
    logging.basicConfig(
        filename=filename,
        level=level
    )
    return True

def main():
    if not set_up_logging(CONFIG):
        return 1
    return 0

if __name__ == "__main__":
    sys.exit(main())

In Python 3.10 you can see that the traceback only indicates the line numbers:

$ python3.10 foo.py
Traceback (most recent call last):
  File "/tmp/foo.py", line 31, in <module>
    sys.exit(main())
  File "/tmp/foo.py", line 26, in main
    if not set_up_logging(CONFIG):
  File "/tmp/foo.py", line 13, in set_up_logging
    if not config_dict["logging"]["logging"]:
KeyError: 'logging'

In Python 3.11, however, the specific expressions within the line are highlighted:

$ python3.11 foo.py
Traceback (most recent call last):
  File "/tmp/foo.py", line 31, in <module>
    sys.exit(main())
             ^^^^^^
  File "/tmp/foo.py", line 26, in main
    if not set_up_logging(CONFIG):
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/foo.py", line 13, in set_up_logging
    if not config_dict["logging"]["logging"]:
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'logging'

The final entry in the traceback particularly highlights the utility of this, because it indicates which of the keys is the missing one, which is ambiguous in the Python 3.10 version.

Internals

To provide this information required some updates to the bytecode to store the start and end columns of the source code which produced each bytecode instrucion. Because this increases the storage required for these instructions, both in memory and on disk in .pyc files, only a single byte is added for each offset. This means that only offsets from 0-255 characters can be stored, with the column offsets being zero-based. Any expressions extended beyond this limit lose their offset information.

Below is an example script which disassembles the aptly named inefficient_sum() function and shows the line and column information for each bytecode produced. For convenience, the comment at the top shows the column offsets (reach each 2-digit value downwards) — remember that they’re zero-based, so the # at the start is column zero.

offset.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
import dis

# Cols:   111111111112222222223333333333444444
#123456789012345678901234567890123456789012345
def inefficient_sum(values, sum_so_far=0):
    if not values:
        return sum_so_far
    first, *rest = values
    return inefficient_sum(
        rest,
        sum_so_far + first
    )

for i in dis.get_instructions(inefficient_sum):
    opname = i.opname
    arg = i.argval
    start_line = i.positions.lineno
    end_line = i.positions.end_lineno
    start_col = i.positions.col_offset
    end_col = i.positions.end_col_offset
    print(f"{opname}({arg}): line={start_line}-{end_line}"
          f" col={start_col}-{end_col}")

The dis.get_instructions() function produces an iterator of dis.Instruction instances. In Python 3.11 these have an added position attribute of a new dis.Positions class which provides start and end lines and columns, as used in the code shown above.

The output produced should be as follows:

$ python offset.py
RESUME(0).....................: line=5-5   col=0-0
LOAD_FAST(values).............: line=6-6   col=11-17
POP_JUMP_FORWARD_IF_TRUE(10)..: line=6-7   col=4-25
LOAD_FAST(sum_so_far).........: line=7-7   col=15-25
RETURN_VALUE(None)............: line=7-7   col=8-25
LOAD_FAST(values).............: line=8-8   col=19-25
UNPACK_EX(1)..................: line=8-8   col=4-16
STORE_FAST(first).............: line=8-8   col=4-9
STORE_FAST(rest)..............: line=8-8   col=12-16
LOAD_GLOBAL(inefficient_sum)..: line=9-9   col=11-26
LOAD_FAST(rest)...............: line=10-10 col=8-12
LOAD_FAST(sum_so_far).........: line=11-11 col=8-18
LOAD_FAST(first)..............: line=11-11 col=21-26
BINARY_OP(0)..................: line=11-11 col=8-26
PRECALL(2)....................: line=9-12  col=11-5
CALL(2).......................: line=9-12  col=11-5
RETURN_VALUE(None)............: line=9-12  col=4-5

As a final note, if you don’t want the overhead of these offsets added to your code you can either set the environment variable PYTHONNODEBUGRANGES=1 or you can pass the command-line option -X no_debug_ranges to the interpreter. This won’t break any code, but the column offsets will come out as None if queried, and the tracebacks won’t be decorated with the highlighted expressions on each line.

$ python -X no_debug_ranges offset.py
RESUME(0).....................: line=5-5   col=None-None
LOAD_FAST(values).............: line=6-6   col=None-None
POP_JUMP_FORWARD_IF_TRUE(10)..: line=6-6   col=None-None

Conclusion

Some useful changes here, although it’s early days yet so I must admit I’m a little unsure where these will lie on the utility scale that runs from “couldn’t live without it” to “I forgot it was there”. Exception grouping seems useful for some cases, but I suspect those cases will be a little limited, and the mechanics of it seem a little complicated for some developers. That said, for the cases where it’s useful, it’s going to be a lot better than people rolling their own bespoke mechanism for doing the same thing in each library.

Adding notes to exceptions is a much simpler mechanism, and it strikes me this is likely to apply to more situations — although exception chaining could be used in some of these, the notes are likely easier for everyone and I suspect they might catch on well.

Finally, the expression highlighting in errors basically comes for free for developers, if you ignore the slight code size overhead, and I think this probably shows the greatest potential to improve life for all Python developers.

All in all, though, nothing to dislike here, in my view.

The next article in the “Python 3 Releases” series is What’s New in Python 3.11 - Type Hint Improvements
Tue 3 Jan, 2023
31 Dec 2022 at 3:04PM in Software
 |  |