Skip to content

[mypyc] Fix RecursionError from binary dunders without a reverse method - #22105

Open
rheard wants to merge 3 commits into
python:masterfrom
rheard:fix-mypyc-1194
Open

rheard wants to merge 3 commits into
python:masterfrom
rheard:fix-mypyc-1194

Conversation

@rheard

@rheard rheard commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Fixes mypyc/mypyc#1194.

A native class that defines a binary dunder but not its reverse (e.g. __add__ without __radd__) can recurse infinitely when compiled:

class A:
    def __add__(self, other: int) -> int:
        return 1

    def __pow__(self, other: int) -> int:
        return 2

A() + A()  # RecursionError (TypeError when interpreted)
1 + A()    # RecursionError
2 ** A()   # RecursionError

The same happens when the method returns NotImplemented, as in the issue.

When the argument type check fails or the method returns NotImplemented, the slot wrapper calls CPy_CallReverseOpMethod, which looks up and calls the reverse method of the right operand. The class doesn't define __radd__, so for an instance of it the lookup finds the __radd__ wrapper that CPython adds for the nb_add slot, which calls the same slot again with the same arguments.

The wrappers now return NotImplemented instead, as the slots of built-in types do. CPython's binary_op1()/ternary_op() already try the right operand's slot when its type has a different one, so the explicit call isn't needed. CPy_CallReverseOpMethod and the interned reverse method names are removed, since nothing else uses them.

Other changes, all matching CPython:

  • A reverse method on another type is called once instead of twice when it returns NotImplemented.
  • x ** "a" raises unsupported operand type(s) for ** or pow(): ... instead of ... for **: .... Two existing tests are updated for this.
  • If an interpreted subclass defines __radd__, s + s no longer calls it when __add__ rejects the operand, since CPython doesn't try the reflected method for operands of the same type either.

@JukkaL JukkaL left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this also seems to improve performance! I have one optional performance suggestion, which seems to improve performance still a bit more on free-threaded builds.

Comment thread mypyc/codegen/emitwrapper.py Outdated
# ...
generate_bin_op_reverse_dunder_call(fn, emitter, reverse_op_methods[fn.name])
emitter.emit_line("Py_INCREF(Py_NotImplemented);")
emitter.emit_line("return Py_NotImplemented;")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Py_RETURN_NOTIMPLEMENTED;, which would avoid an immortality check on Python 3.12+?

@rheard rheard Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, thats a good idea. I like that and have done it.

The reason I did it that way though was because I was copying the other 2 instances of it in this file, like on line 452. It makes sense to me that if I'm going to be changing it here, I should probably change it across this file. Hope that isn't considered too out of scope. Let me know.

Thanks again for the suggestion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dunder methods that return NotImplemented explode the stack

2 participants