Fix protocol subtyping in case of restricted self-types

[ilevkivskyi]

Fixes #19341
Fixes #20061

This is a bit hacky, but an alternative would be to change return type of a lot of functions in checkmember.py. Also although this is a niche use case, it seems to be important for numpy and pandas.

cc @JukkaL

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/badge.py:106: error: Covariant type variable "AppT" used in protocol where invariant one is expected  [misc]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/series/bool/test_truediv.py:30: error: Unsupported operand types for / ("Series[bool]" and "bool")  [operator]
+ tests/series/bool/test_truediv.py:37: error: Unsupported operand types for / ("bool" and "Series[bool]")  [operator]
+ tests/series/bool/test_truediv.py:44: error: Argument 1 to "truediv" of "Series" has incompatible type "builtins.bool"; expected "Just[int] | integer[Any] | Sequence[Just[int] | integer[Any]] | ndarray[tuple[Any, ...], dtype[integer[Any]]] | IntegerArray | Index[int] | Series[int] | Sequence[builtins.bool | numpy.bool[builtins.bool]]"  [arg-type]
+ tests/series/bool/test_truediv.py:44: note: Following member(s) of "bool" have conflicts:
+ tests/series/bool/test_truediv.py:44: note:     __class__: expected "type[int]", got "type[bool]"
+ tests/series/bool/test_truediv.py:55: error: Argument 1 has incompatible type "builtins.bool"; expected "Just[int] | integer[Any] | Sequence[Just[int] | integer[Any]] | ndarray[tuple[Any, ...], dtype[integer[Any]]] | IntegerArray | Index[int] | Series[int] | Sequence[builtins.bool | numpy.bool[builtins.bool]]"  [arg-type]
+ tests/series/bool/test_truediv.py:55: note: Following member(s) of "bool" have conflicts:
+ tests/series/bool/test_truediv.py:55: note:     __class__: expected "type[int]", got "type[bool]"
+ tests/series/bool/test_truediv.py:62: error: Argument 1 to "rtruediv" of "Series" has incompatible type "builtins.bool"; expected "Just[int] | integer[Any] | Sequence[Just[int] | integer[Any]] | ndarray[tuple[Any, ...], dtype[integer[Any]]] | IntegerArray | Index[int] | Series[int] | Sequence[builtins.bool | numpy.bool[builtins.bool]]"  [arg-type]
+ tests/series/bool/test_truediv.py:62: note: Following member(s) of "bool" have conflicts:
+ tests/series/bool/test_truediv.py:62: note:     __class__: expected "type[int]", got "type[bool]"
+ tests/series/bool/test_truediv.py:73: error: Argument 1 has incompatible type "builtins.bool"; expected "Just[int] | integer[Any] | Sequence[Just[int] | integer[Any]] | ndarray[tuple[Any, ...], dtype[integer[Any]]] | IntegerArray | Index[int] | Series[int] | Sequence[builtins.bool | numpy.bool[builtins.bool]]"  [arg-type]
+ tests/series/bool/test_truediv.py:73: note: Following member(s) of "bool" have conflicts:
+ tests/series/bool/test_truediv.py:73: note:     __class__: expected "type[int]", got "type[bool]"
+ tests/series/bool/test_truediv.py:87: error: Unsupported operand types for / ("Series[bool]" and "list[bool]")  [operator]
+ tests/series/bool/test_truediv.py:94: error: Unsupported operand types for / ("list[bool]" and "Series[bool]")  [operator]
+ tests/indexes/bool/test_truediv.py:30: error: Unsupported operand types for / ("Index[bool]" and "bool")  [operator]
+ tests/indexes/bool/test_truediv.py:30: note: Following member(s) of "bool" have conflicts:
+ tests/indexes/bool/test_truediv.py:30: note:     __class__: expected "type[int]", got "type[bool]"
+ tests/indexes/bool/test_truediv.py:37: error: Unsupported operand types for / ("bool" and "Index[bool]")  [operator]
+ tests/indexes/bool/test_truediv.py:49: error: Unsupported operand types for / ("Index[bool]" and "list[bool]")  [operator]
+ tests/indexes/bool/test_truediv.py:56: error: Unsupported operand types for / ("list[bool]" and "Index[bool]")  [operator]

pytest-autoprofile (https://gitlab.com/TTsangSC/pytest-autoprofile)
+ src/pytest_autoprofile/_patches.py:64: error: Contravariant type variable "Arg" used in protocol where invariant one is expected  [misc]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/io/stata.py:1930: error: Invalid index type "int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]" for "list[str | dtype[Any]]"; expected type "SupportsIndex"  [index]
+ pandas/io/stata.py:1931: error: Invalid index type "int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]" for "list[int | str]"; expected type "SupportsIndex"  [index]
+ pandas/io/stata.py:1932: error: Invalid index type "int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]" for "list[str]"; expected type "SupportsIndex"  [index]
+ pandas/io/stata.py:1933: error: Invalid index type "int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]" for "list[str]"; expected type "SupportsIndex"  [index]
+ pandas/io/stata.py:1935: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/io/stata.py:1936: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/io/stata.py:1937: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/io/stata.py:1938: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/frame.py:2272: error: Invalid index type "int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]" for "list[Any]"; expected type "SupportsIndex"  [index]
+ pandas/core/internals/construction.py:412: error: Invalid index type "int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]" for "list[Any]"; expected type "SupportsIndex"  [index]
+ pandas/core/indexes/base.py:6113: error: Incompatible return value type (got "Any | Index", expected "Hashable")  [return-value]
+ pandas/core/resample.py:2838: error: Incompatible types in assignment (expression has type "Index", variable has type "PeriodIndex")  [assignment]

numpy-stl (https://github.com/WoLpH/numpy-stl)
+ stl/base.py:1092: error: Return type "Iterator[ndarray[tuple[int], dtype[floating[_32Bit]]]]" of "__iter__" incompatible with return type "Iterator[SupportsIndex | slice[Any, Any, Any] | EllipsisType | _SupportsArray[dtype[numpy.bool[builtins.bool] | integer[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool] | integer[Any]]]] | int | _NestedSequence[int] | None | tuple[SupportsIndex | slice[Any, Any, Any] | EllipsisType | _SupportsArray[dtype[numpy.bool[builtins.bool] | integer[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool] | integer[Any]]]] | int | _NestedSequence[int] | None, ...]]" in supertype "typing.Iterable"  [override]

scipy-stubs (https://github.com/scipy/scipy-stubs)
- tests/misc/test_submodules_empty.pyi:3: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports

[ilevkivskyi]

Having mostly new errors in primer is expected, since this is a fix for a false negative. However, some of the, look suspicious. I will double-check. Also cc @cmp0xff

[cmp0xff]

Thank you for the information @ilevkivskyi. The new errors in pandas-stubs are all expected in negative tests (if TYPE_CHECKING_INVALID_USAGE). They were pending # type: ignore annotations explicitly waiting on #20061. We will update pandas-stubs once this lands.

[ilevkivskyi]

@cmp0xff OK, thanks for confirming!

I also looked at two variance inference issues in steam.py and in pytest-autoprofile, both are kind of in a grey area. There is currently a discussion about specifying how self-types should affect inferred variance in the spec, see python/typing#2281. We can adjust the behavior after this is settled.

Few remaining errors related to __index__ look correct, and likely related to numpy/numpy@d4012e6 by @jorenham

@JukkaL this is ready for review.