egl_worker thread never receives CAP_SYS_ADMIN despite correct setcap on the binary — blocks KMS capture entirely (reproducible on both direct-login and xrdp-created sessions)

[jessica-atk]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

None

Describe the Bug

setcap cap_sys_admin,cap_sys_nice=p is correctly applied to the sunshine binary and confirmed present in the file's capability set. However, the dedicated egl_worker thread (introduced by the privileged-EGL-worker-thread security architecture in GHSA-fp6g-27w5-489j / commit 7a987ee) never actually holds cap_sys_admin in its permitted capability set at runtime — only cap_sys_nice is present. This is reproducible identically in two architecturally different session types on the same machine, ruling out a session-specific cause:

• A normal direct physical console login (XFCE, X11)
• A session created via xrdp/xorgxrdp (also XFCE, X11, but a different X driver — xrdpdev rather than nvidia)

As a direct consequence, KMS capture (capture = kms) fails to initialize at all (Unable to initialize capture method, Platform failed to initialize), and every encoder subsequently fails during startup probing with Failed to gain CAP_SYS_ADMIN (nvenc, vulkan, vaapi, and even the software encoder). Sunshine correctly falls back to NvFBC capture when capture is left on auto, so basic streaming still works — but this closes off KMS as a usable capture path, which is otherwise the one path (via its gl_cuda_vram_t encode-device class) known to support AV_PIX_FMT_P010LE and therefore genuine 10-bit output, unlike the NvFBC+CUDA path (cuda::cuda_t, tracked separately as an 8-bit-only limitation).

Expected Behavior

The egl_worker thread (or whichever thread actually calls the KMS/DRM-master-acquiring code) should hold cap_sys_admin in its permitted capability set at the point it attempts to use it, given the binary has the correct file capability and the security-hardening redesign was specifically intended to preserve this capability for exactly this worker thread while dropping it elsewhere.

Additional Context

`$ getcap /usr/local/bin/sunshine
/usr/local/bin/sunshine cap_sys_admin,cap_sys_nice=p

$ getpcaps
: =

$ grep -E "CapPrm|CapEff|CapBnd" /proc//task/<egl_worker tid>/status
CapPrm: 0000000000800000 # cap_sys_nice only — cap_sys_admin (bit 21) absent
CapEff: 0000000000000000
CapBnd: 000001ffffffffff # cap_sys_admin present in bounding set, so it's not a bounding-set restriction`

Confirmed identical on both a direct physical login session and an xrdp-created session on the same host, ruling out session-manager/PAM-stack causes (also checked and ruled out: NoNewPrivs is 0 in both cases; systemd --user's own CapabilityBoundingSet= includes cap_sys_admin; the sunshine.service unit file has no NoNewPrivileges= directive).

This was found while diagnosing a related, separate limitation: NvFBC+CUDA capture (cuda::cuda_t) only ever produces AV_PIX_FMT_NV12/AV_PIX_FMT_YUV444P frames (no P010/10-bit path — already tracked, e.g. PR #5429's description). KMS capture was the natural workaround to test, since its gl_cuda_vram_t encode-device class is documented to support AV_PIX_FMT_P010LE. This capability-inheritance issue is what's actually blocking that workaround from being tested at all, independent of whatever the eventual fix for the NvFBC 8-bit limitation turns out to be. Happy to provide full sunshine.log output, the full systemd unit file, or run further diagnostics on request — this was reproduced carefully across two session types specifically to rule out environmental causes.

I do have a fully documented build log that I am happy to share privately with you.

Host Operating System

Linux

Operating System Version

Rocky Linux 9.6 (Blue Onyx), glibc 2.34

Architecture

amd64/x86_64

Package

other (self built)

GPU Type

NVIDIA

GPU Model

RTX 4000

GPU Driver/Mesa Version

NVIDIA 580.65.06, CUDA 13.3 toolkit

Capture Method

KMS (Linux)

Apps

Log output

[timestamp]: Info: config: 'capture' = kms
[timestamp]: Info: /dev/dri/card0 -> nvidia-drm
[timestamp]: Error: Unable to initialize capture method
[timestamp]: Error: Platform failed to initialize
[timestamp]: Info: Trying encoder [nvenc]
[timestamp]: Error: Failed to gain CAP_SYS_ADMIN
[timestamp]: Info: Encoder [nvenc] failed
[timestamp]: Info: Trying encoder [vulkan]
[timestamp]: Error: Failed to gain CAP_SYS_ADMIN
[timestamp]: Info: Encoder [vulkan] failed
[timestamp]: Info: Trying encoder [vaapi]
[timestamp]: Error: Failed to gain CAP_SYS_ADMIN
[timestamp]: Info: Encoder [vaapi] failed
[timestamp]: Info: Trying encoder [software]
[timestamp]: Error: Failed to gain CAP_SYS_ADMIN
[timestamp]: Info: Encoder [software] failed
[timestamp]: Fatal: Unable to find display or encoder during startup.

Online logs

No response

[psyke83]

Please share a full log and the output of the following two times; once when kms capture is set in the config and startup was attempted, and then once more when NvFBC capture is set and running:

for i in $(ls /proc/$(pidof sunshine)/task/); do echo -e $(getpcaps $i; cat /proc/$i/comm); done | grep -Ei "cap_sys_|_worker"

[girlypop2000]

Yeah, I tried to use the KMS method on Linux Mint 22.3 with Rtx3060 and it just doesn't work. The cap_sys_admin is all over in the logs even when using other capture methods. Tried setting cap_sys manually in terminal and even running the program as a service or straight from terminal instead of gui but no luck. Might be nvidia related...

[girlypop2000]

.deb package is what i have just to clarify

[psyke83]

I still need to see full logs and output from the command I posted.

The original bug report also misdiagnoses the problem. There are two privileged worker threads; drm_worker holds cap_sys_admin and cap_sys_nice, but egl_worker only needs to hold cap_sys_nice for creating EGL contexts.

There's a part of the code where the drm_worker drops its privileges if kms or nvfbc capture is not selected/detected. That might be getting triggered erroneously. More logs would help.