The practical answer for right now is that I don't want software screen capture, but it also does not do any of the things I wanted to be able to do by having remote inputs directed to the hardware rather than simulated in software. Namely it requires software installation on the server, which means OS support becomes a thing, you can't use it in BIOS/firmware, etc.
If you're like playing Apex/Valorant/PUBG/Dota and such, and scared that use of remote plays might get flagged, you could just use HDMI-in MPEGTS-out Ethernet capture streamer building scale HDMI extender thing(I don't know the actual categorical name). Inputs can be handled by a separate device, or just wireless keyboard/mouse.
Note that if anti-cheat is a concern, common dev boards like Arduino Leonardo and Pi Pico could also trigger flagging, so maybe that needs to be done slightly differently too(assuming the intent is good and this don't count towards anti-cheat evading in whatever you're doing).
Good point, I do play some games with anti-cheat like ARC Raiders. I don't think software screen capture would necessarily be an anti-cheat issue, maybe software mouse inputs are more likely to be.
As for dev boards, it wouldn't ever show up as a dev board to the PC, it would simply be simulating a dumb hardware keyboard. That's one of the reasons I wanted the board itself to have its own wired network.
Sunshine works on linux and windows, screen capture uses parts of the gpu meant to do that as result you get lower latency than separate capture and encode.
KVM is useful for when you need to be able to control earlier stuff in the boot process too, or even simple stuff like entering the password for full-disk encryption, or choosing a boot entry in GRUB, or changing BIOS settings... Many use cases for KVM and remote KVM.
Windows behaves badly during screen capture. I don't care if the gpu has a dedicated encoder and it is absolutely not lower latency than the gpu's native display output.
