afaik correct me if im wrong but rust desk is not really the right tool for that. Rustdesk connects devices by connecting all the way through the internet to the rustdesk servers and then to the other device.
VNC or RDP are the right approaches here, cause you can use them to connect through the local network, never leaving your LAN/intranet. In addition to not needing a connection to the internet it should be more reliable that way.
If you use KDE then I think their krdc gui is pretty good. Otherwise Remmina.
For the server part, tigerVNC is fine.