Learn Remote Desktop - Troubleshooting VNC/RDP Connections

#Introduction

No matter how tidy your configuration is, at some point a connection will fail. Episode 20 trains you to become a detective: reading symptoms, checking the right layers, and solving problems with sequential steps — not by guessing.

You'll learn the main diagnostic tools (ss -tulpn, firewall, vncserver -list, journalctl), proper client-server tests, and then a solution map for the four most common problems: black screens, color corruption, slow refreshes, and input that doesn't work.

#Diagnosing the Network Layer

#Checking Ports with ss

The first step is always confirming that the service is really listening on the expected port:

Checking remote service ports

ss -tulpn | grep -E ":(5900|5901|3389)"

The ss -tulpn | grep -E ":(5900|5901|3389)" command shows the processes listening on the VNC and RDP ports. If the port doesn't appear, the service is dead or failed to start — move on to the service check.

#Checking the Firewall

If the port appears but the connection still fails, suspect the firewall. Check the rule status on both sides:

Checking firewall rules

sudo ufw status verbose
sudo firewall-cmd --list-all

Remember from episode 16: test from another machine, not from the server itself. nc -zv is the fastest tool to separate network problems from application problems:

Testing the port from the client

nc -zv 192.168.1.20 5901

#Diagnosing the Server Layer

#Checking the VNC Session

For TigerVNC, look at the list of running sessions:

Listing VNC sessions

vncserver -list

vncserver -list shows the active displays. If the expected display isn't there, the session failed to be created — the common causes are xstartup or the password.

#Reading Logs with journalctl

systemd services store all their logs in the journal. For an active service, read the logs live:

Reading the VNC service log

journalctl -u vncserver@:1 -f

For x11vnc and xrdp, change the unit name to match each service. journalctl -u vncserver@:1 -f displays new log lines as they appear — the best way to see the error that occurs exactly when a connection opens.

#TigerVNC Session Logs

Manually run VNC sessions write logs to ~/.vnc/:

Reading manual session logs

tail -20 ~/.vnc/host:1.log

Errors like "unable to connect" or "Authentication failed" almost always appear here before you start guessing the cause.

#Proper Client-Server Tests

#The Complete Check Sequence

When everything fails, run this sequence from the client machine:

Diagnosis sequence from the client

ping -c 3 192.168.1.20
nc -zv 192.168.1.20 5901
vncviewer 192.168.1.20:5901

ping fails → network or routing problem.
ping succeeds, nc fails → port closed (firewall or dead service).
nc succeeds, viewer fails → authentication or protocol problem.

The nc -zv command is the most important marker: once the port accepts TCP connections, the problem is no longer on the network.

Info

Write down the result of every step. "Ping works, port is open, but authentication fails" already narrows the problem to the password or passwd file — far faster than changing configuration at random.

#Common Problems and Their Fixes

#Black Screen

A black screen on connection almost always comes from two sources: a display problem or an authentication problem.

For x11vnc: make sure -display and -auth guess are correct, and the shared display is really active.
For TigerVNC: check ~/.vnc/xstartup — the file must be executable and call an installed desktop.
For xrdp: try an Xorg session instead of the full desktop session.

Checking display and xstartup

echo $DISPLAY
ls -la ~/.vnc/xstartup

#Color Corruption

Wrong or "broken"-looking colors are usually caused by a depth mismatch between server and viewer. Set the depth explicitly on both sides:

Setting the depth explicitly

vncserver -kill :1
vncserver :1 -geometry 1920x1080 -depth 24
vncviewer 192.168.1.20:5901

If colors are still wrong after setting the depth, try a different encoding in the viewer — sometimes certain encodings render colors imperfectly on older machines.

#Slow Refreshes

The screen works but the refresh feels choppy. Causes and fixes:

Turn off compositing effects on the desktop (episode 19).
Lower quality and depth for WAN connections.
Enable caching with -ncache 10 in x11vnc.
Check the server CPU — heavy Tight compression can overwhelm the server.

#Input Not Working

The cursor shows but clicks and keystrokes have no effect. Possible causes:

The connection is in view-only mode — check the view-only password and enable-view-only.
Keyboard layout mismatch — set it with setxkbmap (episode 11).
Input events aren't reaching because the server is busy — check the service log.

Verifying the connection mode

x11vnc -display :0 -rfbauth /etc/x11vnc.pass -viewonly

If the server was deliberately run with -viewonly, all connections can only view. Remove that option to allow control again.

#Conclusion

Episode 20 trained systematic diagnosis: checking ports with ss -tulpn and the firewall, monitoring sessions with vncserver -list and journalctl, running proper client-server tests, and solving black screens, color corruption, slow refreshes, and input that doesn't work.

Key takeaways:

Start from ss -tulpn, then test from the client with nc -zv.
vncserver -list and journalctl reveal problems on the server side.
nc succeeding means the problem moved from the network to the application.
Black screens come from display/auth or a problematic xstartup.
Broken colors mean the server and viewer depths don't match.
Dead input often comes from view-only mode or keyboard layout.

In the next episode, episode 21, we look to the future: modern tools and roadmap — TigerVNC 1.16 with w0vncserver, x11vnc 0.9.17, TightVNC 2.8.88, and GNOME Remote Desktop 50.x, complete with a migration map from older tools. See you there!