Simplifying Remote VNC Logins

Virtual Network Computing (VNC) is an increasingly popular remote-access protocol. VNC is available for Linux, Windows, Mac OS, and others. (If you're unfamiliar with VNC or don't have VNC installed on your Linux system, see this month's "Tech Support" column on page 62 for instructions on how to install and use VNC.)

Virtual Network Computing (VNC) is an increasingly popular remote-access protocol. VNC is available for Linux, Windows, Mac OS, and others. (If you’re unfamiliar with VNC or don’t have VNC installed on your Linux system, see this month’s “Tech Support” column on page 62 for instructions on how to install and use VNC.)

Unfortunately, VNC’s usual method of operation is clumsy in some ways. VNC users may need to log in via ssh and run the VNC server manually. Although VNC asks for a password, it doesn’t ask for a username, so users need to log on using a specific VNC session number — an awkward restriction.

Fortunately, there’s a solution: you can launch a VNC server in such a way that it uses an X Display Manager Control Protocol (XDMCP) server running on the same computer. Linux systems normally run XDMCP servers to handle local GUI logins — so when somebody uses VNC to connect to a computer that’s configured with VNC and XDMCP, VNC displays a normal Linux GUI login prompt. This enables users to enter a username and password, and perhaps even dispense with the VNC session number.

Before proceeding, you should obtain and install a VNC server. The widely-used AT&T VNC package comes with most Linux distributions or you can download it from http:// www.uk.research.att.com/vnc (or you can use a variant such as TightVNC, found at http://www.tightvnc.com). You’ll also need an XDMCP server, such as the X Display Manager (XDM), KDE Display Manager (KDM), or GNOME Display Manager (GDM). At least one of those display managers is normally installed with X.

Let’s look at VNC and XDMCP separately and then see how they can be combined into something more powerful.

VNC’s Method of Operation

On a Linux computer, a VNC server is really two servers in one. The first server typically accepts remote connections using the VNC protocol. This server sends display data to the VNC client and also receives keystrokes and mouse activity from the client.

Figure One: VNC Components and Relationships

The second part of a VNC server is an X server. A traditional X server provides an interface to a video card, keyboard, and mouse, but the VNC X server instead links to the remote computer. This arrangement is illustrated in Figure One.

Users normally run the VNC server using the vncserver script. This script calls the Xvnc program, which is the actual X and VNC server. A user accesses VNC by logging in remotely, running vncserver, and then running a VNC client to access the newly-started VNC server. As part of the startup process, the VNC server returns a session number. In a multi-user environment, each user receives a different VNC session number.

This configuration has a couple of big drawbacks: the user has to login twice — once via ssh (say) to run vncserver and again to login via XDMCP — and the user has to remember the VNC session number.

This typical configuration also has an unusual feature that may be an advantage or a disadvantage, depending upon your needs: VNC sessions are persistent — they remain intact even when a user disconnects from the VNC server. So, a user can start a VNC session, close it without exiting from running programs, and begin again with running programs open and in the state in which they were left. This feature can be convenient, but it can also cause problems (such as excessive memory use) if users forget about their “abandoned” VNC sessions. Moreover, on a system with many users, having many VNC servers running at all times can be a drain on available memory.

Traditional Linux X-Based Logins

Most Linux distributions today install X and configure it to launch automatically when the computer starts. In this configuration, an XDMCP server is used to mediate logins. Typically, a startup script launches the XDMCP server, which in turn starts X. The XDMCP server takes control of the local X display and presents a GUI login screen.

An XDMCP server can do much more than display a local login screen, though. An XDMCP server can accept connections from remote X servers, performing the negotiations necessary to allow local X programs to use the remote X server. Ultimately, we’ll use this feature to have XDMCP improve VNC.

Most Linux distributions ship with security-conscious XDMCP configurations that disable many of the XDMCP server’s capabilities — after all, you don’t want to expose your computer to would-be intruders for no reason. Thus, part of what you must do to enable VNC to work with XDMCP is to reconfigure XDMCP to accept some types of login requests. Unfortunately, this is easier said than done because there are three major XDMCP servers in common use: XDM, KDM, and GDM. Each of these tools has its own configuration files and quirks.

Configuring an XDMCP Server For VNC

To use XDMCP with VNC, your XDMCP server must accept connection attempts from the local computer. Ordinarily, the XDMCP server doesn’t accept such connections. It just manages the local X display. Precisely how you do this depends upon the XDMCP server your system runs. If you’re not sure which XDMCP server your system uses, type ps ax | grep dm and examine the output for one of the XDMCP server names. Of course, the XDMCP server must be running for this to work, and that’s only likely to be true if your system is configured for GUI logins. On most distributions, setting the system to use runlevel 5 will do this. Type telinit 5 to enter runlevel 5 until you reboot or change it manually. You can also edit /etc/inittab and change the number in the id line to 5 to change the runlevel permanently.

The simplest XDMCP server to reconfigure is GDM, which is the default for Red Hat and some other distributions. Open /etc/X11/gdm/gdm.conf in your favorite editor and locate the [xdmcp] section. Change the line that reads enable=false to enable=true. This line reconfigures GDM to function as a network-enabled XDMCP server and to accept remote login requests. If you prefer, you can use the gdmsetup program to edit gdm.conf. Click the “XDMCP” tab and check the “Enable XDMCP” box.

XDM’s configuration files usually reside in /etc/X11/xdm, and some versions of KDM use those files as well. More recent versions of KDM use similar files in another directory, typically /etc/kde2/kdm, /opt/kde/share/config/kdm, or something similar.

For both KDM and XDM, one critical file is Xaccess. The Xaccess file controls who can access the XDMCP server. For the configuration described here, a line that reads localhost should do the job. To open the XDMCP server to any remote system, * works. (Note that the XDMCP server program doesn’t need to accept remote connections for remote, VNC-mediated logins to work — it’s only the VNC server program that needs to accept such access.)

XDM’s main server options are set in a file called xdm-config. To have the server listen for connection requests from a VNC X server or other computers, you must tell it to listen to port 177 with a line like this:

DisplayManager.requestPort: 177

The default configuration on most systems substitutes 0 for 177, which keeps the server from listening for connection requests. In recent versions of KDM, the same task is accomplished through a line that reads Enable=true in the kdmrc file.

Whatever XDMCP server you use, it’s a good idea to implement firewall rules to keep unwanted individuals out. This is particularly true of GDM, which doesn’t include any access control tools akin to XDM or KDM’s Xaccess file.

For instance, to let only localhost access the XDMCP port, you could use rules like this:

# iptables -A INPUT -s -p udp \ –dport 177 -j ACCEPT
# iptables -A INPUT -p udp –dport 177 -j DROP

To make these changes permanent, enter these rules in an appropriate startup or firewall script.

After you’ve created the firewall rules and reconfigured the XDMCP server, you must restart the server, or at least pass it a USR1 signal to tell it to reread its configuration file. In either case, you must log out of any active X sessions, so the simplest way to do it is to log out and then restart the XDMCP server. You can do this by typing killall xdm (or whatever the server executable name is). Chances are that X will shut down, and you may need to restart your XDMCP server manually.

At this point, the XDMCP server should be up and running, but you won’t be able to make a VNC connection yet. If, after completing the VNC configuration (described in the next section), you can’t get it to work, you can try another XDMCP server — sometimes one will work when another obstinately refuses to function with VNC.

For instance, KDM doesn’t work with VNC under SuSE, although GDM works under SuSE, and KDM works with other distributions. To change the XDMCP server, install a second one and then dig around in your configuration files to change the default. In Red Hat or Mandrake, edit /etc/sysconfig/desktop and add a line that sets the DISPLAYMANAGER variable to GNOME for GDM, to KDE for KDM, or to XDM for XDM. In SuSE, change the DISPLAYMANAGER variable in /etc/sysconfig/displaymanager to gdm, kdm, or xdm. In Debian, place the complete path to the XDMCP server’s binary in /etc/X11/default-display-manager.

Launching VNC From a Super Server

Once XDMCP is configured to run, you need to set up VNC. The simplest way to do this is to launch VNC from a super server such as inetd or xinetd). To do that, you must first specify one or more VNC ports in /etc/services:

vnc-800×600 5900/tcp
vnc-1024×768 5901/tcp

If you want to enable multiple VNC logins using multiple virtual resolutions, specify more than one VNC port, as shown immediately above. Port 5900 (the default VNC port) accepts 800×600 logins, while 5901 accepts 1024×786 logins.

If you only want to support one virtual resolution, you need only one entry. The entry for port 5900 should correspond to whatever you want the default resolution to be. Note that you’re not restricted to traditional resolutions, and in fact, using resolutions slightly lower than is normal, such as 950×700 instead of 1024×768, can be useful. Such resolutions leave room for window borders around the VNC client window.

If your system uses xinetd, as Red Hat and Mandrake do, you need to create a file in /etc/xinetd.d corresponding to the new VNC server. You can call this file vnc, and it should include entries like this:

service vnc-800×600
disable = no
socket_type = stream
protocol = tcp
wait = no
user = nobody
server = /usr/X11R6/bin/Xvnc
server_args = :1 -inetd -query localhost \
-geometry 800×600 -depth 24 -once \
-fp unix/:7100

The server_args line is very important, and must be customized for your system. In particular, you should set the -geometry value to whatever resolution you want to use, and -depth to the bit depth. The -fp option is very important: this sets the VNC X server’s font path. This example sets the font path to unix/:7100, which is appropriate for a Red Hat system. Consult the FontPath entries in your /etc/X11/XF86Config file and enter all the directories referenced there on this line, separated by commas, such as -fp /usr/fonts,/opt/fonts.

If your system uses inetd, the procedure is similar, but inetd uses a single-line for each of its entries. The inetd equivalent to the preceding xinetd example is:

vnc-800×600 stream tcp nowait nobody \
/usr/sbin/tcpd /usr/X11R6/bin/Xvnc :1 \
-inetd -query localhost -geometry 800×600 \
-depth 24 -once -fp unix/:7100

The line shown above has been split across several lines due to column width constraints — it should be entered on just one line in /etc/inetd.conf, even if it’s longer than 80 columns.

Of course, if you want to support multiple resolutions, you need to create multiple inetd or xinetd entries, one for each resolution. Each entry needs a unique port number reference — one for vnc-800×600, one for vnc-1024×768 (or whatever names you specified in /etc/services). You should also change the :1 server argument for subsequent entries. This argument sets the local X server display number, which must be unique for each call. You can also implement firewall rules to accomplish the same goal.

If you want to implement restrictions on who may access the VNC server, you can do so using the usual access control tools for your super server — TCP Wrappers for inetd or xinetd‘s built-in access control tools. Implementing such restrictions is a very good idea. Without them, anybody who can reach your computer and who has a username and password can log in.

After you’ve created the super server configuration, you need to restart the super server, or at least pass it a SIGHUP to have it reload its configuration file. Typing killall -HUP inetd or killall -HUP xinetd should do the trick.

Testing the Result

Figure Two: A Typical VNC/XDMCP login display

At this point, you should have a VNC server running that handles logins for all users. On the VNC server system, type vncviewer localhost. A VNC screen should appear, and after a few seconds, an XDMCP login should appear within it, much as shown in Figure Two.

You should be able to type your name and password and be greeted by your usual X desktop. If this works, try the same operation from another computer. If you have problems with either local or remote access, re-examine your configuration. Check /var/log/messages for error messages from the super server when you restarted it; perhaps there’s a typo in your service specification or some similar problem.

If you see a VNC screen but no XDMCP prompt, examine your XDMCP configuration or try another XDMCP server. One common source of problems is with the font path. Try removing the -fp specification, including all its directories. If that works, try adding the-fp specification back and add directories one or two at a time to localize the problem.

Once the VNC/XDMCP combination is working, many users can log in simultaneously, using the same VNC session number. Instead of being tied to users, the VNC session numbers specify different VNC resolutions or other options, as defined by the Xvnc call in the super server configuration.

When logging into a VNC/XDMCP server, the user’s normal GUI environment is run. GDM and KDM both typically allow a user to select the desktop environment to be used. This may be easier for users to manage than the typical VNC X configuration file. One note: a VNC/XDMCP configuration loses the session persistence of a more traditional VNC configuration — when a user logs out, the session is closed and all the programs the user launched are shut down.

On the whole, a VNC/XDMCP combination is very user-friendly and makes it easy to configure Windows clients to access a Linux system. Initial configuration may take a little longer, but the effort is well worth it.

Roderick W. Smith is the author or co-author of ten books, including Advanced Linux Networking. He can be reached at rodsmith@rodsbooks.com.

Comments are closed.