For some time I've been meaning to take a closer look at NoMachine NX because at first glance its pretty hard to figure out what's going on there and how the various bits and pieces fit together. Of course, there's some context here - there are a number of things I'm trying to figure out:
Another caveat is that I haven't actually played with NX yet, this first impression is just from looking at the code and documentation.
The core ideas behind NX are to compress X protocol traffic using differential compression and also reduce the number of X protocol roundtrips across the network by using a proxy X server. The diffential compression idea was first introduced by the Differential X Protocol Compressor (DXPC) project back in 1995. Since then, NoMachine have expanded on the idea and built a commercial (but mostly open source) product.
NX also has a proxy X server which serves to dramatically reduce the number of X roundtrips on the network. In place of the proxy X server NX also allows you use a VNC or RDP client and so display non X based sessions over NX.
NX doesn't just blindly compress the X protocol stream and hope for the best. Better results are achieved by virtue of the fact that the algorithm understands the semantics of the messages being compressed. Using this knowledge, it is possible to discard useless information, cache parts of the message which are likely to be re-usable, apply different image compression algorithms based on the type of image and so forth.
Compressing a given X protocol message begins by first splitting the message into two parts - a fixed length identity part and a variable length data part. The example given in the NX documentation is the PolySegmentRequest which consists of a header followed by a list of segments. In this example the header is the identity part and the list of segments is the data part.
The algorithm then looks up the identity and data parts separately from the message cache using an MD5 sum to index the hash. However, the MD5 sum of the identity part is not calculated using the entire contents of the identity, but rather a specific set of fields which are chosen differently for each message type. These specific fields have been carefully chosen to include only those parts which are likely to not change across message instances, thereby increasing the potential for more cache hits.
If a matching message was already found in the message cache, a message indicating the location in the cache of the message is sent in place of the actual message. If the message was not found to be in the cache, the message is first added to the cache and then sent to the recipient along with its location in the cache. This way the encoding side manages the remote peer's cache explicitly rather than having to query the peer about the state of its cache.
A point worth noting about the message cache is that on the encoding side the cache need only contain MD5 sums and on the decoding side the cache contains only the actual message contents and not the MD5 sums. This cuts down on the amount of memory required to maintain the cache and also the amount of processing the decoder must do since it does not need to MD5 sum the messages.
Non-cached messages aren't sent in the standard X protocol format, the messages are sent using NX's differential encoding - whereby encoder does its best to compress the message as much as possible by discarding padding and implicit data. An example is with the PolySegmentReq message it can use co-ordinates relative to the previous co-ordinates to try and get down to 8 bit co-ordinates rather than the full 16 bit co-ordinates.
Integer values like XIDs are also cached using a simple "move-to-front" algorithm. This way, successive messages containing the same XIDs can encoded the XID with as little as a single byte.
Images are sent using any one of a number of packed encodings negotiated between encoding and decoding sides. A nice side effect of this is that the encoder can choose to use a lossy encoding (like JPEG) for large images and still use a lossless encoding so as to not screw up things like text. NX also implements extra cleverness for handling large images on low bandwidth connections - large images may be split up into smaller chunks and streamed with other X messages interleaved with the chunks, thereby allowing the connection to be efficiently shared between mutliple clients.
First a bit of NX terminology:
NX can be setup to be used in a variety of different ways. The first way is where you interpose a pair of proxy peers between the X clients and the X server:
In this configuration X protocol traffic between the clients and the server is cached and compressed by the proxies. However, this configuration does nothing to reduce the number of roundtips across the network. However, you can also introduce the NX Agent (a proxy X server) on server:
In place of NX Agent, you can use NX Viewer or NX Desktop to allow VNC or RDP to be used:
When you go to download the NX source tarballs from NoMachine you'll see there are in fact 17 source tarballs which is slightly bewildering at first. Here's a look at how the thing is split up and which bits are where:
Mark McLoughlin. July 7, 2004.