The processor on the Raspberry Pi isn't all that fast, so why not offload it? I have a HP ML350 that is mostly idling, except when I need to use PostgreSQL, so I thought to set it up as a desktop server for my Pi.

I set up a RDP server since it works well with Linux and also Windows. First thing's first, install xrdp and all other dependencies:

apt-get install build-essential libpam0g-dev libssl-dev x11vnc vncviewer fakeroot checkinstall x11vnc xrdp rdesktop

Then, install your desktop of choice. I chose XFCE since it's pretty lightweight

apt-get install xfce4

I've had problems getting it to start X remotely, so I fixed it with the following. Do this in the user's account (/home/pi for example)

touch .Xauthority
chmod 600 .Xauthority

And that's it!. Start up your Raspberry Pi, install rdesktop, and connect to your server:

rdesktop -g 1024x768 -P server_address