I have 3gbs of memory in my system and a fast processor, yet my system would start choking when I used to much memory so I sat out to fix it.
What I learned is that there's a setting in /etc/sysctl.conf called vm.swappiness that controls to what level the Linux kernel should use disk based virtual memory from 1-100.
In Ubuntu, this is set to 60 by default, which roughly translates to 60% of all virtual memory being disk based, and therefore, slow.
To change the value simply edit the file (e.g. "gksu gedit /etc/sysctl.conf") and add the line "vm.swappiness = 1" (change "1" to what you think is appropiate) - the change will take effect on next reboot.
If you want the effect now, you can run "sudo sysctl vm.swappiness=1" in a terminal.
A very welcome effect of this change is that Nautilus is now many, many, many times faster!