I have a VM which has 2GB Ram, (full specs) And I am setting up a site which has one table in particular with over a million records. There's little or no usage of this particular database (perhaps once or twice a day) but simply running mysql grinds the whole server to a halt. I've looked through the top results but nothing is really denting the CPU however the memory seems to be the issue. The site isnt even live of taking requests yet. the memory situation looks like this:

# free -m
             total       used       free     shared    buffers     cached
Mem:          2006       1880        126          0          3         53
-/+ buffers/cache:       1823        183
Swap:         2047        345       1702

Are there any good pointers to tune mysql to stop hogging the system memory?

Thanks very much

EDIT: (requested by 8bit):

http://tny.cz/b41a0b12