Using swap is what makes it grindingly slow. If it was out of memory entirely, it would just not work, rather than running slowly.

If the current working set (the data currently being touched by the foreground applications you are using, and all background services that they depend on) is larger than your physical RAM then performance will always suck, however much swap space is available, because swap on disk is at least ten times slower than RAM.