If your web hosting company knew what they were doing you would not need to use a caching plugin. Caching plugins are really only benificial for shared hosting accounts where your paying less than $10/month, those hosting packages are so pathetic that the caching plugins is the only way to get anymore out of them.
You can search this forum for my recommendations server wise, but a good PHP accelerator like PHP-APC on a web server setup with compression, with a well tuned MySQL database and correctly setup hard drive will put you as cached/optimized as you can be. These caching plugins do caching during PHP execution, which is not the right way to address the problem.
The WP Super Cache plugin may still work fine, the thing is you need to get rid of those comments at the bottom of the feed. Snoop around the settings, maybe you can turn off those comments.