Also, your media hosted at blubrry.com or at archive.org means nothing to your feed or blog, that is not the problem. Now if you were hosting your media on the same server as your blog, then yes that would compound the problem, but even then the items per feed setting is too big, and it would require a pretty beefy server (which if you had, you would also have a server administrator constantly improving and managing it).
Even then, if you were able to deliver a really large feed quickly from your web server (which your currently not), you're still throwing a lot of data into someone's feed application which slows that down as well. It's like trying to put 100 gallons of gas in a Honda Civic, you can do it if you add extra fuel cell tanks all about the car, but you're going to piss a lot of people off.