Without a feed for us to look at we're only going to be able to guess what your problem could be.
First guess, you have a page, post or category with the word 'podcast' as the slug value. WordPress only allows you to use the slug keyword 'podcast' once, so if you've used it for a static page or a one word blog post name you're going to get a 404 error. Easy test, go to your site and add /podcast/ and see what it loads, if it loads a page/post then that's the conflict. Example:
www.example.com/podcast/Second guess, you could have a plugin conflict. Disable your plugins one by one and test the feed URL each time. You will want to clear your browser cache every time you disable a plugin, as the 404 page may end up being cached in your browser. If you feed works after disabling a specific plugin, then you know there's a plugin conflict. Please let us know what the plugin is so we can do further testing.
Third guess, Permalink structure you are using is custom and doesn't work well with extended feeds. If this is the case, try accessing your podcasting feed without the permalink by using the following as an example:
www.example.com/?feed=podcast. If that works, your permalinks are messed up. We've seen permalinks in WordPress not work, we can't figure out why but the simple fix is to use the ?feed=podcast instead.