Audacity will record audio only from your input source on your computer, such as a microphone, so in theory you could use it to record an interview ONLY if the person you are interviewing is also sitting with you at your computer, speaking into the same mic.
If that's not the case, for example if you're interviewing someone over the phone or over Skype, then you need another piece of software to capture the audio first, after which you can import it into Audacity and edit it - as Rebecca says to remove static or background hum or hiss, trim bits out or off either end, and add in some music for the intro/exit.
For recording audio there are a number of options - and as many opinions on what is best. Do some Google searching on "free audio recording software" to turn up a long list of options, and take a few of the higher-recommended ones for a test drive. Some of the free programs certainly do a fine job (after all, for audio editing Audacity is a really great application and it's free)....but if you're really serious about producing quality audio podcasts, at some point you may want to spend a bit of money on recording and editing applications.
Personally I use AudioHijack Pro (Mac only software) for recording, then Audacity for editing.
Last piece of advice: You cannot use just any music for your podcast - unless you purchase royalty-free music specifically licensed for podcast use, then it's likely copyright protected and you could get into trouble by using it - lots of folks have found this out the hard way, so be sure to search for royalty-free music and read the license rights carefully.