As gas builds up in the stomach it creates pressure, kind of like a balloon. Fortunately, your stomach won't pop when it's filled with too much air. Instead, the gas rises to the top of the stomach, travels through the esophagus and comes out of the mouth. Sometimes, burps taste like something you ate or drank earlier, which is a good indication that they probably smell like whatever you ate, too.
The sound of a burp comes from the upper esophageal sphincter, a muscle above the esophagus that vibrates when gas passes through it. In America, it is polite to burp quietly by keeping your mouth shut and following up with an "excuse me." Relaxing the back of your throat as the gas comes up can also make a burp quieter.