The oral exam is one of two parts that make up the checkride. Aside from demonstrating that you really know how to fly an aircraft, the FAA requires that you be able to demonstrate your proficiency on the theoretical portion of aviation.
Most student pilots fall in confidence because of the presence of the oral exam. We, as private pilots, understand that it's easier to master procedures and processes when it comes to flying an airplane than memorizing a lot of information. Unfortunately, these information are necessary to the FAA's eyes, especially its own laws. How can it trust you to fly an aircraft if you do not show a proficiency in the rules and regulations that govern the craft? That's the necessity of the oral part of the checkride.
Another reason why the oral exam is so overwhelming is the amount of information that you have to digest when studying for the exam. Making things harder is the fact that there is no telling what questions the examiner will ask you to answer. Because of that, there's always an extreme sense of pressure on the shoulders of student pilot who is only looking to earn his certificate and start flying on his own.
There is no way around that. The best that you could do is to prepare for it. As they say, when you're really prepared then you would not have to worry about anything. Get an oral exam review guide. This guide will more or less give you a feel for what you could expect to get when you're finally there in front of the examiner. These reviewers are quite effective in both preparing you and in refreshing your memory.
Do not forget to relax when you're finally there. The reason that you're taking the private pilot oral exam is that your instructor is confident on your proficiency. You just need to be believe in yourself. Self-confidence and preparation will do wonders to your chances of passing the test.