Hi Hardcrab - I'm sure Dive will comment but if I may jump in as well.
The "Hot" is the electrical feed as the name would imply.
The "Neutral" and "Ground" are both at ground potential - an actual stake deep in the ground. They are redundant, but for a good reason.
In the old days, there was only two-prong wiring. As you said, this is all that's needed to complete a circuit. If you had a power drill or toaster oven, the two wires would run the appliance. Often the metal chassis was either "floating" at no potential or was actually grounded through the "neutral" wire. Perfectly acceptable - as long as the house wiring integrity remained intact. However, if the terminal holding your neutral-wire (either in the receptacle or the fusebox) became corroded or broken, there was no ground anywhere to be found - The whole system, including the chassis, was now at hot potential. Spill some milk on the floor and touch your toaster and you were dead - which happened often enough.
So they came up with the "third prong" - an extra - redundant ground that had to have a separate wire-run all the way down to the stake in the ground, but was still at the same potential as the neutral. The new ground-prong on the plug would be longer, so it would be the first contact made - and it would be connected to the chassis of your appliance. The "neutral" would be exclusively attached to the motor or heating coils or whatever, to take up the juice from the hot. This new ground wire is for added protection only. Lose the neutral and you're still ok. With everything being plastic nowadays, the ground-wire often deadends in the appliance, but it's still part of the wiring.
So yeah, electrically speaking, completing the two-wire circuit, putting breakers on the ground, etc. all makes things operate fine. But the safety features are all now kaputz. If you're ground breaker decides to blow and you're still feeding juice, it's 1930 again so watch out - and don't spill any milk
Cheers,
~Bob