Holmes, who had never won a major title before, timed one minute 56.38 to win a blanket sprint to the line, while Mutola, the dominant force of the event for around a decade, went away empty-handed.
Silver went to Hasna Benhassi of Morocco with Jolanda Ceplak of Slovenia taking the bronze. Both were given the same time, 0.05 seconds behind an incredulous Holmes.
"I can't believe it. I didn't realise I had won," the 34-year-old Holmes told reporters.
"I knew I was in front but had to see the replay twice to make sure. I was just digging in."
Mozambique's Mutola, who finished fourth just behind, had hoped to become the first woman in Olympic history to win two 800 metres golds in succession.
She looked stunned by the result. As Holmes held her hands to her face in disbelief, Mutola stormed out of the stadium without a word or a wave.
Mutola, who has also won three world outdoor titles and six indoors, came to Athens with question marks over her form after a rare defeat in the lead-up to the Games which she blamed on a hamstring injury.
Her defeat, however, was a major shock since most of her opponents, including Holmes and Ceplak, are in awe of both her talent and her brooding menace.
Mutola started in lane five, with Ceplak and Holmes immediately inside her.
At the break, all three were well back, with Ceplak fifth, Mutola immediately behind and Holmes, as so often, right at the back as American Jearl Miles Clark set the pace.
Mutola overtook the Slovenian at the bell, shadowed by Holmes, and the defending champion looked perfectly placed in third with 200 to go.
Her famed kick, however, failed to materialise. She got to the front but, clearly labouring, saw Holmes draw up to her shoulder.
Holmes, tying up in the final metres, just held on while Ceplak, who had looked out of it at 150 metres, stormed back to deny Mutola even the bronze.
More from rediff