Waugh told a hastily arranged news conference at the Sydney Cricket Ground that he would call it quits after the fourth Test with India, starting in Sydney on January 2.
"The upcoming Sydney Test will be my last for Australia, should I be selected," Waugh said.
"My form and fitness suggests I could go on ... but all good things must come to an end.
Waugh, 38, is the most capped player in the sport's history with 164 Test appearances and the second highest run scorer with 10,660 runs.
He is the most successful captain in Test cricket and is second on the all-time list of century makers with 32 hundreds.
Waugh had previously said he wanted to retire after Australia's tour of India in September next year but brought it forward to finish on a high and spend more time with his family.
"I want to leave the game on a high and I feel the time is right now," Waugh said.
"It's been in the back of my mind for a little while now but Sydney's a great place to finish."
Waugh made his Test debut against India in 1985 and has climbed every mountain in the game, including captaining Australia to victory in the one-day 1999 World Cup.
But he has been under increasing pressure from the national selectors and local media to hang up his bat and give a younger player a chance.
DEFIED CRITICS
His career seemed to be on the rocks after a barren run last year but he defied the critics with a stunning century at the start of this year in the final Ashes Test against England, also at his home ground in Sydney.
That hundred seemed to revitalise Waugh who has scored another four centuries since then and is arguably in the best form of his career.
"I'll never forget what happened in Sydney last year, no-one will ever take that away," Waugh said.
"There was a temptation to finish up then but it wasn't about the one good moment. I wanted to go when I was playing well and playing consistently.
"I'm quite happy with what I've achieved in the past. I feel I'm playing consistently and I'm in good form and I wanted to go when there was still some fire in the belly."
One of the most respected figures in the game, some of Waugh's greatest moments were his 200 against West Indies at Sabina Park in 1995; his twin centuries against England at Old Trafford in 1997; and his 120 against South Africa at the 1999 World Cup.
He has also featured in many high-scoring partnerships including an unbroken 464 with his brother Mark for New South Wales in Perth in 1990-91 and another of 385 with Greg Blewett against South Africa at Johannesburg in 1996-97.
A fiercely competitive player, Waugh played 325 limited-overs internationals for Australia but lost his place in the team in February 2002.
He was dropped from the Australian Test team in 1990-91 to allow his brother Mark to make his Test debut, but remodelled his game, putting the hook shot away and adopting a more controlled approach, to become one of the prized wickets of world cricket.
Mark retired a year ago when he was left out of the Australian team for the Ashes series.
More from rediff