McGrath captured four wickets for just 16 runs, including two in four balls when New Zealand were pressing for victory. The Kiwis crumbled to 226 all out in reply to Australia's 236-7.
New Zealand looked to be heading towards an unlikely victory after opener Nathan Astle made a patient 65 and Hamish Marshall weighed in with 76 off 69 balls.
They required 21 runs off the last three overs with four wickets in hand when the Australians suddenly regained control.
New Zealand needed 13 off the last two overs with Marshall still in full flow but the reintroduction of McGrath signalled the end. The lanky paceman finished Marshall's brave performance when the New Zealander played on and then he bowled Daryl Tuffey [ Images ] three balls later.
Earlier, Matthew Hayden [ Images ] (71), Ricky Ponting [ Images ] (61) and Andrew Symonds [ Images ] (53) had all helped Australia give their bowlers a target to defend after the middle-order collapsed.
The world champions had looked to be cruising towards a big total when Hayden and Ponting added 133 for the second wicket but New Zealand all rounder Scott Styris [ Images ] suddenly ripped through the middle-order, capturing 4-40.
Australia dipped to 160-6 but Symonds and Brad Hogg [ Images ] (25 not out) steadied the innings with a seventh wicket stand of 69.
Captain Ponting said: "Glenn was outstanding and we just did enough to win. It's a very hard place to win over here and we found that out tonight."
New Zealand skipper Stephen Fleming [ Images ] said: "It does hurt because it was such a close game. We were out of it, came back really well -- Hamish was outstanding -- but we just lost our way towards the end."
The second match in the five-game series takes place in Christchurch on Tuesday.
A.Gilchrist c Sinclair b Mills 4
M.Hayden b Styris 71
R.Ponting c Vettori b Styris 61
D.Martyn b Styris 7
A.Symonds b Cairns 53
M.Clarke c Marshall b Styris 0
S.Katich c McCullum b Cairns 0
B.Hogg not out 25
B.Lee not out 4
Extras (b-1 lb-1 w-4 nb-5) 11
Total (for seven wickets, 50 overs) 236
Did not bat: M.Kasprowicz, G.McGrath.
Fall of wickets: 1-7 2-140 3-153 4-158 5-158 6-160 7-229
Bowling: Tuffey 8-0-47-0 (nb-2), Mills 9-1-48-1 (nb-2), Cairns 10-0-56-2, Styris 10-1-40-4 (w-1), Vettori 10-0-33-0 (w-2), Astle 3-0-10-0
S.Fleming lbw b Lee 5
N.Astle b Hogg 65
M.Sinclair c Gilchrist b McGrath 0
S.Styris c Clarke b Symonds 14
C.Cairns run out 0
H.Marshall b McGrath 76
C.McMillan st Gilchrist b McGrath 37
B.McCullum c Katich b Lee 8
D.Vettori b Lee 0
K.Mills not out 1
D.Tuffey b McGrath 1
Extras (lb-6 w-8 nb-5) 19 Total (all out, 48.4 overs) 226
Fall of wickets: 1-13 2-16 3-72 4-73 5-113 6-179 7-218 8-224 9-224
Bowling: Lee 9-1-41-3 (nb-3 w-2), McGrath 9.4-3-16-4, Kasprowicz 10-1-62-0 (nb-2), Symonds 10-0-52-1 (w-3), Hogg 9-0-44-1 (w-3), Clarke 1-0-5-0
Result: Australia won by 10 runs.
Man of the match: Glenn McGrath (Australia)