Mike Ruddock's men dominated the first half but trailed 7-6 at the interval after Lote Tuqiri had scored a breakaway try for the Australians.
They slipped further behind when Nathan Sharpe charged over the line before mounting a thrilling comeback.
Welsh forward dominance earned a penalty try before Shane Williams combined brilliantly with Gareth Thomas to put the home side ahead.
The Wallabies hit back to set up a grandstand finish, with Chris Latham touching down in the 73rd minute.
But Wales held on to claim their first victory over a Tri-Nations side since 1999 and their first win over Australia since the third-place play-off at the 1987 World Cup.
Wales made an enterprising start and played with a freedom and confidence which was the backbone of their success last year but had been missing so far this autumn.
Gareth Thomas, Matthew Watkins and Shane Williams sparked thrilling attacks but the Welsh finishing was disappointing and they were restricted to just a fourth-minute penalty from Stephen Jones.
The Wallabies had offered very little in attack when they stung Wales in the 14th minute with a clinical break. Mat Rogers burst into space and slipped the scoring pass to Tuqiri.
Rogers added the conversion but then missed a relatively simple penalty as Wales, led by Thomas and Dafydd James, defended with heart and bravery.
Jones brought Wales back to within a point by the interval with his second penalty, but within two minutes of the restart Australia had scored again.
Shane Williams was snagged in his own 22, lost the ball and after Latham had slithered through James' grasp Sharpe bulldozed his way over the line.
Wales hit back brilliantly and earned a penalty try after Australia's pack had crumbled at three consecutive five-metre scrums.
And then Shane Williams combined brilliantly with Gareth Thomas to finish off a superb counter-attack as Wales surged ahead.
Jones' conversion hit the post but a 45-metre penalty just two minutes later meant Wales had turned an eight-point deficit into a 21-14 lead.
Rogers and Jones swapped penalties before Tuqiri sliced through the Wales defence and chipped forward for Latham to score Australia's third try.
Crucially, Rogers missed the conversion and Wales held on for a famous victory.