The home side had looked to be in control after throwing everything at their visitors for 70 minutes, scoring the first two tries in the process.
However, the appearance of the All Blacks substitutes - particularly Keven Mealamu - made a significant impact on the match at a time when it seemed to be slipping away from them.
The Boks opened with two penalties from Percy Montgomery - while Dan Carter missed two for the visitors - to give them an early 6-0 lead.
But Carter managed to slot his next two to even up the scores just before half-time.
The Boks then took the initiative on the break as they drove their way towards the line and Schalk Burger ended up at the bottom of a pile of bodies to see them go in at the interval 11-6 ahead.
Aaron Mauger dropped a goal early in the second period to bring the scores closer together, before Jaque Fourie put in a half-tackle on Mauger, who popped the ball up to a flying Butch James to intercept and run 40 metres to score.
Montgomery converted to send the Boks 18-9 in front and when Carter's next penalty attempt hit the post, it seemed nothing would go right for the All Blacks.
Carter and Montgomery swapped penalties again and, with 10 minutes to go, the 21-12 scoreline looked secure.
New Zealand then sped up play, shifting left and right and, as referee Allan Roland missed an apparent knock-on, McCaw latched onto the ball and went over to score. Carter converted to bring his side within two points.
From the restart, another wave of attack saw winger Rokocoko break free and run 30 metres to score against a failing defence, and silence the 50,000 crowd in the process.
The Boks tried desperately to come back, but found their time had run out as the All Blacks claimed a significant victory away from home.