The Boks were second best for large periods of the Pool D encounter, with Wales dominating territory and possession, but a late Francois Hougaard try saw them home.
Toby Faletau's first international score had given Wales the lead mid-way through the second-half and the Dragons had the chance to win the game late on but a drop goal from Rhys Priestland and a penalty attempt from James Hook both failed to hit the target.
Hook also saw what would have been a critical penalty attempt ruled out in the first half after referee Wayne Barnes adjudged that his kick had drifted wide when television replays suggested it may have remained inside the posts.
Wales got off to the worst possible start when Frans Steyn fended off Lions duo Shane Williams and Hook to score in the corner in the opening moments.
Morne Steyn converted from out wide and then added a penalty to hand the champs a 10-0 lead. But Wales hit back with two Hook penalties to trail 10-6 at the break.
The gap was cut to just a single point thanks to a third Hook three pointer before young No8 Faletau barged over for a try that needed confirmation from the TMO.
Hook added the extras and Wales had a deserved lead with 25 minutes left to play in New Zealand's capital. Led by man-of-the match Sam Warburton and fellow back rowers Dan Lydiate and Faletau, Wales continued to pressurise the South African line, with only untimely handling errors preventing them from moving further ahead.
The Boks, boasting the most experienced side in Test history, refused to buckle, however, and regained the advantage when Hougaard took a clever line off scrum-half Fourie du Preez 10 minutes later.
Wales worked their way into scoring chances but failed to capitalise and the Boks successfully ran down the clock when Lydiate was penalised late on.
Victory would have handed Wales just their second-ever win over South Africa in 106 years of trying but the performance itself will have given Gatland plenty to enthuse about ahead of their remaining group games against Namibia, Samoa and Fiji.