A penalty to flyhalf Derick Hougaard saw the lead extended to 17-0 midway through the first half, but the experience of the Wallabies, led by Larkham in his 100th Test match, was enough to see them overcome the spirited Boks.
Connolly said he was satisfied with his team's fight back after a terrible start but said the night belonged to his veteran halves, who were playing their final Test match on home soil.
"This game for us was very much about Stephen and George. The players were very conscious of recognising what they've done for Australia," Connolly said.
"They're probably two of the greatest players that have played the game in Australia so it was an important night for everyone.
"We made it easy for them (Springboks), gifting them a try and then the intercept, and then Hougaard, every time he got the ball he turned us around so we had to play our way into the game."
"We were always confident of working our way back into it but we just started so slowly," Connolly said.
Springboks coach Jake White described the match as an emotional rollercoaster and said a lot of positives came out of the match, but he agreed with Connolly that the experience of the Australia backline was decisive.
"At times I thought we were outstanding and I was enjoying it and other times I was leaning over in the box trying to defend," said White.
"I thought we started well - I wouldn't have put money on Derick (Houghaard) missing those two kicks - we would have been 23-0 ahead and I think that would have been the difference.
"But the one thing the Wallabies have shown is that they never give up, they keep coming back at you and with guys like Gregan and Larkham at nine and 10, they just had too much ball running at our very inexperienced backline for us to keep them out."
The Springboks travel to New Zealand to take on the All Blacks in game five of the six-match series next weekend.