It means the world champions will have to do without their leader for the second clash with the Kiwis in Dunedin this weekend and their first date in the tournament with Australia in Perth the following week.
New Zealand lock Brad Thorn received a one-match ban for the dangerous tackle he produced on Smit in the first-half at the Westpac Stadium that led to the injury.
A scan on the injury showed Smit has suffered a strain of his hip adductor and he will return to South Africa for treatment on Wednesday.
The Springboks have already called-up Western Province hooker Schalk Brits to act as cover and he will join Sharks No 2 Bismarck du Plessis as front row options for coach Peter de Villiers.
Springbok National Team Manager Andy Marinos described Smits injury as "a massive blow". Victor Matfield will captain the side in his absence.
"John is, in my view, the most respected and experienced international rugby captain at the moment, so his absence is a big setback for the team," said Marinos.
"We have two tough tour matches remaining but are fortunate to have in Bismarck du Plessis and Schalk Brits two very good replacements."