The Clermont hooker will join up with his Springbok team-mates on Tuesday after his club side lost out in the French Top 14 final against Toulouse this weekend, but believes his side face a massive task to beat New Zealand at home in their opener.
"We've always gone there to win but it's always been difficult," Smit told South African newspaper the Sunday Times.
"Whether our chances are better this time, only time will tell. But I don't think mouthing off is the cleverest thing to do.
"We've all had good incoming tours but haven't really been tested. We've got a new coach, New Zealand have lost a lot of experience in guys like Carl Hayman, Chris Jack, Jerry Collins and (the injured) Richie McCaw, and Australia are also under new management."
Smit also paid tribute to Australia and New Zealand's longevity at the top of the game.
"If you were to think of the top teams in the world, Australia and New Zealand are always there," he said.
"You're being tested against the best in the world in their back yards. We've shown how difficult it is to come to SA and win, so home ground is still important.
"You're also playing against guys who are pretty good at what they do."