The 27-15 defeat came just a week after a 19-0 hammering by the All Blacks in Cape Town and ensured the home side were roundly booed off the field by their own supporters.
"We're the Springboks, we're World Cup champions and we've lost two home games in succession," said Matfield, who took over as South African captain after John Smit suffered a groin injury in the opening game of the competition.
"For us, to play like that is unacceptable.
"If you could look into the dressing room, you'd see the players are feeling worse than the supporters are.
"This is our work, our passion and our life."
Matfield and his side have the chance to make some sort of immediate amends when they face the Wallabies once more in their final Tri Nations fixture in Johannesburg this coming Saturday.