Giteau, who joined the squad in camp in Coffs Harbour on Friday but did not train, first injured his back against the All Blacks in Sydney.
He then took another knock to the same area in Perth last weekend against the Springboks.
In a statement, Wallaby coach Eddie Jones said: "Matt showed an enormous amount of courage to get through the Test in Perth and while he's made some improvements, it's not enough to ensure he'll be available for selection.
"He will return to Canberra [on Saturday], and continue to receive treatment on the injury.
"Depending on how he responds to treatment he could be out for anywhere between one to five weeks, so the rest will do him good."
Fly-half Elton Flatley will miss the rest of the season and the mini-European tour in October-November suffering side-effects from concussion so Jones may consider adding another inside back to the squad.
In other injury news, reserve hooker Stephen Moore is the main concern at the moment with an injury to his lower leg.