Oliver has been in a race to get fit after injuring his calf while playing for the Highlanders Super 12 franchise against the Crusaders on May 7.
The veteran hooker had anticipated being out of action for six weeks but his recovery has been slower than expected and he is still at least two weeks away from full training.
That means his first appearance for the All Blacks is now likely to be against the Lions in the second Test in Wellington on July 2.
All Blacks coach Graham Henry said the slow nature of Oliver's recovery had been a source of frustration for the 43-Test hooker.
"He's frustrated I think is fair, but he's handled it very well," Henry told One Sport.
"That's just the reality, he's done everything he can to get on the track. It's taken longer than he had hoped and what the original diagnosis had suggested.
"You have just got to get on with it really and so does he. He has been helpful in other areas, he's helping the coaches, the forwards coach with the lineout and the scrum."
Number eight Mose Tuiali'i is also in doubt for the first Test in Christchurch on June 25 as he continues to struggle with a pinched nerve in his back.
There is better news regarding Jono Gibbes, however. His foot injury is being managed and he should be up for selection for the first Test.
Henry, though, has not ruled out resting the flanker-cum-lock who was outstanding for the New Zealand Maori in their win over the Lions last Saturday, at some point during the year.
"At some stage he's going to have an extended break from the game, whether it's after the Lions series, during the Tri-Nations and the NPC or at the end of the year," Henry said.
"So we can manage him through as best we can but at some stage he's going to have two or three months off."