The Harlequins number eight has failed to recover from a fracture to his right hand, suffered while captaining an England XV to victory over the Barbarians last weekend, and he left Auckland on Sunday.
Easter, 29, is the fourth player to pull out of the squad, following Wasps fly-half Danny Cipriani (broken ankle), his replacement Dan Hipkiss (fractured cheekbone) and winger Paul Sackey (knee).
Specialists initially cleared Easter to fly out with the squad last week - but England officials confirmed the injury to his second metacarpal had failed to settle as anticipated.
"I'm obviously very disappointed to leave," said Easter, who was on his first tour of New Zealand at any level of rugby.
"Like all the players in the squad I was competing for a place to face the All Blacks, which is always a massive challenge and one I was really looking forward to.
"The chances to tour here don't come around very often but it's essential that I get my hand sorted out."
Easter had been favourite to start at number eight against the All Blacks at Eden Park on Saturday but England's tour manager Rob Andrew has decided against calling in any reinforcements.
"I won't be flying in anyone to replace Nick as we brought out seven back row forwards for the tour so we have sufficient cover," he explained.
With Easter gone, Gloucester's Luke Narraway, Leicester's Tom Croft and Wasps duo James Haskell and Joe Worsley are competing for two places in the back row.
Andrew will name his team in the early hours of Tuesday morning and it is likely to contain just a handful of names that started England's last international, the 33-10 victory over Ireland in mid-March.
On top of the injury withdrawals, England have travelled without Phil Vickery, Simon Shaw, Jonny Wilkinson, Josh Lewsey and James Simpson-Daniel who might all have come into Test match contention but are in different stages of rehabilitation.
As captain, Steve Borthwick is a certainty to start while prop Andrew Sheridan, hooker Lee Mears, loose forwards Croft and Michael Lipman, centre Toby Flood and scrum-half Richard Wigglesworth are the only other potential survivors.
Elsewhere, Andrew is set to recall Olly Barkley and Mike Tindall, who suffered a lacerated liver on his last England appearance, against Wales in the RBS 6 Nations, and he could hand a Test debut to London Irish winger Topsy Ojo.
Andrew returned to England's Auckland base-camp on Sunday after watching the All Blacks' 21-11 win over Ireland at a rain-swept Wellington.
New Zealand are also having to rebuild after losing the likes of prop Carl Hayman, lock Chris Jack and centre Luke McAlister to the northern hemisphere following the World Cup.
They were unconvincing in horrendous conditions at the Cake Tin but Andrew was quick to dismiss any suggestion the All Blacks could be an easy target.
"It was a tough game for both sides in very difficult conditions but New Zealand proved again that you cannot write them off as some people have done in recent weeks," he said.
"They were very physical and impressive at the breakdown and their set piece was very efficient.
"Conrad Smith's break that created the try for Sitiveni Sivivatu and Dan Carter's break that led to Ma'a Nonu's try were both top draw.
"The game underlined that we're facing a tough and exciting challenge on June 14 and it's one we're looking forward to."
The All Blacks have called up the uncapped Crusaders prop Ben Franks after John Afoa suffered knee ligament damage against Ireland.
Afoa is out of both Saturday's clash in Auckland and the second Test in Christchurch on June 21.