Hogg, a two-time Lion who featured in the victories over the New Zealand Provincial Barbarians and the Crusaders, suffered a facial bone fracture in the match in Christchurch.
Click here to follow the Lions clash with the Highlanders live!
The British & Irish Lions medical team have been working in consultation with Glasgow Warriors and Hogg will return to Scotland for further treatment.
British & Irish Lions Head Coach Warren Gatland said: “It is hugely disappointing for Stuart to have to return home early and for us to lose one of our touring party.
“Stuart has been a key member of our squad on and off the field and it is disappointing to see injury cut short his time with us in New Zealand.
“We all wish him the best with his recovery and we look forward to seeing him back in action in the future.”
Hogg said: “I’m gutted to be leaving to be leaving the Tour at this stage.
“It is an honour to have been involved in a second British & Irish Lions tour and I have hugely enjoyed my time with the squad and I’m very excited about the potential of this group.
Download our Official App for iPhone or Android!
“I’m disappointed to be leaving in this way but wish all the squad the best in the coming weeks and will be cheering them on from home.”
There will be no replacement call-up at this time.