The 58-year-old is set to take over the Premiership club when he returns from coaching duties under Sir Clive Woodward with the Lions this summer.
Warren Gatland is the current director of rugby for the English and European champions but will leave at the end of this campaign.
McGeechan led Scotland to the Grand Slam in 1990, was in charge at Northampton for six years up to 1999 and has coached the Lions on their 1989, 1993 and 1997 tours.
He said: "While I am delighted to accept the new post at London Wasps this has been a difficult decision for me and everyone in Scotland may rest assured that my passion for the game here remains undiluted."
SRU interim chief executive Fred McLeod admitted losing the Lions coach was a big blow.
McGeechan, who succeeded Jim Telfer as director of rugby at Murrayfield after the World Cup in 2003, stepped down as Scotland coach to let Matt Williams take control.
McLeod said of McGeechan: "For many of those in Scottish Rugby who recognise what he has achieved, this is extremely regrettable news.
"Geech's contribution on the field as a distinguished player, then as a coach and in his post as director of rugby has been immense.
"He will remain in post until he departs on the British & Irish Lions tour on May 13 and will be fully involved in all aspects of the Union's business in the meantime."