The former England fly-half will now take on the job of backs and skills coach at Franklin's Gardens, with New Zealander Peter Sloane taking Grayson's former position.
Ex-Auckland boss Sloane was already part of the Saints' coaching staff, while club chairman Keith Barwell has confirmed that discussions to appoint a new director of rugby have started.
Saints' defence coach Frank Ponissi and technical analyst Paul Larkin, meanwhile, will leave the club.
Northampton will play in National League One next season after finishing bottom of the Premiership by one point behind Worcester.
Though Saints beat London Irish on the final day of the regular league season, Worcester's 22-7 victory over Saracens meant they stayed up.
Northampton directors met on Monday to thrash out a way forward, with former Australia supremo Eddie Jones and ex-Italy coach John Kirwan among those who have been linked with the director of rugby post.
Barwell said: "We maintain our belief in the potential of Paul Grayson as a coach for the future, but accept his appointment as Saints head coach was premature despite his success in the second half of the previous season.
"The board recognises that we need to strengthen our coaching team with the appointment of an experienced world-class coach if the club is to achieve its objective of consistently challenging for top honours.
"A shortlist of potential candidates is being compiled, and we have had some extremely positive conversations with several world-class coaches.
"I believe we may be in a position to make an appointment in a month or so's time, but that should not stop any likely candidate from getting in touch with our chief executive.
"It is important, however, that there is no delay in our preparations for the coming season while our search for a director of rugby proceeds.
"In Peter Sloane, we already have someone with great experience, and we believe he will give us a harder edge and help us to achieve our immediate objective of getting straight back to the Premiership."
Jones, who has already sampled the Premiership in an advisory role with Saracens, is currently coach of Super 14 strugglers Queensland Reds.