Former England and Leicester prop Rowntree will now work alongside John Wells as well as Mike Ford and Brian Smith having spent the last year working with the national academy set-up.
Johnson said: "Graham is an excellent coach and his knowledge of scrummaging is first rate. I enjoyed playing alongside him during our time at Leicester Tigers and I look forward to working with Graham as a member of the coaching team."
Rowntree added: "The last year at the RFU has been very enjoyable and I've learned a lot. I've gained a lot of experience from working alongside the senior squad and as part of the academy structure and it has all been invaluable to my development as a coach."
Elite rugby director Rob Andrew said: "Graham's appointment underlines one of the Elite Rugby Department's objectives of providing opportunities for our coaches from within our own Elite Coaching programmes and structure.
"John Wells and now Graham have come through that system and while we will always keep an open mind on other appointments to the coaching team I'm delighted that Graham has made such rapid progress."