Sir Tasker, a recipient of the Victoria Cross for his bravery with the Welch Regiment in northern France in 1944, was one of Britain's greatest war heroes and an eminent retired judge.
He died in the University Hospital of Wales, Cardiff, in the early hours of Sunday.
WRU chairman David Pickering said: "Sir Tasker was one of the greatest living Welshman throughout his fantastic life. He will be remembered as one of our nation's heroes; a man who was an inspiration to so many people in so many ways."
Sir Tasker was elected WRU president in 1993 and held the position until 2004.
When he decided to step down, the WRU created a new post of honorary life vice-patron especially for him. The WRU's patron is the Queen.
WRU chief executive Roger Lewis said: "This is a sad day for Welsh rugby because Sir Tasker was a great follower of the game and a great Welshman.
"He was widely known and respected at all levels of the game but he was especially passionate about his own club Glamorgan Wanderers, where he was president."
Former Wales captain Colin Charvis, who received his first cap from Sir Tasker in 1996, said: "During good times and bad, he was always there with some words of inspiration for myself and the team.
"Although the public didn't see too much of him, he was inspirational for the players and our condolences go to his family and close friends."