The ex-England international died after a long battle against Alzheimer's.
Williams had been having treatment for the illness at the Harts Leap Hospital in Sandhurst but lost his long battle earlier today.
The Old Millhillians scrum-half travelled to South Africa with the Lions in 1955, playing eight matches on a memorable tour.
Williams featured against Transvaal Universities, Orange Free State, South Western Districts, Rhodesia, Central Universities, Eastern Transvaal, North-Eastern Transvaal and Natal, scoring five tries in the process.
Two of those scores came in the 21-14 win over Central Universities at Kingsmead.
Williams, who won nine caps for his country between 1954 and 1965, was unbeaten as a Lion.
The then 23-year-old won seven of his eight games in South Africa, with the only blemish on a potential 100-per cent record being the 17-17 draw with Eastern Transvaal.
Former England and Lions international Dickie Jeeps described Williams as 'a superb scrum-half who represented his club, county and country with distinction.'