Williams, who is currently playing club rugby in Japan and was due in Australia as a radio summariser, lined up on the left wing as one of four newcomers in the Lions team.
He became the oldest back to play for the Lions in his 15th appearance in the famous jersey, beating rish great Mike Gibson's record.
Gibson, who went on a record equalling five Lions tours and played 68 times for the tourists, was 35 years, 221 days old when he made his final appearance against Bay of Plenty in 1977.
The oldest player to play for the Lions is the Leicester Tigers and England flanker Neil Back, who was 36 years, 177 days old on his final outing for the Lions in New Zealand against Manawatu in 2005 - a game in which Williams scored five tries.
But on this occasion the wing wizard could not help the famous tourists as Tevita Kuridrani grabbed an early try and the Brumbies held on.
"We did all we could out there and ran our socks off but in fairness the Brumbies spoiled every facet of the game that we tried to play off," said Williams.
"The breakdown was a nightmare, they had good pressure on us in defence and it was tough for us to play.
"We would have liked to have had a little bit more ball but it wasn't meant to be.
"I had one team run yesterday. Obviously it is never easy to fit back in and I had to get used to the calls and patterns but I'm not using that as an excuse.
"We trained hard, did our homework and had a look at the Brumbies, our calls and moves but as I said it just wasn't meant to be."
The Lions ultimately paid the price for a lacklustre defence and Williams admits it was a difficult opening 40 minutes before a late rally.
"We were put under immense pressure, as we knew we would be , and it was a good Brumbies side," said Williams.
"We were second best in most facets of the game; tempo - the Brumbies were far quicker to the break down than us and if there was any loose ball they would get to it before us. Their line speed was good, ours wasn't and it certainly didn't make Andy Farrell happy.
"Rightly so he had a go at half-time and we tried to react in the second-half."