Scotland's summer tour has now been reduced to just three games, against Australia, Fiji and Samoa, and they will take a smaller 28-man squad.
"We have had confirmation from the Waratahs that due to injuries and the requirements of the Wallabies squad they would be unable to fulfil the fixture," said Scotland team manager Gavin Scott.
"We have looked into alternatives but, unfortunately, nothing else fits into our tour schedule."
The Lions will face the Waratahs during the 2013 tour of Australia.
Scotland's summer tour:
Tuesday June 5 - Australia in Newcastle, kick-off 7.30pm local (10.30am BST)
Saturday June 16 - Fiji in Lautoka, kick-off 2pm local time (3am BST)
Saturday June 23 - Samoa in Apia, kick-off 3.30pm local time (3.30am BST)