Williams has been sidelined for three weeks after requiring an injection to manage the problem.
Wales head coach Warren Gatland is already without leading scrum-half options Mike Phillips and Dwayne Peel plus Grand Slam winners Gavin Henson, Lee Byrne, Martyn Williams and Huw Bennett.
But he has decided not to call up a direct replacement for Williams because Gareth Cooper will be free to depart with the squad on May 29 after Gloucester failed to reach the Premiership final.
Cooper and uncapped London Irish scrum-half Warren Fury will travel as the only specialist scrum-halves with winger Shane Williams to be used as emergency back-up.
Wales physio Mark Davies said: "Andy suffered a recurrence of a back problem during training last week and as a consequence he has been deemed unfit to tour."
Shane Williams, one of the world's leading Test wingers, began his rugby career as a scrum-half and briefly returned to the role when Phillips was sin-binned during Wales' Triple Crown victory over Ireland.
Gatland is set to add Richard Hibbard to the touring party after the Ospreys hooker replaced Bennett in the training squad this week.