Phillips, who was one of the pivotal figures on the 2009 Lions tour, was dropped by the Ospreys as it became clear that his future lay away from the Liberty Stadium, while a whole host of former internationals called for him to be replaced in the Welsh side for both the autumn series and the RBS 6 Nations.
Worse was to follow for the former Scarlets and Blues player as he was suspended from Wales' pre-World Cup squad for 10 days following a late-night altercation with bouncers outside a fast food restaurant in Cardiff last month.
But Phillips, who was recently re-instated into the Welsh party that are now training in Poland, insists he does not see his 2010/11 campaign as a failure.
"It's a bit harsh at times," Phillips said of the widespread criticism.
"I'm always looking to improve and, bar a poor performance against England, I'm happy with my season.
"There's a lot more to my game than my pass. I bring positives in attacking and defensive areas, in my work-rate and my kicking ability.
"As a nine you rely on other people around you to play well. When people are critical they like to pick out one or two people, but it's a team game so it's very hard to criticise individuals."
Phillips is heading to France after this autumn's World Cup and he admits he is looking forward to a new challenge in different surroundings.
Having signed for big-spending Top 14 outfit Bayonne for the next two years, Phillips is in positive mood ahead of his move away from the Ospreys.
"It will be really good to have a fresh start - that's definitely what I need," Phillips told Rugby World magazine.
"People in Wales have been great to me over the years, but sometimes I haven't been able to cope with being recognised as well as I should have done and things have been difficult.
"I've lived in Cardiff for 10 years now and have played rugby my whole life in Wales, for three different regions, so I want to experience different things.
"I don't want to look back and not to have played anywhere else. Rugby is a short career and this is a great opportunity to go and live in France.
"No matter what happens over there, I'll know I've given it a shot. Leaving friends and family will be tough, but I know I'm going to love it there.
"Bayonne is a top-notch place with nice people, a great stadium and awesome supporters. As a player you want to play in a top league in big atmospheres and big games. It's going to be a great experience and a massive challenge, but that is what you want in life."