So when Horan says Elsom is not the right man to lead his country, it is no surprise that the Australian public are willing to listen.
Elsom has skippered the Wallabies since replacing Stirling Mortlock in 2009 but he is yet to recapture the form that saw him named Wallaby of the Year in 2008.
The powerful blindside was one of the globe's form players during his one-year stint with Leinster that ended in 2009, with the 28-year-old winning 11 man of the match awards in his last 13 Magners League outings, being named European Player of the Year and featuring in ERC's Dream Team to celebrate 15 years of the Heineken Cup.
But although Elsom has never truly been in danger of losing his starting spot on the international stage, Horan believes he needs to be relieved of the captaincy if the Wallabies want to see him at his inspirational best.
"I believe the captaincy burdens Rocky too much," Horan told AAP.
"The only way I think you see the best from Rocky Elsom is when he doesn't have the (c) or (vc) next to his name.
"I want to see the same Rocky Elsom that played over in the UK.
"If I was Robbie Deans I'd be giving the leadership to a couple of other players and let Rocky get back into form because, to win the World Cup, you need Rocky Elsom to be playing at 100 per cent."
Elsom has been absent for the first six weeks of the new Super 15 season through injury and is not expected to return to action for another month.
A place in the Brumbies side to face the Cheetahs on April 29 would leave Elsom with just six matches to prove his form and fitness prior to the international season which begins against Samoa on July 17.
For Horan, whose stellar Test career started just after the 1989 Lions tour and ended the year before the 2001 series, expecting Elsom to find his best form and lead his country in such a short space is simply asking too much.
"At best he's only going to play five or six games and is that enough to get him where you want him at the start of the Test season?" continued Horan.
"I'm sure he's good enough to play for the Wallabies and be captain but is that a distraction to him and preventing him from being the best player he can be?"