Matt Williams' new look Scots side were beaten 23-10 by Wales in their opening game of the RBS 6 Nations Championship in Cardiff.
And with the world champions next up at Murrayfield, the Glasgow flanker feels the margin of defeat could be even greater if the Scots put in the same type of performance again.
He said: "Obviously the boys are very disappointed in the result against Wales, but we are still pretty upbeat.
"But England are the world champions and we have to front them up or we could get beat by 50 or 60 points. That is a real danger.
"If you see England play, they are obviously a very good defensive side and they can attack from anywhere on the pitch, so we definitely have to be on top of our game to live with them.
"The problem is we're too nice and we get bullied by teams and we need to get a harder edge about us.
"I'm pretty sure we will have that harder mentality against England because Italy fronted up against them and I'm sure there were a few sore England players after the match.
"But in saying that England still had the skill and ability to score 50 points against them."
Mather claimed the Scots do not necessarily lack the bulk in key areas and insisted one of the biggest problem against Sir Clive Woodward's men will be the sheer quality of players England have at their disposal.
He said: "I played against these guys in the Premiership and they are not that much bigger, although maybe they are slightly better technically.
"In the past great rugby sides have been built around five or six world-class players at best, but England have 15 world-class players. This is why they are the best team in the world."