After losing 36-0 to South Africa at the Stade de France Brian Ashton's men face the likely ignominy of becoming the first world champions to go out of the tournament at the group phase if they lose to Samoa in Nantes.
But second-row Kay woke up the morning after the terrible night before determined to erase England's worst World Cup defeat from his memory.
He said: "It's important that the guys don't let the negativity which comes with losing get to us. If we do we would take the field against Samoa having beaten ourselves.
"They'll be rubbing their hands now thinking this is our chance to beat England and it's down to the guys to knit together, batten down the hatches and not let that negativity get to us.
"We have to turn up against Samoa and put in the performance of our lives.
"It's completely knockout for us now. If we have another slip-up, if we don't play to our ability, we will be going home."
Kay admitted many of his team-mates were "close to tears" in the changing rooms after the defeat and were desperate to make amends for their poor showing so far.
He said: "If we went out it would be the lowest point of every guy's career. And for the fans who have travelled over here and for everybody back home in England it would be unthinkable. The players are fully aware of what the next two games mean to us.
"We've got to knuckle down as a 30 and forget what happened in 2003."
That means a week of hard work and studied analysis to work out what is missing.
"That's the million dollar question," said Kay. "I think the area we need to focus on is the breakdown area. We need to clear the ball quickly so that we can get some momentum. The last couple of games too many guys have been receiving the ball standing still.
"As soon as sides such as South Africa and New Zealand seize on turnover ball they rip you apart. The key is not turning the ball over. Against such sides they will hurt you."
Meanwhile, forwards coach John Wells said: "There was no lack of effort at any point during the game and no lack of support from the crowd who were superb.
"We're disappointed with the score and the margin of defeat. We never looked like rattling the scoreboard. We have a lot of work to do to get it out of the system."