They took the Springboks to the wire at HQ but came up agonisingly short with Alberts' try proving the difference.
The defeat ends any hope of England finishing in the IRB's top four ahead of next month's World Cup draw and they face the fearsome All Blacks next week.
Toby Flood missed an early penalty attempt but made no mistake minutes later to boot England into the lead.
It did not last long as Pat Lambie hit-back for South Africa but both sides struggled to maintain any continuity or momentum in testing conditions at Twickenham and England saw Flood depart to the blood-bin.
The duelling fly-halves continued to exchange shots at goal as the Springboks edged into a 9-6 lead after 25 minutes.
Both sides threatened in attack but were hampered by unforced errors and the dogfight continued until the interval without further scores.
But South Africa surged further in front after pouncing for a fortuitous try within minutes of the restart.
After winning a penalty Jean De Villiers opted for the corner and the Springboks pushed for a try.
Juandre Kruger lost the ball as he charged for the line and Ben Youngs hacked through to clear but the ball bizarrely rebounded off JP Pietersen.
The loose ball was fumbled by Tom Wood and fell to Alberts, who flopped over the line for the crucial try which Lambie converted.
The try sparked the game into life and both sides were caught in a brawl as temperatures began to rise.
South Africa piled the pressure on England, with Ruan Pienaar and Patrick Lambie combining to good effect. But Manu Tuilagi intercepted a wayward pass and tore into the Springbok territory.
Tuilagi floated a superb pass to Chris Ashton on the left but the flying wing's pass was slightly behind Mike Brown and the move broke down.
Owen Farrell responded on the hour mark for England to reduce the deficit to 16-9 and he added a second penalty to set-up a tense final period.
Trailing by four points and with two minutes left Chris Robshaw controversially opted to kick at goal and Farrell was on target.
England were left with just seconds to spare to snatch a last gasp win and although South Africa's restart went out on the full, the Springboks hung on.