But Gatland could not hide his disappointment after his men missed a number of chances to grab a historic win in a spirited display at the Millennium Stadium.
"It's a painful experience we went through because good teams handle pressure and they convert situations into points and we've got to learn from that," said Gatland.
"But I'm really proud of the performance and we know we're not too far away. I've had a real go at the players in the changing rooms and said that if we want to be where we think we can be, good teams nail big moments."
Wales trailed 20-3 before four penalties from fly-half James Hook edged them back into a hard-fought contest in Cardiff. But Gatland insists his players need to believe in their own ability to reach the next level and claim a prized scalp over one of the southern hemisphere giants.
"Our scrum, line-out and kicking could have been better but when you get the opportunities, you've got to take it. We should be furious with that because we had a chance to beat the world champions and we let ourselves down. We've got to be more clinical.
"We need to have the confidence and self-belief to handle the big moments and that's the progress this side needs to make.
"When the players look back they will realise they were good enough to be in the game. As coaches we are always questioning the players deep down to see if they believe. When they do, you will see us make further steps."
Wales now face Canada next Friday while Peter de Villiers's side travel north to take on Scotland in Edinburgh next Saturday.