SYDNEY (AFP) - Pressure was building on Tuesday on defeated Australian Prime Minister Kevin Rudd to quit politics, with senior figures within his Labor Party saying he would be a destabilising influence.

Mr Rudd's popularity with the Australian public came crashing down on Saturday when he was soundly defeated by conservative Tony Abbott and he announced he would resign as Labor leader.

But several high-profile Labor figures want the 55-year-old former diplomat to go even further and stand down from his Queensland electorate and exit politics altogether to end three years of bitter infighting within the party.

"Kevin should seriously contemplate leaving the parliament," former minister Brendan O'Connor told Sky News.

"If you have a former prime minister sitting in your party room on the backbench, that spectre looms large."

Mr Rudd was elected prime minister in the 2007 general election but within his first term he was dumped by colleagues fed up with his style of management and demoted to foreign minister.

His sudden downfall mystified the Australian public who had elected him, and this, coupled with the unpopularity of his predecessor Julia Gillard, prompted Labor to return to Rudd's leadership a second time in June to salvage the party vote in 2013 polls.

Former trade minister Craig Emerson, a staunch Gillard supporter, said this was a mistake as Mr Rudd had a history of destabilising his colleagues to take their positions as he worked his way to the top and called on him to quit.

"The election was a disaster for Labor and Kevin Rudd's continuing presence in the parliamentary Labor party will see him do what he has always done and that is willingly, wilfully, recklessly destabilise Labor leaders," Mr Emerson told the ABC.

"If he stays for any length of time, he will be a favourite of the media, they will ring up Mr Rudd, ask for some off-the-record comments that they can use, just as has happened in the last three years under Julia Gillard's leadership.

"If he stays, he will destabilise. It's in his nature."

Former defence minister Stephen Smith and former climate change minister Greg Combet have also urged the party to put the Rudd-Gillard years behind them.

"We have to put a line underneath the politics of personality and the politics of personal division," Mr Smith said on Saturday night.

His view was echoed by Mr Combet who said there had to be a fresh start and "the new leader cannot be looking around and seeing over their shoulder the spectre of that division in the form of Kevin Rudd or anyone else".

Mr Rudd has not spoken since Saturday's election night concession speech but his close colleague Kim Carr said he intended to remain in parliament as the MP for Griffith for a full three-year term.