SYDNEY (AFP) - Asylum-seekers drowning on the treacherous boat journey to Australia presented a "god-awful" problem, Home Affairs Minister Jason Clare said on Wednesday after four more deaths were reported following a dramatic sea rescue.

Authorities pulled 144 people from the surging waters off the Indian Ocean territory of Christmas Island late on Tuesday, but they also recovered four bodies after the ship carrying an estimated 150 people capsized and sank.

"This is a wretchedly difficult area and it has been poisoned by politics," Mr Clare said of boatpeople.

"If we are going to fix this god-awful problem then we need to work together."

Australia has struggled to stem an influx of asylum-seekers arriving by boat, with record numbers turning up in 2012 and more than 13,000 so far in 2013.

Hundreds have drowned making the journey and Canberra's plans to send asylum-seekers to remote Pacific islands for processing has so far failed to stop the numbers from increasing.

Mr Clare said the centre-left Labor government - whose scheme to transfer asylum-seekers arriving by boats to refugee camps in Malaysia was blocked by the conservative opposition - was working on changes to its policy.

But he said the problem required domestic and regional cooperation and Australians wanted the political parties to work together on the issue.

"We have been fighting about this for more than 10 years," he said.

"The government should be given the power it needs to stop people dying at sea."

In the latest tragedy, the boat capsized as it was being escorted in heavy weather by two Australian navy ships to Christmas Island, after issuing a distress call earlier in the day.

The boats pulled survivors from the water as a military aircraft dropped life rafts.

Rear Admiral David Johnston, commander of Border Protection Command, said the 30-metre boat had not been as "jam packed" as other vessels that had been intercepted and appeared quite solid.

But he said the vessel, which was carrying men, women and children, sank quite quickly once it overturned.

Australian border protection officials have been faced with an increased tempo of arrivals in recent months, despite scores of drownings.

On Friday, a boat carrying 97 asylum-seekers sank, claiming the life of a baby boy and leaving eight others missing.

In a bid to slow down arrivals, Australia's refugee tribunals have reportedly been ordered to take revised country assessments into account when looking at asylum claims from people from Iran, Afghanistan and Vietnam.

This directive came after Foreign Minister Bob Carr said many people seeking asylum, particularly from Iran, were economic migrants, not refugees.

Prime Minister Kevin Rudd is expected to soon announce a new-look asylum-seeker policy - an issue set to be key in elections due this year.