AMSTERDAM (REUTERS, AFP, BLOOMBERG) - Dutch Prime Minister Mark Rutte easily defeated a strong challenge by far-right rival Geert Wilders in key elections, offering huge relief to other EU governments facing a wave of nationalism.

With 95 per cent of votes counted, Rutte's Liberal VVD party won 33 seats, making it the largest in the new 150-seat parliament. Wilders and his Freedom Party (PVV) was second with 20 seats, while the Christian Democratic Appeal (CDA) and the centrist D66 party were tied for third with 19 seats each, the Dutch news agency ANP said.

Millions of Dutch flocked to the polls in a near-record turnout, with the stakes high in an election pitting the pro-European Rutte against his anti-immigration and anti-EU rival.

Following last year's shock Brexit referendum and Donald Trump's victory in the US, the Dutch vote was being closely scrutinised as a gauge of the rise of populism on the continent ahead of crucial elections in France and Germany.

At 78 per cent, turnout was the highest in a decade. The election that was a test of whether the Dutch wanted to end decades of liberalism and choose a nationalist, anti-immigrant path by voting for Wilders and his promise to "de-Islamicise"the Netherlands and quit the European Union.

The result was a relief to mainstream parties across Europe, particularly in France and Germany, where right-wing nationalists hope to make a big impact in elections this year, potentially posing an existential threat to the EU.

Far-right candidate Marine Le Pen is set to make France's presidential election run-off in May, while eurosceptic, anti-immigrant Alternative for Germany is likely to enter the German federal parliament for the first time in a September election.

Rutte received congratulatory messages from European leaders and spoke with some by telephone.

German Chancellor Angela Merkel's chief of staff, Peter Altmaier, could not restrain his joy, tweeting: "The Netherlands, oh the Netherlands you are a champion!..... Congratulations on this great result."

France's foreign minister Jean-Marc Ayrault tweeted:"Congratulations to the Dutch for stemming the rise of the far-right."

The exit polls helped the euro rise to its highest against the dollar since Feb. 7.

But Mabel Berezin, professor of sociology at Cornell University in the United States, said defeat for Wilders, who has been in parliament for nearly two decades, should not be considered a sign that European populism is waning.

"He does not represent a populist wave. Rather, he is part of the political landscape and how his party fares does not tell us much about European populism," she said. "The real bellwether election will be Marine Le Pen's quest for the French presidency, starting April 23 - that is where the populist action is and that is what we should be focusing upon."

Rutte got a last-minute boost from a diplomatic row with Turkey, which allowed him to take a well-timed tough line on a majority Muslim country during an election campaign in which immigration and integration have been key issues.

The spat did not hurt the chances of Denk, a party supported by Dutch Turks, which looked set to win three seats, becoming the first ever ethnic minority party, in a possible sign of deepening ethnic division.

But while Rutte averted what in the early stages of the campaign looked like a possible victory for Wilders, years of austerity pushed down his share of the vote. His junior partner in the outgoing coalition, Labour, suffered its worst ever result, winning just nine seats, down from 38 last time.

Rutte could possibly turn to the CDA and the D66, which won 19 seats each, to form a coalition.

But with the three parties' combined total of 71 seats, Rutte would still need another party to reach the 76-seat majority. That means it will likely take months for him to negotiate a ruling coalition.

Wilders, whose Party for Freedom (PVV) increased its parliamentary seats to 20 from the 15 won in 2012, tweeted: "We won seats! The first victory is in! And Rutte has not seen the last of me yet!!"

The result, however, is down from his 2010 high of 24 seats.

Wilders said he would be prepared to work in the new government if asked. "I would still like to co-govern as the PVV, if possible. But if that doesn't work ... we'll support the Cabinet, where needed, on the issues that are important to us."

One of the biggest winners of the day was the young, charismatic Jesse Klaver, leader of the ecologist leftwing GroenLinks.

In a remarkable turnaround, the party won 14 seats compared to just four in the outgoing parliament.

With the Netherlands' complex system of proportional representation, even small parties can get parliament seats, enabling them to play an important role in shaping the make-up and viability of the next government. The seats are allocated in exact proportion to each party's vote share.