Prime Minister Julia Gillard announced a Multi-Party Reference Group last week to devise a workable asylum seeker policy alongside an expert panel led by former Defence Force chief Angus Houston.

It comes after a compromise bill by independent MP Rob Oakeshott was defeated 39 votes to 29 in the Senate.

The group will consist of three Labor MPs, two Greens MPs and two independents, including Mr Windsor.

The Coalition was offered three places in the group, but Mr Abbott has refused to co-operate unless the Government sets a terms of reference.

“We don’t need an external committee to tell us what our policy is,” Mr Abbott told the media.

“Rather than appoint new committees to tell her what the policy should be, the Prime Minister should be using the committee set up for this purpose, the Cabinet.”

But Mr Windsor said terms of reference would defeat the purpose of the committee by giving it a political agenda.

“If the Prime Minister sits down and sets out these detailed terms of reference, she risks creating a political agenda within the committee,” he told The Express.

“The smart way to handle this is for people to leave their party politics at the door, with the focus on figuring out how to fix this issue. Let’s find a workable solution that can also get through the Senate.”

Labor will rely on the support of either the Greens or Coalition to pass any legislation through the Senate.

Mr Windsor said he was willing to consider all options, including Mr Abbott’s proposal to send boats back to Indonesia, if they were seen by the expert panel to be a viable solution.

“If Nauru or turning boats around are a good solution to the issue and provided it’s tested by those people who know more than us about the issue, then I’d have a good look at it, because I don’t have a specific agenda,” he said.

“But Tony Abbott doesn’t seem to have enough confidence in his own ideas that they be tested in this non-partisan committee.”