The Morrison government ended a messy week in parliament with its state colleagues in NSW enduring mixed results in a clutch of by-elections, including the likely loss of the coastal seat of Bega to Labor for the first time ever.
Federal minister Keith Pitt played down the impact of Saturday's results on the federal election in a few months time.
"Generally these things don't reflect into a federal result. Voters know the difference between each level of government," Mr Pitt told Sky News' Sunday Agenda program
He also brushed off comments by NSW Treasurer Matt Kean who said the recent dramas around the federal government didn't help the coalition brand heading into the four by-elections.
"There is always plenty of free advice in this job," Mr Pitt said.
"If Mr Kean is looking for someone to blame he might want to have a look in the mirror."
The by-elections came after a week in the federal parliament where five Liberal MPs crossed the floor on a Labor amended sex discrimination bill, as part of the religious discrimination act, and alleged leaks from the cabinet.
But Mr Pitt insists the ministry is a very strong unit, while adding that coalition MPs are entitled to cross the floor, whereas they would be expelled if they were in the Labor party.
But he was disappointed by those who crossed the floor because they did not tell their colleagues or the prime minister of their intentions.
However, Mr Pitt said the government is determined to get changes to the discrimination act done.
"It's clearly not going to happen in this term of government, which I think is unfortunate," he said.
Opposition Leader Anthony Albanese this government is driven by "disunity, dysfunction and dishonesty" and needs some time in opposition.
"My problem with the prime minister is he is always seeking to divide," Mr Albanese told Sky News.
"And that is what he did this week on this legislation that he brought forward, they were seeking division rather than bringing people together."
He also hit back at the government's attacks on him in parliament last week, bringing up comments from more than 30 years ago and more than nine prime ministers ago.
"I think what it shows is how desperate they are," he said.
"They haven't been able to attack anything I've done in 25 years in the national parliament with two decades as a front bencher with senior portfolios."