The NSW government has been pushed further into minority after Labor won the seat of Bega for the first time in one of four by-elections held in the state.
ABC chief elections analyst Antony Green said Labor's Michael Holland was set to win the seat with a swing of more than 13 per cent.
The electorate was created in 1988 and has never been won by Labor.
Liberal candidate Fiona Kotvojs's primary vote was about 35 per cent.
Her Coalition colleague Trent Zimmerman, the federal MP for North Sydney, described the result as "a huge disappointment".
Votes were also held in the electorates of Willoughby, Strathfield and Monaro.
Former transport minister and Bega MP Andrew Constance resigned from the NSW parliament last year to concentrate on a tilt at federal politics.
Dr Holland claimed victory just before 9pm on Saturday, saying, "it was time to celebrate a bit of history".
"I hope to go on and represent my community now as a simple rural local Member of Parliament," he said.
Mr Constance was considered a high-profile MP and, with his personal popularity no longer a factor, the government's 6.9 per cent margin evaporated.
The Liberals and their candidate, Tim James, sustained a swing against them in Willoughby.
The seat had been vacated by former premier Gladys Berejiklian, who resigned amid a corruption probe into her ex-boyfriend last year.
Mr James claimed victory three hours into the count but the government's once-comfortable margin of 21 per cent looked set to take a hit.
While Labor isn't running a candidate in the seat, independent Larissa Penn is polling well.
Green said he was "pretty confident" Mr James would win.
Mr James described himself as a "loyal long-term Liberal".
"So, I just want to say how proud I am as a Liberal to be here standing before you and to thank all of you, party staff, members, volunteers and friends, standing in the rain as you did, thank you so, so much for your great contribution," he said at his victory function.
Meanwhile, there was a swing to Labor in the regional electorate of Monaro — former deputy premier John Barilaro's old seat — but Green called it for The Nationals just after 8:30pm.
Nichole Overall had a buffer of about 5 per cent over Labor's Bryce Wilson with more than 30 per cent of the vote counted.
Ms Overall said the campaign had been a "tough fight".
"The idea that Monaro is ever a safe seat is never actually the case," she told the ABC.
"We always knew it was going to be a tough fight here.
"I'm really pleased that I've spent every day of the past four months fighting as hard as I can."
In Strathfield, high-profile Liberal candidate Bridget Sakr was trying to win the seat off Labor for the first time since 2015.
But as counting progressed, Labor's Jason Yat-Sen Yi pulled ahead.
Green called it for the Opposition with just over 20 per cent of the vote counted.
Speaking at a function in Strathfield, NSW Premier Dominic Perrottet praised Ms Sakr's campaign, predicting she would win the seat at the next state election.
"You have done an amazing job. Make no mistake that Labor has gone backwards," he said.
However, speaking at Labor's party in the electorate, Opposition Leader Chris Minns had a different take.
"Many people in New South Wales sent the Premier of New South Wales a message on their behalf at the ballot box today," he said.
"The Premier of New South Wales needs to listen to that message. The government needs to change direction."
Thousands of people across the four electorates cast their votes early at pre-poll centres and by using postal ballots.
Roughly 50 per cent of registered voters cast their ballots before polling day in Bega, while that number was around 40 per cent in the other three seats.
The first postal votes will not be opened until next weekend, meaning the final results will not be known until later this month.
