Tory MPs are urging the Cabinet to show some "backbone" and oust Boris Johnson as backbench anger mounts against the Prime Minister.
Storm clouds are gathering again for Mr Johnson after the disastrous loss of the Red Wall seat of Wakefield, and Tiverton and Honiton in the Tories' southern heartlands.
The dramatic double by-election blow on Friday came only weeks after the PM clung onto power by winning a Tory no confidence vote.
Under Tory guidelines, he cannot be challenged again for a year - unless the backbench 1922 Committee decides to tear up its rulebook.
The by-election disaster has infuriated nervy Conservatives who fear for their own seats and fuelled Mr Johnson's leadership woes.
Rumours about at least six Tory MPs could defect to Labour prompted Red Wall Tory Dehenna Davison and ex-minister Caroline Nokes to publicly deny they were planning to switch party.
Former Deputy PM Damian Green called on Cabinet ministers to step in to end "the long agony" and replace the embattled PM.
"It's no secret that many people in the Cabinet are setting up potential leadership campaigns," the senior Tory told Channel 4's Andrew Neil Show.
"If this long agony for everyone concerned - from the PM down - is to be brought to a head one way or the other, then maybe somebody in the Cabinet might wish to take some action."
Mr Green added: "The way for this to be brought to a head would be for one or more members of the Cabinet to take a decision to say, ‘Look, we can't we can't carry on like this’.
"I think funnily enough this wouldn't rule them out, looking at historical precedent it might rule them in. So they should think about it."
Senior backbencher William Wragg also told top ministers to show "backbone and leadership".
Mr Wragg, who has previously called on the PM to quit, told the BBC's Westminster Hour: "[Tory chairman] OIiver Dowden has resigned and credit to him for doing so in taking an element of responsibility.
"But so far I think it is fair to say, and it’s all very well of colleagues whispering to each other in the tearoom and in the corridors, that the sense of disappointment that there is on the backbenches towards the Cabinet is palpable because you would have expected for some of them at least to show a bit of backbone and indeed leadership.
"Indeed any of them with leadership aspirations might wish to consider this and do something about it."
The Prime Minister told reporters at the G7 summit in Bavaria that he wasn't worried by Tory plotting at home as questions about his leadership had been "settled".
Mr Johnson is trying to shift focus onto his diplomatic efforts during a series of summits but his time abroad has been dogged by domestic woes.
Environment Secretary George Eustice said MPs shouldn't be distracted by "navel gazing" about leadership.
He branded speculation about possible Tory defectors as "tittle tattle" but admitted he didn't know if any colleagues would jump ship to Labour.
Mr Eustice told LBC: "We've got some really big challenges that we're wrestling with as we come out of the pandemic. Pressure on the cost of living; a global security crisis with the war on Ukraine at the moment; a backlog on the NHS, because they've been preoccupied with COVID, and we've got to get the support to them.
"There's a lot for this government to be getting on with, without us being, you know, distracted by such tittle tattle or even indeed, you know, navel-gazing about leadership contests and so on."
