In the end, it wasn’t a double blow, but a triple one.

Boris Johnson lost not just two by-elections but his party chairman too, who became the first of his cabinet to publicly express misgivings over the PM’s leadership, saying volunteers and staffers “deserve better than this”.

The by-election losses he was reluctantly expecting, but Oliver Dowden’s resignation he was not, finding out when he returned to his room in Kigali after his 6am morning swim (5am back in the UK) and checking his phone.

Mr Dowden called soon after and then sent out a letter which must have been painful to read.

The former party chair told the prime minister that supporters were “distressed and disappointed by recent events” and said he “shared their feelings”.

“We cannot carry on as business as usual and someone should take responsibility,” he said as he offered his resignation.

A source close to the PM told me later the letter was “unhelpful”. Others would argue it was devastating as Mr Dowden offered his loyalty to the Conservative Party but said nothing of loyalty to Mr Johnson.

But this is not just a (continued) crisis of leadership for the prime minister, it is potentially a wider crisis for the Conservative Party.

Take the Tiverton election result. It was an earth-shaking result and a record-breaker, coming in as the largest percentage majority loss for the Tories in any by-election, and one of the biggest post-war by-election swings against them.

Their 24,000 majority, in what should have been an ultra safe seat, was overturned in a 29.9% swing to the Lib Dems.

For context of just how safe his seat was meant to be, if the Conservatives lost every seat they hold with a smaller majority than Tiverton at a general election, the party would be reduced to having just 72 MPs.

By-elections are of course very different beasts to general elections, but I make the point to illustrate why Mr Johnson’s MPs will be feeling very anxious.

They will be wondering if their seats are safe under his leadership, whether the “proven vote winner” is going to lead them into redundancy at the next election.

They will be asking themselves if Mr Johnson, once hailed as a Heineken politician who could reach parts of the country that other Tories could not, is now a busted flush.

Mr Johnson was keen in Kigali to put the losses down to midterm blues, but it looks more serious than that – and his party fear it.

Election guru Professor John Curtice put the results into context, pointing out that the recent by-election results – losing four out of five contests – were the party’s “worst run of by-election results since the Tories first returned to power in 2010”.

He pointed out that on average in the last five contests, the party’s share of the vote has fallen by over 20 percentage points.

You’d have to go back to the 1992-1997 John Major years to find anything comparable – and he went onto lose the subsequent election.

The entire party – cabinet ministers, MPs, members, staffers, volunteers – will be inevitability asking themselves whether Mr Johnson remains the right leader, whether these losses are a blip linked to economic circumstances or something more profound.

Voters have shown they will vote tactically, in this instance, to give Mr Johnson a bloody nose.

And that brings us to the inevitable question, again, about his continued leadership of the country and the Conservative Party as prime minister.

The Johnson camp in Kigali are clear – the PM is working all hours, getting on with the job, representing British interests on the world stage across different summits, and across a host of agendas. From pressing on trade and girls education with Commonwealth leaders, to discussing issues of food and energy security, and the ongoing war in Ukraine at the G7 and NATO.

And for Mr Johnson personally, the matter was settled earlier this month when he won the confidence vote in his leadership.

But 4,000 miles away from Rwanda, a cabinet minister, a former Conservative leader Lord Howard and his MPs are again questioning his suitability for the role.

Lord Howard told the BBC’s World at One: “The party and more importantly the country would be better off under new leadership. Members of the cabinet should very carefully consider their positions.”

But this is the nub of it. There is no stand-out successor, and none of the would-be runners want to go first.

Anti-Johnson MPs – having blown their chance to oust Mr Johnson in that confidence vote – are now agitating for cabinet colleagues to wield the knife.

But so far, no one has followed Mr Dowden, while the PM has packed his top table with loyalists rather than taking his team from different wings of the party in order to better insulate himself against such threats.

One former cabinet minister told me on Thursday the cabinet weren’t going to do much at all.

“They don’t have the nerve,” said the MP. “Had one walked at the confidence vote, they would have been run away favourite for leader now.”

Another told me: “The mood is bad, but I don’t think it changes anything for now.”

Follow the Daily podcast on Apple Podcasts, Google Podcasts, Spotify, Spreaker

Meanwhile, another supportive cabinet minister told me the “the moment has passed” to try to unseat Mr Johnson, telling me of Mr Dowden’s resignation: “Oliver hated being party chairman.”

But from Kigali there’s no doubt the PM’s team will be on resignation watch, from the cabinet downwards.

As one source close to the PM told me this morning, they simply don’t know if others will offer their resignations, but say they’d be surprised.

The other mechanism by which to ease the PM out of No 10 would be if the 1922 committee, with the backing of the parliamentary party, change internal rules to allow a vote within a year.

That thought is very much back on the agenda today.

Then there’s the potential for another huge blow-up when the privileges committee rules on whether the PM misled parliament over partygate later this year.

Two former cabinet ministers told me recently that a guilty verdict there could be the moment when the PM’s hand is forced by cabinet resignations or a vote against him – either via the 1922, or even the floor of the House.

But for now, despite the disquiet, the discontent, and the ballot box losses, the PM stays on.

Three blows today, but still no knock-out punch for the PM.