Keir Starmer has given his strongest hint yet of tax rises to come in October’s budget, warning he will have to make “painful” decisions after finding what Labour says is a £22bn black hole in the public finances.

Giving his first major speech from No 10 since becoming prime minister, Starmer said on Tuesday it would take years to clean up after the previous Conservative governments, which he said had overseen “14 years of rot”.

Speaking from a sunny Downing Street garden to about 50 members of the public whom he met during the campaign, he drew a contrast between his speech and how the garden was used under Boris Johnson, when it hosted lockdown-breaking parties.

But while the weather and the mood were upbeat, the prime minister’s message was pessimistic, warning the public to expect more unpopular decisions in October’s budget.

Starmer said: “There is a budget coming in October and it’s going to be painful. We have no other choice, given the situation that we’re in. Those with the broadest shoulders should bear the heavier burden, and that’s why we’re cracking down on non-doms [by making it harder to claim non-domiciled tax status].”

He added: “I know that after all that you have been through, that is a really big ask and really difficult to hear. That is not the position we should be in. It’s not the position I want to be in, but we have to end the politics of the easy answer, that solves nothing.”

Starmer and Reeves have ruled out raising income tax, national insurance or VAT in the budget. But having repeatedly said their plans for government did not require further tax rises, they now say they have been surprised by the scale of the budgetary problems they face.

The prime minister said: “In the first few weeks we discovered a £22bn black hole in the public finances. And before anyone says ‘oh this is just performative or playing politics’, let’s remember the OBR [Office for Budget Responsibility] did not know about it – they wrote a letter setting that out.

“They didn’t know because the last government hid it and even last Wednesday, just last Wednesday, we found out that thanks to the last government’s recklessness we borrowed almost £5bn more than the OBR expected in the last three months alone. That’s not performative, that’s fact.”

Reeves has been looking at a number of ways to raise money at her first major fiscal event, including raising capital gains tax and inheritance tax. She is also thought to be planning to stick to the tight departmental budget constraints forecast under the last government’s spending plans, and changing the way debt is measured to exclude the Bank of England.

The chancellor has already announced she will end winter fuel payments for about 10 million pensioners – a decision that has caused upset among Labour MPs but that Starmer defended on Tuesday.

“I didn’t want to means-test the winter fuel payment but it was a choice that we had to make,” he said.

The government may relent to pressure from Labour backbenchers to extend the £2bn household support fund, which helps tens of thousands of households at risk of destitution, beyond September.

Starmer is hoping to avoid a major voter backlash for decisions such as cutting winter fuel payments and allowing criminals to leave prison early by blaming the previous government.

He gave details on Tuesday of how he spent much of the riots in the Cobra emergency meeting room in Downing Street, trying to work out if there were enough prison places to house those who were being arrested.

“I can’t tell you how shocked I was when I discovered the full extent of what they’ve done with our prisons, and it’s going to take time to fix it. I can’t build a prison by Saturday,” he said.

“I shouldn’t be sitting in the Cobra room with a list of prison places across the country on a day-by-day basis, trying to work out how we deal with disorder. But that’s the position I was put in.”

He blamed the riots on the prisons crisis, saying those who committed violence did so in part because they did not expect to be punished.

“They saw the cracks in our society after 14 years of failure and they exploited them,” he said. “That’s what we’ve inherited, not just an economic black hole, a societal black hole, and that’s why we have to take action and do things differently.”

The tone has not pleased everyone on the left. Sharon Graham, the general secretary of the Unite union, called it “bleak”, saying it was “time to see the change that Labour promised”.

She added: “We don’t need more excuses about fiscal responsibility or talk of wealth creation. We should not pit pensioners against workers, that is not a choice that should be on the table.”