Two days after defecting to Labour, the former Conservative Natalie Elphicke stood in front of a pure red backdrop as she introduced Keir Starmer in her Dover constituency – on the frontline of the battle over stopping small boat crossings.

Before a slogan stating “border security”, Labour’s newest MP railed against her former Tory colleagues for having failed to control migration, and praised Starmer for stepping into the centre ground that Rishi Sunak had “abandoned”.

The problem for Labour is that many within the party are unsure they want to stand on that same centre ground as Elphicke, whose staunch rightwing views while a Conservative MP made her one of the last people suspected of planning to cross the floor. And unusually for a defection, it angered more people in Elphicke’s new party than her old one.

On top of tension over the leadership’s stance on Gaza and whether it is watering down some leftwing policies, some party insiders wonder whether Starmer is pushing the base and parliamentary faithful too far.

Labour MPs were particularly upset about Elphicke’s controversial defence of her ex-husband, a convicted sex offender.

A series of leftwing MPs publicly criticised the decision, but for the most part those in the centre of the party shared their concerns privately.

One figure in the Labour leader’s office accused the parliamentary party of “talking about itself”, adding dismissively that it was “not Keir’s job to go round managing how they feel”.

But after an angry response, a Labour spokesperson swiftly distanced the party from the “unauthorised and unacceptable briefing”.

Starmer aides believe the political benefits of taking on Elphicke are worth a bit of internal turmoil and that most voters will just see headlines about another defection. That it was the MP for Dover, a constituency dealing with the effects of the small boats crossings, was a bonus.

There are already hints that, after two Tory defections in the last fortnight, there could be more to come.

However, perhaps in a nod to the unhappiness inside the party over Elphicke, the shadow health secretary, Wes Streeting, suggested on Friday there were limits. “If Liz Truss were to want to cross the floor, and I don’t imagine she would, I would rather take the lettuce,” he said.

Labour insiders acknowledge that on some issues, Starmer is pushing not just MPs and members, but also the party’s base, to the limits.

“We’ve got to show that it’s a changed party and that we can win in places and with people who wouldn’t have come near us with a barge pole five years ago,” said one.

At the heart of the strategy is demonstrating fiscal responsibility, even if that means dishing out some “hard truths” on what Labour can deliver in office.

“I’m under no illusions about the scale of the challenge that I would inherit if I become chancellor later this year,” Rachel Reeves said in a speech in the City of London earlier this week. “The worst economic inheritance that any chancellor has faced since the second world war in terms of taxes, growth, living standards and the public finances.”

Within weeks of taking office, Reeves would have to announce a one-year spending review, spelling out how much she intends to give each Whitehall department for Labour’s first year in power.

A combination of tax cuts already announced by the Tories, which Labour has promised to retain, low growth forecasts and high debt costs mean real-terms spending cuts for many departments are on the cards unless she can find the money elsewhere.

Some in the party are talking about holding an emergency budget immediately after taking office.

There are tensions within the party, however, about whether a new Labour administration should adopt a tough plan to get the economy back on track or try to give people some reasons for hope in the early months.

“Our research shows us that people want honesty as much as they want hope,” one party insider said. “If you promise the earth they don’t think you can deliver it.”

The party is expected to deliver some “early wins” on the economy, they added, with planning reform and a national wealth fund likely to be introduced as soon as possible.

Labour has already rowed back on some commitments, in part because of the fiscal backdrop and to make sure they work in government, but also to stem Tory attacks. So far, these have included watering down pledges on the £28bn green investment fund and workers’ rights, and refusing to commit to scrapping the two-child benefit cap.

Yet some MPs are suspicious that these developments are not just about “bomb-proofing” policies, and suggest some in Starmer’s office see them as an opportunity to take on MPs.

“Keir wants to push as far as necessary and he doesn’t take any pleasure from it but there are people around him who want to go as far as they can go,” said one Labour frontbencher. “It feels like they take joy from punishing the PLP and Labour members.”

They added: “There are times we’ve got the balance wrong and we don’t make a principled argument about what we’re doing.

“On the two-child policy, for example, our body language was that we didn’t mind the policy rather than saying we want to get rid of it but that we can’t afford to now …

“It seems designed for the media, rather than the voters we need to win. They want to see a party that is confident in its values and its vision for the country. We’ve had enough internal warfare with the Tories.”

On the Gaza conflict, however, the leadership appears to recognise that it pushed its core support too far, after local elections showed some voters were willing to express their anger at the ballot box.

Reeves became the latest senior party figure this week to acknowledge concerns among some voters, including in Muslim communities, over the party’s position, which insiders admit was initially clunky and took too long to be reached.

She said she regretted whenever people said they could not bring themselves to vote for Labour and that the party would “seek to rebuild the trust” it had lost.

One of the biggest issues Labour is grappling with is deep levels of mistrust among voters that politics can bring any level of change.

“Time and again, we find in research that many voters do not trust government and think all politicians are all the same. Labour must change that,” said Josh Simons, the director of the Labour Together thinktank that has conducted focus groups on this.

“That requires honesty and frankness with a public who are tired of false dawns and a singular focus on demonstrating that voting can secure real change.”