Labour has appointed an expert panel to advise the party on ways to tackle tax avoidance following its plan to reap £5bn from a crackdown on tax dodgers.

The party’s shadow financial secretary, James Murray, said the independent group would also advise on how to modernise the tax authority, which has come under fire from MPs for failing to claw back an “eye-watering” amount of owed tax. He said the panel offered decades of experience.

Polls are predicting Labour will win the general election but it will inherit a difficult financial legacy.

This week shadow chancellor Rachel Reeves said she would raise £5bn a year by the end of the next parliament from a clampdown on tax evasion after considering how to fill a hole in her financial plans created by Jeremy Hunt in his budget.

Last month, the chancellor offered tax cuts, paid in part with two initiatives – an extended windfall tax on oil and gas companies and higher taxes on non-domiciled residents – that Labour was relying on to boost schools and NHS funding.

Reeves said a tightening of rules on non-domiciled taxpayers, forcing them to pay £2.6bn more tax than expected under rules brought in by Hunt, will also help fill the financial hole left by Hunt’s budget.

Hinting that a draconian crackdown on tax avoidance was likely, Murray said Labour may need to extend the powers given to tax inspectors to make sure businesses and households pay the correct amount of tax. However, he would take advice from the expert panel before putting forward detailed proposals.

Sir Edward Troup, a former Treasury special adviser on tax and head of HMRC, will be joined on the panel by Bill Dodwell, former tax director of the Office for Tax Simplification and a retired senior accountant at Deloittes.

The four-person group will also include the Labour MP Dame Margaret Hodge, a former chair of parliament’s public accounts committee, and Mike Bracken, founding partner at the consultancy Public Digital, and founder and former executive director of the UK Government Digital Service.

Murray said: “I am grateful to the panel for agreeing to support us with this incredibly important piece of work.”

Several tax lawyers and accountants said they were dismayed that Labour planned to squeeze the incomes of wealthy non-doms.

Sophie Dworetzsky, a partner at the law firm Charles Russell Speechlys, said the move was “disappointing”.

She said: “These measures feel like trying to squeeze out every last drop until there’s nothing left. The UK operates in an environment of tax competition, and if the UK makes itself yet more unattractive from a tax point of view, we could lose out to other countries, such as Italy, which have more favourable tax regimes.

“One can but hope this is realised before any legislation is implemented,” she added.

Nigel Huddleston, the financial secretary to the Treasury, said Labour’s initiative represented a return to “the same old Labour policies of higher spending, higher borrowing and higher taxes”.

He added: “Without a plan to pay for Labour’s billions of pounds of unfunded spending promises and their unfunded £28bn a year decarbonisation promise, taxes on working people will rise.”

Last year the cross-party Commons public accounts committee said an “eye-watering” amount of tax was owed to HMRC, while also criticising tax collectors for lacking ambition to tackle fraud and error.

HMRC collected £731.1bn in taxes and duties in the financial year 2021-22, the highest on record. However, the group of MPs tasked with holding the government to account over public spending said billions that could be spent on improving public services was being lost through unpaid tax.

Reeves said her plan to invest more than £500m in HMRC to increase the number of tax inspectors would begin to close a £36bn tax gap – the difference between the amount of money HM Revenue and Customs is owed and the amount it actually receives.

She said it was not “rocket science” and Labour could “ramp up” the number of HMRC staff “pretty quickly” if the party won the election.

“At the start you might need to bring in extra resource but then you need to train people up within the government to do this work,” she said. “This isn’t rocket science. Previous governments have managed to close that tax gap, as it’s called.”

She added: “The government’s plans that they announced in March about non-doms, they said they were taking our policy; well, it turns out they’ve taken it but left a load of loopholes in it.”

The accountants UHY Hacker Young said fresh data showed that HMRC collected a record high of over £39bn from tax investigations in the year to 31 December, up 22% from £32bn in 2022.

Nikhil Oza, a partner at at the firm, said HMRC was already using new technology, including artificial intelligence, to claw back “missing” tax receipts.