Tens of thousands of people who earn too little to pay tax are still receiving hundreds of pounds in fines for failing to file a tax return on time, according to new data seen by the Observer.

More than 83,000 people earning less than the threshold were issued with a £100 penalty by the tax authorities in 2021-22. Only 17,000 of those fines were later cancelled on appeal. By contrast, only 20,000 people earning £100,000 or more were penalised.

Those with low incomes fare even worse when it comes to the £300 fines handed out for filing a year late. Half of all the 61,000 fines were to people who earn too little to pay tax. Some 12,000 were successful on appeal. Only 5,000 people earning more than £100,000 a year were fined.

The figures have been uncovered under the Freedom of Information Act by the Tax Policy Associates (TPA) thinktank. It has raised concerns about the extent to which people who are vulnerable or facing financial hardship are being issued with the penalties for failing to submit a self-assessment tax form on time, which self-employed workers must do. Currently, anyone who earns less than £12,570 does not pay income tax.

There are now calls to go back to an old system in which anyone who did not pay any tax was effectively spared a fine. “It’s shameful that tens of thousands of people on very low incomes, often with difficult lives, have their lives made more difficult by HMRC penalties,” said Dan Neidle, the founder of TPA. “We should go back to how things were before 2010: nobody should have to pay a penalty if their income is too low for them to have a tax bill.”

One of the thousands to have faced the issue, a self-employed photographer who wished to remain anonymous told the Observer that he had been issued with many fines, despite the tax authorities acknowledging he did not earn enough to pay tax. He said he had not earned enough to pay income tax since 2017 and had subsequently lived off savings, small amounts from odd jobs and money from a legal settlement that is non-taxable.

He said confusion had played a major part in his case. “I was sent reminders for tax, I ignored them and they sent more and the fines kept increasing,” he said. “And then they sent me a letter saying, ‘we can now see you haven’t been making enough to tax so actually you’re not due anything’.”

He said he believed this was the end of the issue, yet a few months later he received an even larger demand. “Two or three months later they sent me a bill for £3,000, not for tax due but for fines,” he said. He said that the fines, combined with the struggling to restart his business during the pandemic, had left him in a “dark place”.

“Mentally I’ve struggled and all of this is quite burdening. There seems to be an attitude of ‘you are guilty unless you can prove you’re innocent’.”

The latest figures on fines, however, do suggest their use is falling. In the previous year, 92,000 people among the lowest paid 10% of the population were fined by HMRC for late filing of their tax return. In an attempt to deal with the issue, a new system is being introduced that gives people warnings before they are given a fine. However, tax campaigners say that delays to the new system have caused confusion and unfairness.

“The current penalty regime will continue alongside the new regime for a few years,” said Joanne Walker, a technical officer for the Low Incomes Tax Reform Group. “This has the potential to be unfair on those who remain in the current, arguably harsher, regime, especially those on low incomes as it is recognised that the new regime will be fairer.

“We suggest that, until the new system can be introduced for everyone, HMRC should take steps to make late filing penalties fairer and more consistent for all taxpayers.”

An HMRC spokesperson said: “The government recognises that taxpayers who occasionally miss the filing deadline should not face financial penalties and reform of the system is under way. Our aim is to support all taxpayers, regardless of income, to get their tax right and avoid fines.

“The overwhelming majority of customers file on time.”