More than 184,000 people have been fined by HMRC, even though they did not make enough money to pay tax.
The fines were handed to people who earned less than the personal allowance - which is the amount you can earn before you start paying tax and currently sits at £12,500.
The reason for the fines was failing to complete a self-assessment tax form on time for the 2020-21 financial year.
According to data collected through a Freedom of Information request by Tax Policy Associates, HMRC fined 92,000 of the lowest-paid 10% of the population for filing their tax return.
In the same year, just 39,000 of the highest-paid were fined.
The majority of taxpayers are not required to submit a tax return since their income primarily comes from their employer and tax is deducted from here.
However, about 11million people have to file a self-assessment tax return if they have other sources of income - or have done in the past.
Between 2018-22, more than 660,000 fines were issued to people on low incomes - around 600,000 are estimated to be given to people who owe no tax whatsoever.
Only 180,000 have managed to successfully appeal them.
However, Tax Policy Associates noted that the 600,000 figure does not represent the number of people, just the number of penalties that were given - some may have received several penalties.
If you file your self-assessment tax return late - after January 31 or October 31 if you are submitting paper forms - you face an automatic £100 fine.
After three months, the fines can grow by £10 each day, and if you have not paid within six months you face an extra £600 on top.
You could also face another £300 fine if your return is not filed within 12 months of the deadline - the total amount of penalties can reach up to is £1,600.
Over a number of years, Tax Policy Associates said this can amount to several thousand pounds which could cause severe financial strain to those on low-incomes.
Until 2011, the penalty was scrapped for those whose returns showed there was no tax to pay.
Dan Neidle, a tax campaigner and founder of TPA, said: “We believe the law and HMRC practice should change. Nobody filing late should be required to pay a penalty that exceeds the tax they owe.”
“People are falling into debt and, in one case we’re aware of, becoming homeless as a result of HMRC penalties. Advisers working with low-income taxpayers see this kind of situation all the time, and filing appeals for late-payment penalties often makes up a significant amount of their work.”
An HMRC spokesperson told the Mirror: “The Government has recognised that taxpayers who occasionally miss the filing deadline should not face financial penalties, and has already announced reform of the system.
“Deadlines for returns are necessary for the efficient functioning of the tax system, though, and we strongly encourage anyone who does not need to file a return to tell HMRC.
“Our aim is to support all taxpayers, regardless of income, to get their tax right, and details of what to do if a person no longer needs to file a return are included in reminder letters every year.”