Labour has written to Conservative candidates urging them not to hand out election material that they say is paid for by the party’s biggest donor, Frank Hester, who has been widely condemned for allegedly making racist remarks.

Hester, who has donated £15m, is bankrolling about 40% of the Conservative party’s national spending in the election.

Rishi Sunak has been under pressure to hand the money back since the Guardian reported in March that Hester said to colleagues in 2019 that looking at Diane Abbott made you “want to hate all black women” and that she “should be shot”.

The prime minister faced fresh calls to give up the money on Thursday after the Guardian revealed further allegations that he referred to a staff member as the “token Muslim”, imitated people of Chinese descent and remarked that one individual was attractive for a black woman.

Hester apologised for the remarks about Abbott in March, while maintaining that they were not to do with race. All the new allegations were put to Hester this week. He did not respond to the claims.

On Friday, Anneliese Dodds, the Labour party chair, challenged Tory candidates to take a stand against Hester’s money.

In her letter, she said: “Back then the suggestion was that Frank Hester’s words, saying that Diane Abbott made him want to hate all black women, were a one-off and that his other comments – about ‘the Indians’ among his staff, or the ‘Asian corner’ – were misconstrued.”

She said the Guardian’s fresh allegations revealed Hester had a history of making unacceptable remarks “year after year”, such as referring to a staff member as “tinged” and subjecting staff to impressions of Chinese people as well as saying one person was attractive for a black woman.

“Rishi Sunak and Richard Holden may not care about this,” she said. “They were happy to take thousands of pounds from Hester just days after the first allegations broke and they have been happy to make the Conservative election campaign overwhelmingly reliant on his donations. But if they will not return that money in light of these latest revelations, then you have a decision to make. Will you do the right thing and stand up to them?”

Dodds told the candidates that if they did not, “every leaflet you fold and every social media advert you post makes you personally reliant on the money” of a donor with a record of allegedly making racist remarks.

It was confirmed on Thursday that Hester had donated an additional £5m to the Conservatives this year, as well as a further £150,000 after the party was aware of his remarks about Abbott. His £15m of donations in total equate to about 40% of the overall £34m limit on national spending during the year before an election.

The Lib Dems called on Friday for the Conservatives to donate the latest £5m given by Hester to a veterans’ charity.

Richard Foord, the Liberal Democrat defence spokesperson, said: “Yesterday it emerged that the Conservative party accepted another £5m from Frank Hester, a man who has made the most appalling racist and misogynistic comments. Rishi Sunak must personally ensure this money is donated instead to a veterans’ charity as an apology for his absence [at a D-day ceremony] yesterday. It’s the least that our veterans and service personnel deserve.”

The Conservative party did not directly address questions about what due diligence, if any, it had carried out on Hester before accepting further donations from him.

A spokesperson said: “Mr Hester has rightly apologised for comments made in the past. Mr Hester has apologised and shown contrition and we consider the matter resolved. The Conservative party is funded by membership, fundraising and donations. All reportable donations are properly and transparently declared to the Electoral Commission, published by them, and comply fully with the law. Indeed, such observations can be made about who our donors are, precisely because our donations are transparently published.

“Fundraising is a legitimate part of the democratic process. The alternative is taxpayer funding of political campaigning, which would mean less money for frontline services like schools and hospitals – or being in the pocket of the trade unions, like the Labour party.”