Rishi Sunak abandoned his “legacy” policy to ban smoking for future generations amid a backlash from the tobacco industry in the form of legal threats, lobbying and a charm offensive aimed at Conservative MPs, an investigation reveals.

The UK had been on course to become the first country to ban smoking for future generations, via the tobacco and vaping bill, which Downing Street hoped would help define Sunak’s place in British political history.

An investigation by the Guardian and the Examination, a non-profit newsroom that investigates global health threats, has uncovered how the UK’s largest cigarette companies fought against the policy, which would have raised the smoking age by one year every year.

After months of fierce opposition from the industry – and intervention from MPs and thinktanks with ties to tobacco firms – the proposal was excluded from the “wash-up” process, when outgoing governments choose which policies to fast-track and which to drop.

The policy, which in effect banned smoking for anyone born after 2008, was left out despite MPs having voted in favour of it.

Documents and freedom of information requests reveal how four of the world’s largest tobacco firms – the UK’s Imperial Brands and British American Tobacco (BAT), Japan Tobacco International (JTI) and US-headquartered Philip Morris International (PMI) – put ministers on notice of a legal backlash.

Imperial and BAT wrote to the health secretary, Victoria Atkins, in February, to claim the consultation process preceding legislation was “unlawful” because industry views had not been considered.

The Department of Health and Social Care has said it did not need to consider industry views, pointing to guidance included in a World Health Organization global treaty, signed by the UK, that says governments should form smoking policy without influence from cigarette companies.

The Marlboro-owner PMI and JTI, which makes Camel and Benson & Hedges, said the treaty permitted interactions with cigarette firms if they were “necessary”.

Imperial, which owns Lambert & Butler and Gauloises, followed up its warning with a legal letter threatening a “judicial review” challenging the consultation process.

Government lawyers responded by saying legal action might “derail” a bill that ministers believed could save tens of thousands of lives and billions of pounds in NHS costs.

BAT, JTI and PMI were named as interested parties in Imperial’s letter, giving them the right to join as co-claimants if a judicial review went ahead.

Imperial, which sells half of all of the cigarettes smoked in the UK, has not filed court proceedings but a spokesperson said the company was “keeping the situation under review as we monitor legislative developments”.

The legal threats came after the industry opposed the legislation in its submissions to the consultation, despite claiming publicly that they wanted to phase out cigarettes.

PMI’s chief executive, Jacek Olczak, indicated in 2021 interviews that it could stop selling cigarettes in the UK within 10 years.

However, the company’s UK subsidiary told the consultation it “did not support the age of sale ban as outlined”, arguing instead for “further restrictions” on “combustible tobacco” – ie cigarettes – instead of an outright ban.

PMI told the Guardian the bill risked “confusing” consumers because it included restrictions on some smoke-free products such as vapes, adding that it “firmly believes in phasing out cigarettes, to the benefit of the 6.4 million adult smokers in the UK”.

BAT, which has previously advocated for a “smokeless” future, proposed raising the age of sale to 21 instead.

As the government pressed ahead with its plans despite opposition, tobacco firms courted rightwing and libertarian Tory MPs. In January, three months after Sunak announced his policy on smoking, the then Clacton MP, Giles Watling, attended a “business lunch” with officials from JTI. Two months later, he went to the company’s annual party at the British Museum in London.

In May, he proposed an amendment that would have replaced Sunak’s proposals with a new minimum age of 21.

“We are strongly of the opinion that engagement results in better and more informed policy and is therefore in the best interests of all relevant parties,” JTI said. Watling did not return requests for comment.

Other MPs targeted by the tobacco industry included the business secretary, Kemi Badenoch. The Imported Tobacco Products Advisory Council wrote to her in March complaining about the “open dismissal of industry views in the decision making process”.

Its secretary general, Tatiana Camacho, accused the health department of “taking a position that seems to go against the spirit of collaboration and inclusiveness”.

A month later Badenoch voted against the smoking ban at its second reading in the Commons, posting on X at the time: “The principle of equality under the law is a fundamental one … We should not treat legally competent adults differently in this way, where people born a day apart will have permanently different rights.”

Two Conservative MPs, including Badenoch’s closest political aide, also attended a lunch and drinks reception hosted by the smoking lobby group Forest, days before the tobacco bill was in effect shelved.

The Beat the Ban event, at Boisdale restaurant in Belgravia, London, featured beermats depicting Sunak as a nanny, an allusion to the ban being a “nanny state” measure.

Badenoch’s parliamentary private secretary, Alexander Stafford, attended alongside fellow Conservative MP Andrew Rosindell. Tobacco lobbyists present included Camacho and Richard Cleary, who left Badenoch’s department to join Imperial in January.

Stafford has said he does not smoke but opposed the ban as a “lover of freedom, a lover of choice and a lover of information”.

He and Rosindell did not return a request for comment.

The Forest director Simon Clark said the organisation did not seek to persuade MPs who attended to oppose the bill becoming law but did outline the group’s position in a speech.

The government also came under pressure from rightwing thinktanks funded by the tobacco industry during the consultation process.

In total, there were 307 responses in which the respondent disclosed ties to the tobacco industry, including from the Institute of Economic Affairs (IEA) and Adam Smith Institute. Both have received funding from JTI, while the IEA has also received money from Imperial and BAT.

BAT, the owner of Lucky Strike and Rothmans, said: “We are clear that combustible cigarettes pose serious health risks, and the only way to avoid these risks is not to start smoking or to quit.

“However, we do not believe that a generational sales ban will have the desired impact given the serious unintended consequences that are likely to follow, such as age verification being difficult to manage, and an increase in illicit trading.”

The Conservative party did not return requests for comment.

• This article was amended on 1 July 2024. The smoking ban that was abandoned by the Conservative government would have effectively banned smoking for anyone born after 2008, not 2009 as an earlier version said.