Jeremy Corbyn sought to exploit Tory divisions over the effect of leaving the EU on workers’ rights during prime minister’s questions, highlighting pro-Brexit ministers’ statements on “reducing the burden” on employers of EU protection for workers.

A day after the Sports Direct boss, Mike Ashley, looked set to face a multimillion-pound bill following his admission that the company had failed to pay staff the national minimum wage, the Labour leader said David Cameron should distance himself from Priti Patel’s words that workers’ rights in EU law were a “burden”.

Ashley “would make Scrooge look like a good employer,” Corbyn said. “It shows we must strengthen and not weaken workers’ rights, especially when there are criminal activities involved,” he said, citing Patel, the pro-Brexit employment minister, who said the UK could “halve the burdens” on employers by leaving the EU.

“If we could just halve the burdens of the EU social and employment legislation we could deliver a £4.3bn boost to our economy and 60,000 new jobs,” she told the Institute of Directors in May.

The Labour leader, who has previously appeared reluctant to exploit Tory divisions during PMQs, returned several times to the cabinet disunity during his questioning.

“Does [Patel] speak on behalf of the government when she promises to reduce the burden, as she describes them, of employment legislation?” Corbyn asked, also referring to the justice secretary, Michael Gove, who said he “could not guarantee every person could keep their job” after a leave vote.

Gove appeared “equally relaxed about employment rights”, Corbyn said. “Can’t [Cameron] do something about that?”

Cameron said the EU’s underpinning of workers’ rights was one of the reasons many people would vote to stay in, but said the UK went over and above the rights demanded by the EU, on flexible working, maternity leave and shared parental leave, and more annual leave.

“There are ministers in the government who, in a personal capacity, are campaigning on another side of the argument,” Cameron said, emphasising that he did not agree with Gove or Patel’s statements.

Brushing over the divisions, Cameron said it should be celebrated that he and Corbyn, as well as the Lib Dems, the SNP, the Greens and Northern Irish parties, agreed on the principle that Britain should stay in.

Ashley, who may be disbarred as a company director, admitted Sports Direct effectively paid workers less than the minimum wage because of the time it took for them to be searched as they left the warehouses at the end of their shifts.

He admitted that 80% of workers were on zero-hours contracts, a practice which Corbyn asked the prime minister to ban.

Cameron said he “abhorred” Sports Direct’s failure to pay the minimum wage and that the government was taking strong action against companies who failed to pay the minimum, but said a consultation last year had found some workers preferred more flexible contracts.