Standing on the shingle of Hastings beach beside his fishing boat, Paul Joy says his family have fished these waters since before the time of William the Conqueror. He’s done the research, and the Joys have had boats here since records began. His boat had been out at dawn that morning, hugging the coastline with a skipper, two crew and the “boy” ashore, aged 80, who winches the Kaya up the pebbles with a rusty bulldozer. “My father, Will, born in 1906, made an adequate living with a smaller boat than mine. Now everyone subsidises fishing with other work. I earned the same as a carpenter, now you’re lucky to get a Tesco shelf-stacker’s pay.” Fishing quotas are killing the small fishermen, he says.

Stand here to breathe in the heart of Brexit. Last week, in ports around the country, fishing boats protested – sending up flares to oppose the transition deal that leaves them in the common fisheries policy (CFP), but without a seat at the EU table sharing out the fish. Protest organisers said: “We are sickened by remainers gleefully peddling the deliberate narrative that fishing doesn’t matter.”

Does it matter? The fishing industry is worth less than 0.5% of GDP. They rightly fear this pinprick to the economy will be traded for bigger prizes – finance, cars, pharma, airline routes. Why else, they ask, is the government’s fisheries white paper delayed time and again? Thirteen Rees-Moggites and one DUP MP swear they’ll vote down any deal unless the UK breaks from the CFP. As these hard Brexiteers want no deal anyway, fishing is their perfect pretext, though no deal would be the fishermen’s apocalypse.

Here the referendum was lost, in the romance of the sea, the rugged cliffs and coasts of our island story among old salt spirits of a seafaring nation. This is the symbolism the remainers fatally forgot, tone deaf to the draw of John Masefield’s poem Sea Fever or Millais’s The Boyhood of Raleigh. Economics says fishing is of nugatory value, but politics says fishing is deep-dyed in national identity, down to the last fish and chip shop. That heart-versus-head error is why remain lost – and why its campaigners still fail to connect with unreasoning national sentiments. They forget those sentiments can be cashable too: Hastings’ 27 picturesque boats, huts and fish stalls bring the impoverished town £5m a year in tourism.

Joy heads the small-fishers’ organisation, Nutfa. He is a sharp campaigner, steeped in the fiendishly conflicting complexities of international fishing policy. If “take back control” meant anything, it was Britannia ruling her own waves. But that’s not going to happen, because it can’t.

Michael Gove, the secretary of state for environment, food and rural affairs, casually promised instant freedom from the fisheries policy, but backed down embarrassingly when our know-nothing negotiators were taken by surprise (again) to find other EU fishing nations just as adamant: those with their own fierce fishing traditions feel just as passionately. Who knew?

There always were and always will be fishing treaties between countries, now more than ever to preserve precarious stocks. Unless navies are to go to war to defend their fishing fleets, there must always be agreements. Fishing treaties existed before Britain joined the EU. Quotas fixed long ago often don’t seem fair now, nor do they preserve stocks well enough. But the EU is only one element: ask Grimsby fishers about friction between English and Scottish rights, as global warming sends cod migrating north. Fish don’t obey treaties.

Here’s the Brexiteers’ dishonesty. Each country is free to share out its national quota as it chooses – but free-market Britain, unlike others, let fishers sell their quotas abroad. The Dutch ship Cornelis Vrolijk, registered in Caterham, owns 23% of the entire UK quota. “Slipper skippers” sold their quotas abroad – it was easier to put their feet up than to fish. Could Gove seize it back post-Brexit? No more than Jeremy Corbyn could seize back rail or energy companies from foreign owners without hefty compensation. It’s not prevented by the EU, but by basic property law.

Gove could redistribute quotas between our own big ships and small boats. In Britain, 77% of the boats are less than 10 metres long, employing most of the UK’s 12,000 fishers, yet owning just 4% of local quota. A key article in the CFP says quotas should be allocated transparently and objectively, and include “social, economic and environmental criteria”. Small boats matter most for coastal life and do least environmental harm, so should take priority. Talk to Joy long enough about all the complexities of different fish quotas and treaties, and his most pressing need is for the British government to take from the big boats and give to the under-10m flotilla.

Gove has the power to do it, says Thomas Appleby, associate law professor at the University of the West of England, a fisheries law specialist. Britain could and should have banned the sale of its quotas, but Gove could repair some of the damage: so long as UK quota is taken back proportionally from all, shares could be redistributed to the small boats. Why doesn’t he do it? Joy points to the power of the big companies: two-thirds of UK quotas are taken by three multinationals. And he talks of business influence over the Conservatives. Here’s an irony: the EU fisheries fund has given Joy’s small boats lobby a grant to set up its own producer organisation as a political counterweight to fight the big boats.

What of the “taking back control” dream, our navy chasing foreign boats from our waters? Impossible, when 80% of UK fish is sold in the EU, which would retaliate with prohibitive tariffs and delaying checks at Calais to let fish rot on the quay. Bremerhaven is just one port eager to seize our fish-processing business, which employs 18,000 people. Consider what a trade war would do to the Scottish smoked salmon industry, worth more than all of the UK fishing industry.

At sea, the Brexiteers are coming face to face with hard truths: we are not alone. No man is an island entire of itself; every man is a piece of the continent. We trade, we exchange, we buy and sell. Fish was a bad emblem for the hard Brexiteers to choose because it’s an archetypal example of the need for cooperation – in fishing, conservation and sales. Trade-offs, haggling, deals and environmental necessity demand treaties between us and our neighbours, in or out of the EU. Except that outside it, our negotiators have a weak hand. The romance of the sea was good referendum propaganda; now it stands as the best example of why standing alone is impossible.

• Polly Toynbee is a Guardian columnist