Ray Kelvin never wanted to be the face of the fashion brand he founded in 1988, so he called it Ted Baker – a name that came up in conversation. A reclusive character, he has rarely been photographed without a prop to cover half of his face.

Kelvin has said he invented the fictional character Ted Baker to separate himself from the brand in case it went bust. He has now quit as chief executive from the chain he built from a single shirt shop in Glasgow into a global brand – one of Britain’s most successful fashion stories – following allegations of harassment reported by the Observer in early December.

The 63-year-old north Londoner, who was awarded a CBE for services to the fashion industry in 2011, has fashion in his blood. His father Alf owned a small blouse factory in Tottenham, and he worked in the family tailor’s shop from the age of nine alongside his mother Trudie, who later worked in Ted Baker stores until her death in 2011.

In the early 1970s, Kelvin started supplying womenswear to high street retailers through PC Clothing Ltd, before spotting an opportunity to specialise in men’s shirts. Ted Baker was born as a shop selling 400 types of shirt in Glasgow with the backing of the Scottish department store group A Goldberg & Sons.

The first few years were tough – the founder sometimes slept on the floor in the shops – but quirky details on Ted Baker’s shirts, suits and dresses helped the brand stand out in the fashion world.

Kelvin, who was paid £1.3m last year and is leaving without a payoff, ran the company as chief executive from the outset and floated it on the London Stock Exchange in 1997. Ted Baker now has 550 outlets worldwide and annual sales of nearly £600m.

Described as “the man closest to Ted” on the firm’s website, Kelvin managed to keep a low profile, even though he was married to the Emmerdale star Georgia Slowe, his first wife with whom he had two sons, now in their 20s. He married his second wife Clare, with whom he has a daughter, in 2012.

Kelvin told the Wall Street Journal in 2017 about the “hug zone” – a demarcated area outside his office where employees would receive an embrace. The area was removed three months ago, after more than 300 current and former staff signed a petition on the Organise campaigning platform calling for an end to “forced hugs” and alleged harassment.

The Guardian also revealed that before the harassment allegations, Kelvin’s board had given him a formal warning following an incident in which he shoved a senior executive against a wall. The chief executive had lost his temper after learning he hadnot been invited to the colleague’s wedding.

Ted Baker called in a City law firm to investigate and began an overhaul of its culture. Kelvin then took a voluntary leave of absence after the company’s board was made aware of “further serious allegations.” He still owns about 35% of the company, and has indicated that he will hold on to the stake, worth about £300m.