Walthamstow is a large suburb of north-east London that bustles with all the messy life and colour of the multicultural inner city. On a short, smart, pedestrianised stretch of road to the east of its high street, however, there is a sharp change in mood.

Here on Orford Road, close to what was once the original rural village of Walthamstow, the lively buzz gives way to something much more refined. Instead of the betting shops and halal butchers of the cheerful high street, there is an antiques shop, a tapas deli and some very nice independent cafes.

But it is also now at the centre of a public spat, as the diners and shoppers of Walthamstow village this week found themselves at odds with one of Britain’s fastest-growing cafe groups. Rumours are afoot that the upmarket bakery chain Gail’s plans to open an outlet on the street, and some locals are not happy.

A petition started by a local man, James Harvey, to “prevent Gail’s from establishing a store on our high street”, had attracted several hundred signatures by midweek – and with it widespread media coverage.

The brand brought “a risk of overshadowing our much-loved local stores”, Harvey wrote. “Let us together safeguard the soul of our beloved neighbourhood.”

Gail’s has been prized by many for its artisanal-style produce and neighbourhood cachet since its first outlet opened in Hampstead, north-west London, in 2005. But having doubled its number of locations in three years thanks to a venture capital-fuelled expansion, it is no longer found only in the leafiest urban villages.

There are now 131 branches of Gail’s in the UK, and plans for 35 more this year. Its managing director has said there may be scope for 300-500 across the UK. Is Walthamstow’s protest, then, the start of a backlash? And why has this particular middle-class neighbourhood objected to a brand that has thrived in plenty of others?

On a sunny mid-morning this week, Jodie Davis was entertaining her young son and nephew at a table outside the Village Bakery, one of the longest-established cafes on Orford Road. Davis lives in Chelmsford, Essex, 25 miles (40km) away, but said she often came here because “it’s just very quiet and villagey, everyone is polite, everyone knows everyone else”.

Like many to whom the Guardian spoke in the village, she was aware of the debate over the Gail’s proposal, and generally opposed it. “I don’t think they should put it here,” she said. “Everything is unique down here. There’s nowhere else like this.”

But not everyone agreed. “We would welcome it, to be honest,” said Helen Porter, who lives nearby and was feeding her baby daughter, Beatrice, outside another cafe, Eat 17. She and her partner, Todd Calas, like Gail’s and sometimes go to the branch in nearby Wanstead for a change of scene while she is on maternity leave.

“My only worry would be if it was a waterfall effect – I wouldn’t want the whole street to become a chain,” she said.

Some reporting of the petition in rightwing outlets has focused on a small number of comments objecting to the pro-Brexit views of the retail entrepreneur Luke Johnson, a minority investor, or on loaded social media commentary about its Israeli baker founder, Gail Mejia. The reports have suggested lefty Walthamstow’s objection to Gail’s is political.

But very few to whom the Guardian spoke knew much about the company’s background other than what they had read this week. “Everyone has got their own [political] views but I don’t think it’s that,” said Billy Smith, the manager of The Kitchen, a small restaurant. “This is just a lovely independent street. Having a chain like Gail’s come in is not ideal.”

Gail’s told the Guardian it was “a UK-based business with no specific connections to any country or government outside of the UK and does not fund Israel”. It said: “We understand the concern around chains, but our view is that a healthy high street is one with a diversity of quality offers, each delivering their best … All of our bakeries exist in areas where the choices are wide and growing – we should be celebrating the improvement in our food landscapes.”

Amy North, the editor of British Baker magazine, said she thought the Walthamstow objections, while hardly welcome PR, were unlikely to check the brand’s progress.

“Opening a bakery is a huge investment and they’ll have done their research before looking at this neighbourhood. And as much as it has generated a lot of press, the number of people to sign that petition is still under 1,000. I’m sure there are many more people than that in Walthamstow that would use it,” North said.

Adam Leyland, the editor-in-chief of the Grocer, said there were “many cautionary tales” of beloved neighbourhood businesses expanding too fast but Gail’s had shown it “knows its market”.

“Gail’s is very successful. It’s hugely popular. It increases the value of houses and property in its neighbourhoods, and it has a formula that is winning,” he said. “If it thinks that it can double in size, then … what that says is that there’s an awful lot of middle-class neighbourhoods.”