Rishi Sunak has said MP Lee Anderson’s remarks that sparked an Islamophobia row “weren’t acceptable, they were wrong”, as he denied the Conservative Party has Islamophobic tendencies.

The Prime Minister addressed the row on Monday after facing growing calls to speak out about the former Tory deputy chairman’s remarks. 

Mr Anderson lost the Conservative whip over the weekend after failing to apologise for claiming “Islamists” had “got control” of Sadiq Khan and London. However critics including the London mayor and Tory peer Baroness Warsi hit out at Mr Sunak for failing to explicitly condemn the comments.

Speaking to local BBC radio stations during a visit to North Yorkshire, the Prime Minister said: “I think its incumbent on all of us, especially those elected to Parliament, not to inflame our debates in a way that’s harmful to others.

Lee’s comments weren’t acceptable, they were wrong. And that’s why he had the whip suspended.”

Mr Sunak continued: “Clearly his choice of words wasn’t acceptable, it was wrong. Words matter, especially in the current environment where tensions are running high and I think its incumbent on all of us to choose them carefully.”

When asked whether his party has an Islamophobia problem, the Prime Minister said: “No, of course it doesn’t”.

A Conservative party source had defended Mr Anderson’s comments on Friday night, before he was stripped of party support on Saturday amid mounting condemnation from across the political divide.

It came after one of the most fractious weeks in Westminster in recent years, which saw Parliament descend into chaos over a row about the handling of a Commons vote on Gaza and concerns for MPs’ safety.

Mr Anderson, the Ashfield MP who is standard bearer for the Tory right, will now sit as an independent unless he defects to another party that chooses to offer him its backing.

Earlier, Cabinet minister Mark Harper left the door open for Mr Anderson’s possible return to the Tory party.

When asked what he needs to say to be welcomed back, the Transport Secretary old Sky News: “I hope he will reflect on what he said and he will retract those comments and apologise… He’s contributed a lot in the past. I’d like to see him be able to contribute to the Conservative Party in the future.”

When asked what message the consideration of a return to the party sends to British Muslims, Mr Harper said his suspension “sends a very strong message that we don’t tolerate people saying such things in the Conservative Party”.

He declined to say whether Mr Anderson’s remarks were racist despite being pressed repeatedly, reiterating only that they were “wrong” and “not true”.

Labour chairwoman Anneliese Dodds said condemned Mr Anderson’s remarks as “appalling” and said they were “clearly not to do with who Sadiq Khan is, it was a slur that was directed at him because of Islamophobia”.

She said she has written to the Conservative Party chairs seven times over the past three years calling for them to take action against Islamophobia in the party.

During BBC Radio 4’s Today Programme Ms Dodds said: “This isn’t something that popped up five minutes ago, it is a continuing concern.”

Liberal Democrat deputy leader Daisy Cooper said: “Rishi Sunak needs to condemn Anderson’s comments for what they are, Islamophobic and racist, and make clear he won’t be let back into the Conservative Party.”