The problem is that by that point it will have grown beyond manageability. You know the "Nazi bar" saying.
There's a bunch of people (who are Nazis) and they seem cool, quiet, well spoken, just having a drink. And they bring their friends and those guys are cool too. Then those guys bring their friends and those guys are less cool and now normal people don't drink at the bar anymore and you look around and it's a Nazi bar and you can't make them leave or they'll start causing "problems". So. I'm all for just using the brutal hammer of censorship.
It's not a free speech platform and no one ever said it was.