Best piece I read in a long time about the "free market/profit/capitalism versus ethics" conundrum. Thank you.
Doing nothing effectively is passively accepting/normalising nazi content. Libertarians naively overestimate human intelligence. But quantity does have its own 'quality'. Volume matters. Zero isn't realistic but near-zero is necessary to protect democracy and the rule of law.
My analogy was how much black mold is acceptable to have in your home but I'm landing the same place you are and I think you've added some important context. Thanks Dave.
"Tech libertarianism is, fundamentally, an ideology for people who are both cheap and lazy.": That's the heart of the matter. More broadly, it explains why so much that comes from "Silicon Valley" is so vapid or scammy.
If you take it seriously, content moderation is indeed difficult and hence a revenue sink. It's far easier to design, build, and run the user-facing functionality of a website like Substack than to design, build, and run good moderation for it. (I speak as someone who designs, builds, and runs web applications for a living.) Moreover, the latter requires different skills, in that the core of good moderation is intelligent human judgment. Software can help with applying that judgment systematically, but moderation isn't primarily a software-development problem. (And no, what passes for artificial intelligence these days isn't nearly up to the job.)
Of course, that doesn't excuse Substack - or Facebook or Twitter or anyone else - for not making the effort. For me, as a decent person (and a software developer), the acceptable options for a website that publishes user-generated content are either do it well, including content moderation, or don't do it at all. Doing a half-assed job or blowing moderation off altogether aren't acceptable options for decent people.
A nice summary of ugly reality. The essence of libertarianism really is "yeah, that's a problem. Somebody should do something about it" when doing something doesn't involve making a profit. Though I think everyone is leaning a little too hard on "mmm, Nazis are bad, ok?" as the totality of the argument for banning Nazis, pointing to Indiana Jones and Blues Brothers memes when pressed on it. I get that arguing about a sloppy pastiche of white supremacy, Nordic mysticism, and government enforced misogyny is like nailing Jell-o brand gelatin to the wall, but John Q. Public oft times needs this shit spelled out. I like that the letter simply asks Substack to chirp or get off the twig, as my mom would say. Own this shit or deal with it. I'd be proud to sign that letter.
Love the Paul Dawson flashback! He was a great professor.
But in this case, Substack keeps getting reports about the sources of the mouse poop and they have the tools to eliminate each specific source (knowing that more sources might emerge and poop before they know about it). I do think that a regulation that says "if you see a mouse in your facility that you can eliminate, you are required to try to do so" is a pretty low bar.
"The one thing with writing stories about the rise of fascism is that if you wait long enough, you'll almost certainly be proved right. Fascism is like a hydra - you can cut off its head in the Germany of the '30s and '40s, but it'll still turn up on your back doorstep in a slightly altered guise."
Keep in mind we’re talking about text newsletters here. There is not a simpler use for an LLM than “read all the monetised newsletters and flag any that might be white nationalist content”.
So if the delta between “zero” and “near zero” is the cost of moderation, that delta is now an order of magnitude smaller than it was two years ago. Substack truly has no excuse. By not taking any action they are explicitly saying “Nazis are OK”.
Thank you for writing this Dave, it’s spot on. I put my frustrations down in words last week and moved my newsletter elsewhere. I hope the mass of noise today makes a difference and there’s some genuine movement from those in charge - sadly seems to be an approach that favours economics over ethics so far.
So, in order to suppress nazi content, you advocate and employee the exact tactics used by the nazis to suppress thought, speech and freedom of their people?!?
I see Substack as a magazine rack, full of a variety of different magazines. First, I don’t even have to walk by it; if I do, I don’t have to stop and look. However, if I want to stop by and pick up the latest recipe magazine, I can do that, without having to buy, a single copy of Vogue, Southern Living or any other publication I fail to fancy.
You want to do something useful and meaningful; figure out the real environmental impact of lithium, cobalt battery powered EVs.
...Are you sure these are the *exact* tactics used by the nazis?
And also, if I was walking through Barnes & Noble and they had a copy of, I dunno, "White Nationalist Monthly," I would probably do something extreme like approach the manager and say "hey, um, why are you carrying that publication?"
Mentioned this elsewhere but In Germany where “hate speech” is banned the far right (but now somewhat moderate) are not only surging but expected to form part of the government next election if not win outright.
Nazism is an abomination. I can’t believe I have to write this. Tolerating it gives it power. It doesn’t deserve freedom of speech, and the only reason the tech bros permit it is, as the other commentators attest, laziness and cheapness.
The good thing, though, is when they strut their weird 1930s stuff, you can dox them and let everyone know where literal copraphages live.
This is excellent. Seems like a companion piece to Popehat’s work to help understand how the first amendment and private content moderation interact with each other.
Great piece. I’m wondering about this in the context of higher ed and the free speech debates. Trial number of hate speech activities but not zero might be the only answer there too
I’m not in higher ed but really care about education as a whole. If you have to require only historically accurate arguments be made, then a lot of these bigots won’t be able to make arguments because their goal is to change history in the favor of their aesthetic ideals. It’s why you won’t find them participate in real debates or have fact checkers
Your entire article stating why Substack should start moderating Nazis is based on the ‘bandwagon fallacy’. It's basically saying that if a couple hundred people have signed a letter demanding that Substack take action and everyone is discussing it, it therefore must be a legitimate request. But this reasoning itself fails because there are tens of thousands of Substackers who didn't sign the letter and are unaware of Substack's ‘Nazi’ problem.
Most of the people who haved signed the letter have never launched or managed a social media platform. The managers of Substack have to focus on the long view with every decision they make. Substack's approach to content moderation is similar to that of Google and Amazon and many other marketplaces. They basically align with the Supreme Court's view of first amendment rights which is that speech is legal so long as it is not causing immediate danger to people, but it can be reasonably regulated as to the time and place. This is why you can still find Nazi sponsored content on Google and Amazon.
Substack is a social marketplace where people can exchange content for money. Most of that content comes in the form of ideas. Writers write down their ideas and if people like the ideas they can subscribe to read more. So, if you are building a marketplace of ideas, it makes no sense to eliminate certain ideas from the market based on the idea itself. How can we even assess the value of an idea if it's not allowed onto the market.
The power of the pen is mighty, but the power to silence the pen is mightier, and that isn't a power that I want to give to any individual or organization. What if the 200 or so Substackers who signed the letter and are now posting about having signed the letter had instead written articles denouncing and critiquing the content of the most popular Nazi substacks. We would be having a different kind of discussion, right now. Instead, the signers of the letter have made a literal appeal to Big Brother, someone mightier than the pen, to step in and determine what ideas can and can't be allowed into the marketplace of ideas. And to think that it will stop with 12 Nazi accounts is ridiculous. It would just be the beginning. Jonathan Katz has been complaining about the content of other substacks since the day Notes was launched and there are plenty of other ‘deplorable’ accounts that will be future targets of the Thought Police.
Look at how much power the managers of Substack have reserved for themselves by simply not discussing the letter. Do you want them to have more?
If you believe that Google and Amazon are pure marketplaces and do not take down any hateful/offensive/outright-Nazi content, I don't know what to tell you. Read anything about content moderation published in the past decade.
If you believe, post-Charlottesville, that the only proper way to confront White Nationalists is to treat their ideas as though they are legitimate and serious, and then thoroughly debunk them, I don't know what to tell you. It sounds as though you travel in circles where the greatest potential threat comes from imaginary Thought Police. That must be nice.
Substack is a business. 200 of its writers have informed the business that we are concerned with their lax enforcement of existing content moderation policies. Switching from Substack to another platform would be a bit of a hassle. If remaining with Substack becomes the greater hassle, we will all leave. That won't be great for Substack's business. That collective voice deserves a response, not because of the bandwagon fallacy, but because if the company pisses off the writers, that becomes the company's problem.
The actual thought police aren't liberal college students complaining about crappy banh mi but are instead Florida governors demanding the removal of any reference to the existence of homosexuality and transgendered people.
"It's the marketplace of ideas!!". Sure, but different storefronts sell different things, and serve different customers. Not every store has to sell dung.
The only reason SubStack wants sell Nazis is they want to be the ONLY store. Tech Monopolies again.
If they give up the monopoly there is no issue - but there's also no "Infinite Growth" story for their VCs
Best piece I read in a long time about the "free market/profit/capitalism versus ethics" conundrum. Thank you.
Doing nothing effectively is passively accepting/normalising nazi content. Libertarians naively overestimate human intelligence. But quantity does have its own 'quality'. Volume matters. Zero isn't realistic but near-zero is necessary to protect democracy and the rule of law.
My analogy was how much black mold is acceptable to have in your home but I'm landing the same place you are and I think you've added some important context. Thanks Dave.
"Tech libertarianism is, fundamentally, an ideology for people who are both cheap and lazy.": That's the heart of the matter. More broadly, it explains why so much that comes from "Silicon Valley" is so vapid or scammy.
If you take it seriously, content moderation is indeed difficult and hence a revenue sink. It's far easier to design, build, and run the user-facing functionality of a website like Substack than to design, build, and run good moderation for it. (I speak as someone who designs, builds, and runs web applications for a living.) Moreover, the latter requires different skills, in that the core of good moderation is intelligent human judgment. Software can help with applying that judgment systematically, but moderation isn't primarily a software-development problem. (And no, what passes for artificial intelligence these days isn't nearly up to the job.)
Of course, that doesn't excuse Substack - or Facebook or Twitter or anyone else - for not making the effort. For me, as a decent person (and a software developer), the acceptable options for a website that publishes user-generated content are either do it well, including content moderation, or don't do it at all. Doing a half-assed job or blowing moderation off altogether aren't acceptable options for decent people.
A nice summary of ugly reality. The essence of libertarianism really is "yeah, that's a problem. Somebody should do something about it" when doing something doesn't involve making a profit. Though I think everyone is leaning a little too hard on "mmm, Nazis are bad, ok?" as the totality of the argument for banning Nazis, pointing to Indiana Jones and Blues Brothers memes when pressed on it. I get that arguing about a sloppy pastiche of white supremacy, Nordic mysticism, and government enforced misogyny is like nailing Jell-o brand gelatin to the wall, but John Q. Public oft times needs this shit spelled out. I like that the letter simply asks Substack to chirp or get off the twig, as my mom would say. Own this shit or deal with it. I'd be proud to sign that letter.
Thank you for this (I had been wanting to make a similar, "more than 0 but few" argument, and this is well done).
Love the Paul Dawson flashback! He was a great professor.
But in this case, Substack keeps getting reports about the sources of the mouse poop and they have the tools to eliminate each specific source (knowing that more sources might emerge and poop before they know about it). I do think that a regulation that says "if you see a mouse in your facility that you can eliminate, you are required to try to do so" is a pretty low bar.
"The one thing with writing stories about the rise of fascism is that if you wait long enough, you'll almost certainly be proved right. Fascism is like a hydra - you can cut off its head in the Germany of the '30s and '40s, but it'll still turn up on your back doorstep in a slightly altered guise."
― Alan Moore
Keep in mind we’re talking about text newsletters here. There is not a simpler use for an LLM than “read all the monetised newsletters and flag any that might be white nationalist content”.
So if the delta between “zero” and “near zero” is the cost of moderation, that delta is now an order of magnitude smaller than it was two years ago. Substack truly has no excuse. By not taking any action they are explicitly saying “Nazis are OK”.
Thank you for writing this Dave, it’s spot on. I put my frustrations down in words last week and moved my newsletter elsewhere. I hope the mass of noise today makes a difference and there’s some genuine movement from those in charge - sadly seems to be an approach that favours economics over ethics so far.
https://open.substack.com/pub/tumshie/p/so-long-and-thanks-for-all-the-bile?r=2ceix&utm_medium=ios&utm_campaign=post
So, in order to suppress nazi content, you advocate and employee the exact tactics used by the nazis to suppress thought, speech and freedom of their people?!?
I see Substack as a magazine rack, full of a variety of different magazines. First, I don’t even have to walk by it; if I do, I don’t have to stop and look. However, if I want to stop by and pick up the latest recipe magazine, I can do that, without having to buy, a single copy of Vogue, Southern Living or any other publication I fail to fancy.
You want to do something useful and meaningful; figure out the real environmental impact of lithium, cobalt battery powered EVs.
...Are you sure these are the *exact* tactics used by the nazis?
And also, if I was walking through Barnes & Noble and they had a copy of, I dunno, "White Nationalist Monthly," I would probably do something extreme like approach the manager and say "hey, um, why are you carrying that publication?"
Why? How does it become any of your motherfucking business what ideas other people are exposed to? Are you a Nazi? You’re acting like one.
Always a nazi accusing others of being nazis so they can tell themselves they aren’t nazis
Mentioned this elsewhere but In Germany where “hate speech” is banned the far right (but now somewhat moderate) are not only surging but expected to form part of the government next election if not win outright.
Nazism is an abomination. I can’t believe I have to write this. Tolerating it gives it power. It doesn’t deserve freedom of speech, and the only reason the tech bros permit it is, as the other commentators attest, laziness and cheapness.
The good thing, though, is when they strut their weird 1930s stuff, you can dox them and let everyone know where literal copraphages live.
I wrote my own, if you’re interested.
https://open.substack.com/pub/zerograv/p/the-atlantic-has-a-fallacy-problem?r=lqhl5&utm_medium=ios&utm_campaign=post
This is excellent. Seems like a companion piece to Popehat’s work to help understand how the first amendment and private content moderation interact with each other.
Great piece. I’m wondering about this in the context of higher ed and the free speech debates. Trial number of hate speech activities but not zero might be the only answer there too
I’m not in higher ed but really care about education as a whole. If you have to require only historically accurate arguments be made, then a lot of these bigots won’t be able to make arguments because their goal is to change history in the favor of their aesthetic ideals. It’s why you won’t find them participate in real debates or have fact checkers
Your entire article stating why Substack should start moderating Nazis is based on the ‘bandwagon fallacy’. It's basically saying that if a couple hundred people have signed a letter demanding that Substack take action and everyone is discussing it, it therefore must be a legitimate request. But this reasoning itself fails because there are tens of thousands of Substackers who didn't sign the letter and are unaware of Substack's ‘Nazi’ problem.
Most of the people who haved signed the letter have never launched or managed a social media platform. The managers of Substack have to focus on the long view with every decision they make. Substack's approach to content moderation is similar to that of Google and Amazon and many other marketplaces. They basically align with the Supreme Court's view of first amendment rights which is that speech is legal so long as it is not causing immediate danger to people, but it can be reasonably regulated as to the time and place. This is why you can still find Nazi sponsored content on Google and Amazon.
Substack is a social marketplace where people can exchange content for money. Most of that content comes in the form of ideas. Writers write down their ideas and if people like the ideas they can subscribe to read more. So, if you are building a marketplace of ideas, it makes no sense to eliminate certain ideas from the market based on the idea itself. How can we even assess the value of an idea if it's not allowed onto the market.
The power of the pen is mighty, but the power to silence the pen is mightier, and that isn't a power that I want to give to any individual or organization. What if the 200 or so Substackers who signed the letter and are now posting about having signed the letter had instead written articles denouncing and critiquing the content of the most popular Nazi substacks. We would be having a different kind of discussion, right now. Instead, the signers of the letter have made a literal appeal to Big Brother, someone mightier than the pen, to step in and determine what ideas can and can't be allowed into the marketplace of ideas. And to think that it will stop with 12 Nazi accounts is ridiculous. It would just be the beginning. Jonathan Katz has been complaining about the content of other substacks since the day Notes was launched and there are plenty of other ‘deplorable’ accounts that will be future targets of the Thought Police.
Look at how much power the managers of Substack have reserved for themselves by simply not discussing the letter. Do you want them to have more?
<deep sigh>
If you believe that Google and Amazon are pure marketplaces and do not take down any hateful/offensive/outright-Nazi content, I don't know what to tell you. Read anything about content moderation published in the past decade.
If you believe, post-Charlottesville, that the only proper way to confront White Nationalists is to treat their ideas as though they are legitimate and serious, and then thoroughly debunk them, I don't know what to tell you. It sounds as though you travel in circles where the greatest potential threat comes from imaginary Thought Police. That must be nice.
Substack is a business. 200 of its writers have informed the business that we are concerned with their lax enforcement of existing content moderation policies. Switching from Substack to another platform would be a bit of a hassle. If remaining with Substack becomes the greater hassle, we will all leave. That won't be great for Substack's business. That collective voice deserves a response, not because of the bandwagon fallacy, but because if the company pisses off the writers, that becomes the company's problem.
The actual thought police aren't liberal college students complaining about crappy banh mi but are instead Florida governors demanding the removal of any reference to the existence of homosexuality and transgendered people.
Another fucking censor who wants to ban us from seeing dangerous ideas.
You know what to do when you see Nazi related content?
JUST DONT LOOK JUST DONT LOOK JUST DONT LOOK!!!
And your tiny little overly sensitive brains won’t get their feelings hurt.
Ah, there’s the nazi
“Nazi” = “someone who beat me in an argument”. Sort of a broad definition.
Nazis always say they win “arguments” to hide their bigotry
"It's the marketplace of ideas!!". Sure, but different storefronts sell different things, and serve different customers. Not every store has to sell dung.
The only reason SubStack wants sell Nazis is they want to be the ONLY store. Tech Monopolies again.
If they give up the monopoly there is no issue - but there's also no "Infinite Growth" story for their VCs