I owned Trust & Safety, and cut the bill 52%+.

The four forces were safety, voice, scale and cost, and none of them could lose.

I oversaw moderation for our social apps. Most people hear that and think of approving and denying posts. That's the easy part. The real job was holding four things at once:

1. Safety. A space people feel safe showing up in.
2. Voice. Not silencing people, including the ones saying things we'd never say.
3. Scale. A system that makes the call fast and makes it right, while content keeps growing.
4. Cost. Vendor fees, infrastructure and people, all rising with every new post.

Push any one too far and another breaks. Max out safety and you silence people. Protect every voice and the space stops feeling safe. Put a person on every call and the cost runs away. Automate everything and the mistakes pile up.

None of the four ranks above the others. Every call I made had to land in the middle.

The gray area ran on rules that combine a vendor score with our own signals and end in one of four actions.

We partnered with one of the largest moderation vendors out there. It scores every post by category, and it made the clear calls easy. Nobody needs a meeting to take down violence.

The hard part is the line between edgy and harmful. A score in the middle of the range doesn't tell you what to do. The standard move is to pick a cutoff and remove everything above it, and that's how you silence people by accident.

So I wrote the logic for the middle. Each rule takes a gray-band score and asks what else is true. Who is it aimed at? How new is the account? Has anyone else reported it? What does it link to? The answers route the post to one of four actions, and removal is only one of them.

Sensitive and watch are the two that protect voice. The post stays up, behind a filter or under watch, while we learn more. I've left the real signals and thresholds off this page. Publishing them would teach people how to get around them.

The timeout paused posting on a doubling clock, from four hours to a month, and left viewing on.

A ban sends a bad actor off to make a new account. Now you're playing whack-a-mole with someone you can't see anymore.

So we put people on timeout. If you broke a rule, we told you what happened and paused you. You could still browse and like. You couldn't post, comment or send messages. Each repeat roughly doubled the clock.

That shut down the things that do the damage: spam runs, pump-and-dumps, comment pile-ons, harassment in DMs. And it kept bad actors on accounts we already knew. A known account with a history is far easier to manage than a fresh one with none.

The mistakes: we silenced some people by accident, and we told them so.

Not everyone we removed was a bad actor. Some people tripped a rule without meaning to, and some got caught by a model that read their post wrong.

The easy instinct is to defend the call. We went the other way. We said plainly that we use third-party tools, that they get things wrong, and that we keep sharpening them. We thanked people for their patience and fixed what we got wrong.

I built how we talked to people when we took something down. Every notice said what happened, named the rule in plain words, told them what they could still do, and gave them a way to talk to us. The reply path was a real conversation with our team. People could tell us what they meant and hear why we made the call.

We learned that people are fine with mistakes as long as they trust you're being straight with them. Transparency and how fast we acted were what our users valued most.

The cost came down when we stopped sending every post to the vendor and saved people for the calls only people can make.

The vendor charged by usage. As our audience grew, our content grew much faster, and the fees climbed with it. People were the most expensive step of all, so every post that needed a human was the one I most wanted to avoid.

I asked one question of every post. Does this need to leave our system at all? Three things came out of it:

1. Blocked domains. Links to adult and unlawful sites couldn't be posted, so there was nothing to send.
2. Spam strings. Spam repeats itself. We matched the same unusual text on our side, tied it to the account, and stopped it before it reached the vendor.
3. Only what needs a score. Not every post needs a scan. We sent the ones that actually needed a call.

The gray-area rules did their part at the other end. They settled middle calls on their own, so fewer of them needed a person.

The measurement was cost to moderate against content volume, with time to action next to it.

The number I watched was what it cost us to moderate, tracked against how much content came in. If the work was paying off, those two lines would split. If cost had kept pace with content, the filters weren't doing anything.

Next to it I watched time to action, how long a post sat before we made a call. Cheaper and slower would have been a bad trade, since speed was part of what earned trust. On the timeout side, I watched whether people came back clean or came back as someone new.

A moderation bill that grows with the audience is a tax on growth.

The result: our AI moderation bill fell 52%+ while content grew.

As the audience grew and the content library grew with it, our AI moderation bill fell by more than 52%. Time to action held steady, and moderators spent over 26% less time on the queue. Fewer posts left our system, fewer reached a person, and repeat offenders stayed on accounts we knew. When we got a call wrong, we said so, and people told us they appreciated it.

I owned the logic, the timeouts, the messaging and the cost. The vendor and engineering owned the machinery.

Mine

• Leading 16 moderators: eight on staff and eight community volunteers.
• The gray-area logic: the rules that turn a vendor score and our own signals into an action.
• The timeout model and its ladder.
• How we told people about a takedown, and the conversation after it.
• Cutting what we sent to the vendor: the domain block, the spam match, and deciding what needed a score.

Not mine

• The vendor's models and scoring.
• Engineering's build of the moderation pipeline.