AI advice is everywhere. How do you tell which of it is worth listening to?

A while back, an ad popped up on my phone.

The ad said there was this AI workbench that could run 20 agents — agents that would make your calls, run your errands, handle all sorts of things for you.

I couldn't sit still. Wasn't this the minion army I'd been dreaming of?

I jumped on it immediately and built a 15-agent system. The ad made it sound so easy — I figured an afternoon, tops.

Three weeks later, it still wasn't working.

Out of options, I went to a friend of mine, Marc. He's the real deal when it comes to agents — he ships model after model, and each one lands with stunning results.

He walked me through his thinking and his playbook. I fed it all, word for word, to my AI assistant and told it to follow along.

You know what it came back with?

"This won't work. Because he's using a different agent architecture."

Impressive. My AI had just stopped me from copying someone else's homework.

But after the laugh, I broke out in a cold sweat.

Think about it: if even someone like me, who lives in AI every day, got tripped up for three weeks by advice that "looks great on the surface" — what about everyone else?

Right now, AI advice is everywhere. A trick today, a secret playbook tomorrow; "must-read" here, "game-changer" there. The more you read, the less you know who to trust.

So today, let's talk about something far more important than "AI tips":

How do you judge whether a piece of AI advice is actually worth listening to?

An old story first

Why does this happen? Let me tell you a bit of history.

More than twenty years ago, email marketing went through the exact same stage.

Back then, there were no ready-made automation flows, barely any data, and certainly no "best practices." Marketers were feeling their way in the dark: send one email, check the numbers; send another, check the numbers again.

Then people started getting together to compare notes. Meetings, email lists, forums, the occasional dinner in person.

And out of those conversations, a group of "teachers" emerged. They wrote newsletters, blogs, ran big conferences, gave talks — and their voices got louder and louder.

Why did they rise above the crowd?

Because they taught real stuff. Which traps not to step in, which tests produced which numbers, which plays could scale and be replicated — they laid it all out, dissected and explained.

Slowly, a set of so-called "best practices" grew out of those exchanges.

Later, as everyone's experience deepened, they naturally developed a trained eye: who's a real expert and who's a human parrot — you could tell at a glance.

Sound like a familiar script?

AI is replaying it beat for beat.

AI becoming a practical tool in ordinary people's hands — that only happened in the few years since ChatGPT launched. And the moment the tools spread, "do this, not that" advice flooded in.

But how much of it comes from real experts? How much from human parrots?

Nobody knows. The AI world right now is in email marketing's old "flying blind" phase.

So what do you do?

I'll give you three questions. Next time some AI genius trick crosses your feed, run it through all three gates first.

Question one: how far is their claim from consensus?

What does "far from consensus" mean?

It means what they're peddling is wildly different from the standard stuff you've heard — so different that you pause: wait… is this legit?

Let me be clear: being far out is not a sin.

Real breakthroughs often hide exactly in the "far from consensus" zone. Everyone says go east; they insist on west — and west actually turns out to be the way through. That's insight.

The key question is: did they tell you how other people can replicate it?

If someone has a wild take, doesn't explain why it's wild, and can't show you how to reproduce the same result — be careful.

Why? Because that kind of advice is the favorite food of an old monster:

The silver-bullet fantasy.

What's a silver bullet? The fantasy of a tool that wins the fight in one move, works instantly, and lets you coast forever.

Marketers have chased silver bullets for decades. They've poured in huge amounts of time and money — and only a few have paid off in the long run.

How do you dodge the trap? Watch how much the community is discussing it.

If lots of people in the field are using a play, testing it, and their conclusions line up, it will most likely hold up under scrutiny.

Remember: consensus doesn't mean going with the crowd. It means more than one person has already verified it for you.

Question two: did they show you the process?

Back to my friend Marc.

I've seen the results of those agents he builds. My god, they're incredible.

But after seeing them, I deflated: how did he actually build them?

Where did he start? How did he break the problem down? What did he work through along the way? How many times did things crash? No idea.

If I can't see the process, I can't learn anything. However stunning the result, it's his result — not my skill.

This is the classic tell of so much AI content: showing off results, hiding the process.

It's like a dish where you only get the photo of the finished plate — no recipe. However talented you are, all you can do is stare.

So next time you see someone showing off a mind-blowing project, hold off on the worship. Check whether they've also laid out "how it was done."

The traps they stepped in, the dead ends, the detours — those unglamorous parts are precisely the most valuable parts.

Why do I dare say that? Because I'm paying down that very debt myself.

Those three weeks, I went back and forth between GitHub, Trigger.dev, Supabase, and Vercel — errors, retries, tearing it all down and starting over. When I finally get my minion army running end to end, I promise I'll write the full story: how good the system is — I'll tell you. How many traps I hit — I'll tell you even more. Nothing held back.

Oh, and there's one more bill that has to be tallied.

AI burns money. Every time you fire up a model, you're spending real cash and compute quota. Your money, your tokens (the usage units you pay for when calling a model), and your time should go to advice that can survive the first two questions.

Question three: what problem are you actually trying to solve?

This one is for the moment right before you commit.

You see a shiny new tool, a new workflow, and your first reaction is "how do I set it up?"

Stop. Don't ask how yet — ask why.

Strategy first, tactics second. Get clear on "why" before you look at "how."

Before you commit, fire off these questions. Ask yourself, or throw them straight at your AI assistant:

Does this matter to my business? Is it realistic? Can it actually work? Is it solving a real problem, or a problem that only looks new? Does the payoff justify what I'm putting in?

Here's a textbook case from the email-marketing world: abandoned-cart reminder emails.

A customer leaves things in their cart and walks away; the system automatically emails to win them back. Technically, that's been possible forever. But many teams' data took 24 hours or longer to land. By the time the email went out, the customer had gone stone cold.

In 2010, Charles Nicholls, founder of SeeWhy, put out a set of data: reminder emails perform best when sent roughly one hour after the transaction.

Once that data was out, marketers had real leverage — they could slam the table in front of their data teams: get me faster data.

Note: "one hour" is not gospel. It doesn't mean everyone's follow-up emails should go out after one hour. Its value was this: it replaced "random guessing" with "an evidence-based starting point for testing."

The "best practices" floating around the AI world are exactly the same.

Someone says a certain prompt or a certain workflow is the best there is? Don't start worshipping yet. Whether it works depends on your business, your customers, your team.

Someone else's answer is only your starting-line reference. Your answer, you have to test out yourself.

What does real thought leadership look like?

Finally, about these "teachers."

People calling themselves "AI experts" today vastly outnumber the real ones. The ledger between fame and skill rarely balances.

Beyond the three questions above, I usually look at two more things.

One is the foundation: what's their background, what have they built, what have they published in the open.

The other, I think, matters even more: how much have they given back to the industry?

Is the knowledge locked behind a paywall — or poured out for free on podcasts, on stage, in the way they bring newcomers along?

The real ones aren't just smart. They're willing to hand over what they know — and, while they're at it, lift someone else's boat too.

At this point you might fire back: isn't this essay of yours also advice?

Yes. So please, turn those three questions on this piece and audit me. It can take it.

As for my minion army — the moment it's built, you'll be the first to read about it. The process, the traps, the tokens burned: nothing held back.

And as you wade through this wall-to-wall flood of advice, save your ears for nothing but the truth.