"Token usage" is the least thing that you should worry about with AI

It's people who are sure and wrong, armed with an LLM

(This article was not written by AI, I used Grammarly for the most egregious grammar mistakes. Let me know in the comments whether it’s noticeable, good or bad)

I can’t exactly remember the name of my driving instructor, but I do remember a couple of facts about life and cars that he taught me in no particular order when I was 18 years old:

1. “Don’t apply karate” to the stick when shifting gears; use it softly.

2. Never swerve to save a bird that’s flying across the road in front of you, they cannot be killed by cars. → I disproved that assumption 20 years later, but I guess the advice was still good to protect my own life.

3. 90% of all people who drive cars are convinced that they drive better than the average driver.

4. It takes 7 years to properly learn how to drive a car. The most accidents per driven kilometer happen in years 2 and 3 when people start to be overconfident, not when they’re new and inexperienced.

3 and 4 are illustrating that it’s nothing new that humans tend to be overconfident in their abilities if they just do something for long enough, regardless of whether they have any right to or not.

Pair that with AI, and you have the perfect disaster.

“Build me a report for that. Make no mistakes. You’re an expert in everything”

If you believe that it matters that you (and your team) base your decisions on facts and not fiction, then I have some unsettling news for you:

An insane, hard-to-measure, and drastically underestimated cost of AI tooling, mostly LLM’s like Claude or ChatGPT, is the false confidence that people are gaining in all their business decisions and what it really costs you somewhere down the line.

And this goes across all levels and functions.

Some examples from the last year that I’ve experienced directly (with permission, anonymized):

1. A Series A CEO skipping research about Ideal customer profiles, running it all through Claude, and then putting that into the strategy and aligning the entire company against what they found out there. A spot check revealed that the data underneath was shallow at best and directionally completely false. A conversation with 50k tokens that potentially costs them the business 2 years down the line. Didn’t listen to me in the end.

2. A CEO of a 200-people B2B SaaS company drafting a 20-page strategy document with ChatGPT based on a budget sheet with zero additional context attached to it, not reviewed by anyone else. They were asking me how to force everyone in the company to attach that document as context to everything they do with AI. That strategy document was not only extremely bad and unachievable; it was also created by someone who was notoriously uninformed about the reality of the business overall.

3. A PM overestimating the impact of a product redesign by 85% in a pretty one-pager, since they didn’t understand how many users are actually using the product before they started building it with their team of 5 people for 2 months. (They used the total number of users of the business, instead of their feature usage)
   They self-served all their data through Amplitude without checking or understanding it, and nobody bothered to cross-check either. The only reason why I know that they used Amplitude's AI prompting interface is that I asked. It was completely devoid from the 1-pager. Everything was stated as a fact without any sourcing.

Ain’t nobody got time for (checking) that

Every good business and partnership I’ve been part of ran on the same principle: Trust is essential because you can’t and shouldn’t check everything from other people.

It’s commercially not viable to micromanage everything; you cannot know everything yourself, you don’t have infinite time, and it also demotivates people in your team to the point of them quietly quitting.

We’ve been applying this principle way before AI, and it’s a key skill to know who and when to trust and when it’s time to separate ways.

So, how do you know who to trust in business? It used to be:

1. Output quality: Some form of business output like a report that looks well put together since it costs time to do that → A 2-minute task for an LLM nowadays.

2. Impact: Measurable, continuous $ impact → In product, leadership & engineering, it might take months or years to see it, and it’s hard to tie to individuals.

3. Confidence and storytelling: They sound confident in what they’re saying in an area you cannot evaluate yourself and can answer surface questions confidently. It wasn’t possible in the past to some degree without knowing underlying data → LLM’s made this indicator completely useless. Storytelling is still an important quality to have to align people, but it doesn’t mean that you’re saying the right things.

4. Background: They have an impressive background in their CV that makes it hard to believe that they don’t know what they talk about → We’re all new to the new AI world. I’ve seen ex-colleagues label themselves as AI experts and coaches in 2026, and I’m pretty confident that they struggle to operate a toaster without burning down their house because they never read the manual and are, in general, just sloppy people. Most of what made them successful doesn’t translate to the new AI world either.

5. Trust by proxy: They are recommended by people whom you trust to be trustworthy → This one has become incredibly important, but is dependent on the other factors to evaluate them as well.

6. High agency while doing the work themselves: People who work hard with high agency are people who naturally become good at what they do. → This is still somewhat true, and I gravitate to those people still. However, you can only become good at something (including operating LLM’s) if there is a feedback loop that tells you whether you’re doing it wrong or you can spot that the output is not good / correct.

   1. Feedback loop: Assuming you learn how to play the guitar, you can tell by the sound that you make whether it sounds good or not. Assuming you do hear what sounds bad, you know that you still have to improve. AI is great at removing this feedback loop, and you start to believe that you’re a genius and keep producing the same slop over an over.

The stupid thing about it is, we still don’t have enough time to check everything, and I’m not sure either what the perfect method is in separating fact from fiction, but I know that it has a lot to do with creating an environment where trust can be established again.

And for that I’m convinced we need to deal with what “being confident” means.

AI is great at producing confidently incorrect people; your job is to prevent that.

I feel like a big part of my job as a leader in a business is to reduce the chance of my team (and myself) being confidently incorrect.

The emphasis here is on the word “confidently,” not on being incorrect.

If you want to have intellectual discourse, I recommend https://www.reddit.com/r/confidentlyincorrect/

“Making a fool of yourself” is a phrase that is used to describe someone who does something foolish and then doubles down on it. Phrases like these actually serve a function, because the social stigma of being a “fool” is something we try to avoid since we care a lot about what others think about us, me included.

It’s like a feedback loop that works in certain situations, and I recommend that you introduce the same mechanism with yourself or your teams:

“Responsibility for your AI output”

People have to be responsible for what they release as information, whether it’s handwritten or by AI:

1. If any output is in relation to an important topic, I expect people to write at the end of it “I have verified the above information myself” or be transparent where they are unsure.

   1. If you get spot-checked on data that is important and it turns out to be wrong because of sloppy work that wasn’t checked carefully, there have to be consequences. It can start as simple as calling it out and then learning from it. If it happens multiple times, it becomes a risk and a manager problem.

2. Unimportant topics don’t deserve long documents but short, high-quality paragraphs.

3. All information has to be sourced: Where did it get pulled from, and how (was it you or an AI?)?

4. People need to get comfortable with showing “how” they work, for instance through “pair programming” for knowledge-related tasks: “Let’s figure out something together while I share my screen” is heavily underrated. This is uncomfortable for a lot of people, but that’s the easiest way to uplevel people. Don’t expect miracles, though, if you pair two amateurs with each other.

5. Assume that someone does not know how to use AI (and judge whether information is trustworthy) if you don’t know yet. The default should be that we don’t know, until they prove to you they do, they don’t.

6. If you want to teach people one thing about AI, then it’s this: If you don’t tell an AI what a data point is, it will assume it to be a true fact. If I say “our conversion rate is 15%” then that’s what the model will assume to be true as a standard. It does not, and cannot, know whether a number is unverified, and it won’t check unless you ask it to actively. An LLM is trained to trust you, the user, as well.

If all fails, you can also take away access to sources of typical AI enabled problems:

Avoiding false confidence by barring access to data

Taking away access is a spicy one, because I learned that the best businesses are transparent about what they do. I still believe that, but there are types of information that are too dangerous together with AI because of the adverse effect they have in an organization, with little gain if they are handled correctly:

No self-serve access to critical experiment systems

I consider whether I should take away access from certain people to experimentation data like Amplitude, Growthbook, etc..

Most people in an organization are data illiterate, but with LLMs they become a costly problem when they have access to experimentation data systems:

a) Peeking: They start to look at results before they are statistically significant and state them as facts everywhere in their documents, making it impossible to check them retroactively. (There is a reason why data science is a well-paid profession, especially now)

b) Interpreting results wrong: The cost of creating experiments has dramatically gone down. The one-pagers that come with them make it sound like someone worked on them when it was potentially AI-slop, and the thing you’re building and testing also costs very little to ship. You can teach data-illiterate people about statistical significance, but the danger of interpreting the results wrong is still incredibly big since an LLM often is missing incredibly important context. You’ve proven the metric moved, but not necessarily that the attached hypothesis is correct. The LLM will tell you though that youre hunch is totally correct because you are a super special one.

On this last point, data scientists often trip up as well. For important experiments, the team should review them together. Especially if the results are extreme in one direction or the other.

No access to product - backlog for Sales

Same problem, with a different reason: salespeople start to gain access to product teams' backlogs and use that info within LLMs. They will start to look through them and promise customers features that they don’t really understand too early.

They should not have access to that information; the solution here is to have another roadmap that is made for them: A sales-ready one that contains close-to-release features curated by a human.

That same human should also be responsible for serving as a funnel for sales-related feedback to the product backlog because Sales feedback is often either over- or under-leveraged:

1. Overleveraged: Everything sounds urgent and is built for individual clients, distracting engineers from what they should do otherwise

2. Under-leveraged: Everything from sales is shoved away in a Slack or Teams channel for it to rot away because no one wants to listen to them

I’ve not yet seen any LLM or tool solution that gives me confidence that they know how to categorize and evaluate this in an automated manner. Tools that try to gauge the importance of a feature by how many times it was requested don’t work because Sales is not a representation of your customers. It’s a representation of your acquisition engine and where they focus, not necessarily where the highest leverage for the business is.

The only thing that works, in my opinion, is a product manager that takes care of this feedback and educates back to sales whether something is likely to be handled (because it was evaluated) or not (not important, noise).

People who love confidence over facts don’t have a place anymore

I’m dead serious with this; we need to learn how to deal with AI for sure, but no matter how good we believe we can become with it, false confidence, laziness to cross-check, paired with an inability to learn and be humble, is a costly combination together with LLMs. Way, way more than how many tokens you burn or which model you use.

I’ve worked with context- and knowledge-related models for hundreds of hours already, and I still make the mistakes I called out above because, whether you believe it or not, I am human too.

I just hope that I’m humble enough to have my feedback loops established that whip me into shape. The one I use is a simple one:

If I can’t judge whether something I just produced with AI warrants my confidence and is correct, then I should probably ask people who can help me get that confidence before I put my name on it.

You should hold people around to the same standard.