Do not use AI when the answer is always the same for the same input. If a formula, a database query, a search index or a business rule can give the right answer every time, use that. It is cheaper, faster, exact and easy to test. Save language models for the parts of a problem that need judgement: messy text, ambiguous questions, many possible answers. The rest of this post is how I make that call, and why it matters more as AI gets cheaper, not less.
AI feels almost free. It is not.
For $20 a month, one person can write code, analyse documents, research a market and summarise meetings from one chat window. A few years ago that would have needed a small team.
It is easy to read today’s price as the real cost. It is not. Every request to a language model runs on servers, GPUs, memory, power and cooling. The companies behind these tools are spending at a scale few industries have seen: OpenAI has reportedly planned hundreds of billions of dollars of compute spending through 2030.
That does not mean every subscription is sold at a loss. Light users and heavy users have very different economics. But somebody pays for the infrastructure in the end: subscribers, API customers, enterprises or advertisers. Advertising does not make computing free. It moves the bill to someone else, the same way “free” search and social media are paid for by businesses that add the ad spend to their prices.
For a product team, that means one thing. Every model call in your product is a running cost. It is not a one-off build cost.
The 2 + 2 test
Open a calculator and type 2 + 2. Your phone reads two numbers, adds them and shows 4. No network request, no model, no need to understand language. The cost is effectively zero.
Now ask a language model “What is 2 + 2?”. The answer is still 4, but the path is very different:
- Your text is split into tokens.
- The tokens travel to a data centre.
- A large neural network runs a huge number of matrix calculations to predict the most likely reply, one token at a time.
- The reply is turned back into text and sent to your device.
Same answer. One tool was built for arithmetic. The other can also write software, review contracts and translate languages, and you just used all of that to add two numbers. It is like hiring an engineering team to press the button on a calculator.
One call like that costs a tiny fraction of a cent on a small model. Nobody cares about one. But multiply it by every user, every session and every screen where someone decided “let the AI handle it”. Then add long prompts, attached documents, reasoning steps, images and voice. Then add agents, where one user request can become ten model calls as the agent plans, searches, calls tools, checks its work and tries again. That is when the architecture decision shows up in the monthly bill.
A calculator I built with AI does not call AI
I live this every week on my own site. The Meeting Cost Calculator and the RICE Priority Scorer were built with AI coding tools. AI made building them fast. But neither one sends your inputs to a model. Each is a formula:
Cost = Attendees × Hourly rate × Hours and
RICE = (R × I × C) ÷ E.
A formula gives the exact answer, instantly, for free, and gives the same answer tomorrow.
Could I have added a chat box that “calculates your meeting cost with AI”? Easily. It would have been slower, cost me money on every visit, and once in a while given a wrong number. The AI label would have made the tool worse.
Where AI does help is the fuzzy part: turning a vague idea into a first list of features to score, or explaining what a score means for your situation. That is why my AI prompts for product work sit next to the calculators, not inside them. AI for the judgement, the formula for the maths.
When should you use an LLM, and when should you not?
Before a feature calls a model, I ask these six questions. If most answers point to “no”, a simpler tool will do the job better.
- Is the answer always the same for the same input? Totals, conversions, eligibility checks, status lookups. Use code.
- Can you write the rule down? If a person can write it as a clear if/then, it belongs in code, where you can test it and audit it.
- What happens when it is wrong? If a wrong answer means lost money, a security gap, legal exposure or damaged data, a model should not be the only decision maker.
- Is the input messy human language? Support tickets, interview notes, free text, documents with no fixed format. This is where a model earns its cost.
- Is the value of the answer bigger than its cost, at full scale? Work it out per user per month, not per call. A solution that costs $5 to solve a problem worth 50 cents is not innovation.
- Would a smaller model do? Classifying, extracting and tagging often work well on small, cheap models. Keep the frontier model for the hard reasoning.
The toolbox is bigger than “AI or no AI”. Depending on the problem, the right answer might be a database query, a search index, a formula, a business rule, a small classifier, a specialised model, an automation workflow, a frontier LLM, or a person making the call. Good products usually combine several of them.
But AI keeps getting cheaper, doesn’t it?
It does, and fast. Epoch AI estimated in September 2026 that the cost of reaching a fixed level of AI performance has fallen about 47 percent per quarter since 2023. Small models now do work that needed the biggest models two years ago.
So I am not predicting that every AI request will get more expensive. The opposite is more likely. But note what that figure measures: the price of a fixed level of ability. Your bill depends on how much you use. When a task drops from $10 to $1, teams do not save $9. They run it a hundred times, add an agent loop, and attach bigger documents.
Intelligence is getting cheaper, and our use of it is growing faster. Cheaper models do not remove the need to decide where they belong. They make bad decisions cheaper to start and more expensive to scale.
Choosing the right tool is a product decision
Engineers can build any of these options. Choosing between them is a product call, because it depends on things outside the code: what a wrong answer costs the customer, what the feature is worth, what the margin can carry, and what the team can support. I wrote about the same trade-off for ownership in build vs buy in the AI era: building got cheap, owning did not. Running a model is part of owning it.
This is also where many teams I talk to get stuck. “Add AI” arrives as a roadmap item from the board or a competitor’s launch, before anyone has asked which part of the problem needs it. Speed is not the problem. AI made shipping fast and deciding slower, and choosing where intelligence goes is one of those slower decisions.
What skills matter for product people now?
Knowing the newest AI tool is useful, but it will not last as an advantage. Interfaces change, models change, and today’s clever prompt becomes tomorrow’s button. The skills that last are the ones that decide where the tools go:
- Breaking problems down. Which part needs human judgement, which part needs AI, which part needs plain software, and which part should not exist at all.
- Domain knowledge. Someone who understands payments, healthcare or logistics can put AI where it matters. General AI knowledge is easy to copy.
- Evaluation. Generating an answer is easy now. Knowing whether it is correct, safe and commercially sensible is the hard part.
- Unit economics. Knowing what a feature costs per user, and whether the value covers it.
- Knowing when not to use AI. The most underrated one on this list.
Are you using the right amount of intelligence?
Electricity changed every industry, but nobody solves problems by using as much electricity as possible. AI is heading the same way: a layer under almost every product. Soon “are you using AI?” will be a meaningless question, because everyone will be.
The better question is: are you using the right amount of intelligence for this problem? For 2 + 2, use the calculator. To understand why customers are leaving, read thousands of support conversations or work through an ambiguous strategy question, a model may be worth far more than it costs.
The future does not belong to the teams that use the most AI. It belongs to the teams that understand the problem well enough to choose the right tool.
Tahir Shahzad
Tahir Shahzad is a Product Manager, Product Owner, and technology consultant with over a decade of experience helping startups and organizations build products people actually use. If you are working on a product problem worth solving, reach him at tahirshahzad.com/contact/
