Why I'm Not Betting Everything on Transformer Hype

Every time I integrate AI into a project, someone asks: "But did you use the latest LLM?"

Honest answer? Usually not.

I use what solves the problem. And that's rarely the fanciest thing.

At Seven Suite, I needed to generate ERPNext workflows from natural language descriptions. Everyone said: use GPT-4, Claude, throw transformers at it. So I did. Burned tokens like water. The results were mediocre because the model was optimizing for general language, not my specific domain.

Then I realized something: my domain is geometry. Workflows have structure. Sequential steps. Branches. Conditions. It's not "free-form language generation." It's constrained generation in a learned space.

I built a smaller, simpler model that learned the geometry of valid workflows. It doesn't hallucinate invalid states. Doesn't generate nonsense. Costs a fraction to run.

This applies everywhere I work. Telegram bots don't need bleeding-edge language models—they need reliability. Legacy system migrations don't need transformers—they need semantic understanding of old code patterns. That's different.

The hype cycle pushes everyone toward the same solution. But the best engineers I know do the opposite: they ask what's actually required, then use the minimal complexity to achieve it.

Transformers are incredible. They should dominate NLP. I use them when they're the right tool.

But they're not the right tool for everything, despite what your VC-backed startup tells you.

The questions worth asking:

• Can I solve this with simpler geometry?
• Do I need general intelligence or domain-specific understanding?
• What's the actual constraint—latency, cost, accuracy, or just vibes?

Often the answer is: step outside the transformer paradigm. Build something smaller. Something that fits your problem instead of forcing your problem into an LLM.

That's how you ship things that actually work.