Limits changed on me during a project and the scramble taught me the order to try things in. Cheapest first. 1. Measure before switching anything. Most of my usage was avoidable: re sending whole files where a section…
Narration went from a hobby line to my largest bill. The choice is less about price per character than which compromise you take. Hosted premium. Still audibly ahead for anything longer than a couple of minutes. Priced…
Broke down last month's usage properly for the first time. Output tokens are a small fraction of the cost. The overwhelming majority is input. That surprised me, because the things my product generates are short. I…
I keep changing prompts, tools and models, and my evidence for whether it helped is that recent runs felt better. That is obviously not evidence. A batch of observability products has appeared aimed at this: tracing…
Ollama Pro, $20/month, running from the VS Code extension. One agent turn. About three minutes wall clock. One question: compare two versions of the same Markdown spec (24k words and 18k words) and list the differences.…
Two subscriptions at the same price, two completely different units of account, and both moved in July. What I can state with a source: Cursor's included usage is split into a first party pool (Auto, Composer 2.5,…
Burned through the monthly allowance on the $20 plan in about nine days and I genuinely cannot tell you what did it. I am not looking for "use a cheaper model", I want to know how to attribute the spend while it is…
RAG assistant, prompts average about 9,200 input tokens. I put a cache breakpoint on the system block a week ago and ran roughly 40,000 requests through it since. is 0 on every single response. is populated every time.…
Models are shipping faster than I can form an opinion about them, and every switch decision I have made this year was a vibe that I reversed two weeks later. I want a protocol small enough that a solo dev will actually…
Retrieval is pgvector, top 40 candidates, then a cross encoder reranking down to 8, running on CPU in the same container. Answer quality went up noticeably people stopped complaining about the wrong section being cited.…
Judge prompt asks for a 1 to 5 rating on helpfulness and accuracy given the question, the retrieved context and the answer. As a sanity check I fed it answers with fabricated citations, answers truncated mid sentence,…
Document question answering over long PDFs. A typical document is 200 pages, roughly 120k tokens. About 400 questions a day across all users, and users tend to ask several questions about the same document in one…
Two engineers, about 200 daily active users, one LLM feature that matters. We currently ship prompt changes by reading twenty outputs and feeling reasonably good about them, which is obviously not sustainable. We need…
I have 12 test cases. Every prompt change I make moves the pass rate by 8 or 16 points, which even I can tell is just one or two cases flipping rather than a real signal. So how many do people actually run with? Is 500…