AI and Accuracy

While I’m committed to writing all posts myself, grammar, spelling, and accuracy review notwithstanding, I leaned heavily on AI to build most of this site and its utilities. I’ve built many things by hand in the past, but time, energy, and life always get in the way. AI helps free up some capacity by letting me direct a wildly capable intern with no moral compass or understanding of team norms and dynamics.

As frontier AI systems improve at carrying out tasks, capability does not eliminate hallucination, deception, or shortcut-seeking behavior. During a Black Hat USA 2026 presentation about an internal evaluation incident, OpenAI’s Eric Wallace put it bluntly: “Frontier models really like to cheat.” Their explanations can also be stated with extreme confidence and accompanied by “supporting evidence” that does not actually support the claim.

Take the link shortener as an example. While working with AI, our planning led to a statement about the service being “minimal by design.” The AI knew the building blocks driving the service, its configuration, and how it operates, including Cloudflare Workers, KV, and D1, but it still greatly overstated the privacy available. Is application data collection minimized? Yes. Does the deployed stack provide anonymity or absolute privacy? No. Cloudflare still processes request data, submitted destinations are checked through Google Web Risk, aggregate platform metrics remain available, and an operator can deliberately start a live troubleshooting session that exposes request URLs during that session. Do we want zero observability? No. I like to have the ability to troubleshoot, and we need enough evidence to investigate and respond to abuse.

This is a simple example, but I’ve seen much worse. A citizen developer may not have caught the overstatement and published inaccurate claims about the tool they built.

Understanding fundamentals, applying critical thinking, and maintaining a healthy dose of skepticism are critical to thriving in our modern information environment.

Musings

I’ll write a full post about some of the things I do when setting up my agent harnesses, but one of the first expectations I set is to distinguish verified claims from inference, state confidence when it matters, and provide citations. This can help immensely when assessing the likelihood of a hallucination, but it is fallible. Markdown is not governance, and a model can fabricate a label or citation as easily as any other text.

I’m still working on my rule requiring agents to flag any knowingly introduced false statement with [teehee] or a giggling emoji, but for some reason the models get snarky and rarely use it. /shrug