skip to content
Eric Rahm

Optimizing tools for agents

I’ve been spending a lot of my professional time recently contemplating how to make our developer ecosystem more amenable to agentic usage. Me being me I started out by figuring out how to measure things:

  • Command failure analytics with agent tagging
  • An eval framework focused on overall performance rather than just task completion
  • Tooling to A/B compare changes to tools, skills, docs, etc

My overall goal has been: things should just work and if they don’t work we should change them. This has been pretty effective but somewhat adhoc. I’m taking some time now to gather my thoughts on some basic guidelines outlining what’s worked for me and what hasn’t, how to measure effectiveness, and how to design for humans, agents, and machines. Hopefully more to come!