Finally bit the bullet and started that LLM code token toolkit I’ve been thinking about. It’s called claude-code-token-toolkit (yeah, a bit of a mouthful), and it’s my personal fight against LLM context window limits when I’m working with code. I’m constantly bumping up against token limits, and I want a tool to help me understand the token cost of my codebases, break them down, and maybe even optimize them for various LLMs.

  • First commit felt huge. My “Initial commit: catalog + analyzer” was just a couple of empty files, but getting something down felt like a win. The “catalog” is for organizing code, the “analyzer” for token counting. Super basic, but it’s a start.
  • Naming things is the worst. I went through five names before landing on claude-code-token-toolkit. It’s descriptive, but maybe a little clunky.
  • README work paid off. I spent a surprising amount of time sharpening the README, adding a “vision” and “honest current-state.” This wasn’t just documentation, it helped clarify my own thoughts and set expectations. I also got a basic TODO list going, which is always satisfying.
  • Blank page syndrome is real. The hardest part was just getting that first commit out. I had the idea for ages, but translating it into actual code, even placeholders, felt like pulling teeth. Defining the initial scope was tricky too, but I decided to keep it lean, focusing purely on token counting first.

It felt amazing to finally start. I’ve been talking about this idea for a while, and seeing it take shape, even in this nascent form, is really motivating. It’s a good reminder that the biggest hurdle is often just getting off the starting line. Documenting my intentions early, even for a personal project, helped solidify what I’m trying to achieve and gave me a clearer roadmap. It’s satisfying to build a tool that directly addresses a problem I’m facing.

Next: Implement the actual tokenization logic.