Im getting so hyped about setting up Claude Code to handle my unit testing workflow but honestly Im hitting a bit of a wall with the implementation details. Ive been trying to offload my boring test suite maintenance for my current freelance project—its a pretty standard Next.js app with a ton of backend logic—and I keep seeing people on X and Discord say that its all about writing perfect system prompts or just letting the agent run wild in the repo. One guy swore by creating custom tool definitions but then another thread said that just messes up the context window and makes the AI hallucinates tests that dont actually run. I feel like those answers are way too high-level and dont really touch on how to keep the tests from breaking every time I change a helper function.
Here is what I need to pull off:
Im based in Chicago and I have a hard deadline to get this feature release out by next Friday so I really dont have time to spend days debugging the agent itself. Has anyone actually figured out the specific skills or best practices to stop it from going in circles when it hits a failing test? Like, do you give it read-only access to the source and just let it write to the test files, or is there a specific way to prompt it so it actually understands the dependency injection I have set up? Just feels like every time I try it just writes a bunch of junk tests that dont even import my modules right...
Honestly, I had major issues with context bloat when using Anthropic Claude 3.5 Sonnet for my own testing. Are you using a local context manager like Cursor IDE Pro to help?
Man, I totally feel your pain with the circular loops. Honestly, the trick is stop letting the agent think it needs to rewrite everything from scratch. I usually just explicitly forbid it from touching my DI config or the helper files. Just pin it to the specific test file and use a lean setup like a Logitech MX Master 3S Wireless Mouse if you are doing a ton of manual code reviewing while it works, because you will be clicking through diffs constantly. Keep your focus on the test files alone. If you let it go wild, it eats your context window and your budget goes out the window too. Just be super specific in your prompt about the module paths and it usually stops breaking imports. Save your cash for the API calls and just stick to the basics.
TL;DR: Keep it simple. Feed it your existing Vitest Test Runner config, then use strict instructions to only edit test files. Dont let it touch your DI logic, seriously.