Coding agent

A coding agent is an LLM tool-use loop specialised for repositories. Its typical toolset is file read, file edit (usually exact string replacement or a patch format, since rewriting whole files wastes tokens and invites accidental changes), glob and grep search, a shell for builds and tests, and git. Yang et al.'s SWE-agent (2024) showed that the design of this agent-computer interface matters as much as the model: purpose-built commands such as a windowed file viewer, concise search output and an edit command that rejects syntactically invalid changes outperformed giving the model a raw shell. Long tasks exceed the context window, so agents compact history into summaries, delegate exploration to subagents with separate contexts, and rely on repository instruction files for conventions.

Progress is tracked mainly with SWE-bench (Jimenez et al., 2023): 2,294 task instances built from real GitHub issues and pull requests across 12 Python repositories, where a patch counts as resolved if the tests that the original fix made pass now pass and existing tests still pass. OpenAI's SWE-bench Verified (August 2024) is a 500-task subset screened by human engineers to remove underspecified or unfair tasks, and it became the headline metric. Scores should be read with care: public repositories raise contamination concerns, scaffolding differs between submissions, and the benchmark covers only Python bug-fixing, which is why suites such as Terminal-Bench and multilingual variants have appeared.

Deployment comes in two shapes. Local agents run in the developer's terminal or IDE with that user's file system, credentials and network access, gated by permission prompts or modes; background agents run in ephemeral cloud VMs or containers, work on a branch and return a pull request. The second shape is easier to contain, because the sandbox, network policy and scoped tokens are set per job rather than inherited from a workstation.

Characteristic failure modes include test gaming (weakening or special-casing tests so they pass), sprawling edits beyond the task, calling APIs or packages that do not exist, and looping on a failure. The security concern is sharper than for assistants because the agent both reads untrusted content (issues, web pages, dependency code) and can act. Simon Willison's "lethal trifecta" names the dangerous combination: access to private data, exposure to untrusted content and a channel to communicate externally. Remove at least one leg: run the agent in a sandbox with egress restricted, give it short-lived least-privilege credentials, keep it off protected branches, and send every change through CI and human review.