Your AI Agents Don't Need the Whole Repo: Fixing the Full Clone Problem on GitHub and Bitbucket (2026)

Git was built in 2005 for people working on the Linux kernel. Its big idea was offline-first: every developer gets the entire history on their own machine, so you can commit, branch, blame, and search without a server.

For a human, that's great. You clone once, then work in that copy for months. The download cost gets spread across a long time.

Agents flip that completely:

That last row is the key. Git assumes you might be offline, so it gives you everything up front. Agents are never offline. They'd be totally happy asking a server for files as they need them. Git just wasn't built to work that way by default.

Anthropic's engineering post on this is worth reading in full, but here's the short version:

• The breaking point: merges went from about 2,500 a day in May 2026 to more than 30,000 by September. Its automated jobs were checking out the repo roughly 25 million times a day.
• The design: Scion assumes clients are always online. The server handles history, blame, merges, and rebases. Clients keep only a small local cache and fetch the files they actually need.
• API access: a job that just needs to ask "who last changed this file?" can query an API instead of cloning anything.
• The results: about 200 times less CPU and memory than its old Git mirror setup, checkouts in seconds instead of minutes, and a quarter of its CI capacity freed up.

The catch for us: Scion isn't public, and Anthropic says it isn't meant as a general Git replacement. The company shared the design hoping the big Git platforms pick up the ideas.

So until they do, we work with what Git gives us. Which, honestly, is more than most people realize.

If your repo is small, a full clone is fine and you can skip this post. Once it's big, every agent pays:

1. Startup time. An agent can't start working until the clone finishes. On a large repo, that's minutes of waiting before the first useful thing happens, and you pay it every single time.

2. Disk. Cloud agent sessions usually have a fixed disk allowance. A big repo's history can eat a real chunk of it before the agent installs a single dependency.

3. Rate limits and bandwidth. Ten agents cloning at once looks a lot like ten very enthusiastic people hammering your Git host. Hosts throttle that.

4. Parallel agents fight over one copy. If you try to save clones by sharing one, two agents editing the same folder at once is a fast way to get chaos.

5. Security. This one gets overlooked. An agent with a full clone can read everything: every folder, every old commit, and anything someone accidentally committed and "deleted" three years ago. If an agent only needs one part of the repo, giving it the whole thing breaks the least-access rule we'd apply to any human.

All of these are standard Git and work the same on GitHub, Bitbucket, or anywhere else.

1. Shallow clone: skip the history

git clone --depth 1 <repo-url>

You get the latest version of every file and only the most recent commit. It's the fastest, simplest option.

The tradeoff: no history. git log and git blame are mostly useless, and some hosts reject pushes from a shallow clone. Great for "build and test this" jobs, rough for agents that need to understand why code looks the way it does.

2. Blobless partial clone: history now, file contents later

git clone --filter=blob:none <repo-url>

You get the full commit history and folder structure, but Git only downloads the actual file contents (the "blobs") for what you check out. Anything else gets fetched the first time something asks for it.

This is my favorite default for agents. The agent can still run git log, look at history, and switch branches, but you're not downloading every version of every file ever written.

There's also a treeless version (--filter=tree:0) that skips even more. It's best for throwaway CI jobs, and slow if the agent starts digging through history.

3. Sparse checkout: only the folders that matter

git clone --filter=blob:none --sparse <repo-url>
cd <repo>
git sparse-checkout set apps/web packages/ui

Now the working folder only contains the paths you listed. Pair it with a blobless clone and you're downloading close to only what the agent will actually use.

This is the closest thing to Scion's "fetch what you need" idea that you can use today. It's also a security win: if the agent's working copy only has apps/web, that's mostly what it sees.

4. Worktrees: one clone, many agents

git worktree add ../agent-task-a -b agent/task-a
git worktree add ../agent-task-b -b agent/task-b

A worktree is an extra working folder attached to the same clone. Each agent gets its own folder and its own branch, but they all share one copy of the history on disk.

Claude Code has this built in. claude --worktree starts a session in its own worktree, and you can set isolation: worktree on a custom subagent so parallel subagents don't trip over each other's edits. Worktrees with no changes get cleaned up automatically.

The catch: worktrees help when your agents run on the same machine. They don't help when every agent gets its own fresh container.

5. A shared local mirror: clone once, copy cheaply

# once, and on a schedule to keep it fresh
git clone --mirror <repo-url> /cache/repo.git
 
# per agent
git clone --reference /cache/repo.git --dissociate <repo-url> work

Keep one up-to-date mirror on the machine (or a shared volume), and have each agent clone using it as a reference. Git copies objects from the local mirror instead of downloading them again, and only fetches what's new.

--dissociate makes each agent's clone independent of the mirror afterward, so cleaning up the cache doesn't break anyone. This is a tiny, homemade version of the mirror setup Anthropic ran before Scion.

6. No clone at all

Plenty of agent tasks don't need a working copy. "Summarize what changed in this PR," "find where this function is defined," "read the config file" can all be answered from the host's API.

GitHub and Bitbucket both let you read file contents, diffs, and commit history over their APIs, and an MCP server can give an agent those as tools. Zero clone time, zero disk.

The catch: API rate limits, and it falls apart the moment the agent needs to run tests or build something.

Most of the pain shows up in CI, because that's where agents and automated jobs clone the most.

GitHub makes this easy. The checkout action already does a shallow clone by default and has inputs for sparse checkout and partial clone filters.

Bitbucket is a little more manual. Pipelines does a 50-commit shallow clone by default, and the clone options cover depth, LFS, and turning the clone off entirely. For a blobless or sparse clone, the usual trick is to turn off the automatic clone with enabled: false and run your own git clone commands in the step's script.

One more Bitbucket reality check: a lot of agent tooling is built GitHub-first. (Claude's managed Code Review, for example, is GitHub only, as I covered in my Code Review post.) If you're on Bitbucket, expect to wire more of this up yourself.

Every one of these fixes has a sharp edge:

• Shallow clones lose tags. Tags older than your depth won't be there. If a job reads version tags, give it more depth or full history.
• Shallow clones can't always push. Some hosts reject pushes from shallow clones. If an agent needs to push, test it before relying on it.
• Blobless clones go back to the server. Running git blame or git log -p on old files triggers downloads. That needs network access and working credentials, and it can be slow if an agent does a lot of it.
• Sparse checkout hides callers. If the agent only sees apps/web, it can't see the code in packages/api that calls what it's changing. Tell it what it can't see, in your CLAUDE.md or task prompt, so it doesn't assume nothing else exists.
• Worktrees share one Git database. Two worktrees can't check out the same branch at once, and deleting the main clone breaks every worktree attached to it.
• Large files are their own problem. If your repo uses Git LFS, decide whether agents need those files at all. Skipping LFS downloads is often the biggest single speedup.

Here's how I'd match the checkout to the job:

My rule of thumb: start every agent with the least repo that could work, and only widen it when the agent proves it needs more. That's the same least-access idea we use for permissions, applied to code.

Anthropic's post reads a lot like a wishlist for the platforms the rest of us use. Here's mine:

• History and blame over the API, fast enough to replace a clone. Most agent questions about history shouldn't need a local copy.
• Agent-friendly rate limits. Limits built for humans clicking around don't fit 20 agents working in parallel.
• Access scoped to folders, not just whole repos. Let me give an agent a token that can only read and write apps/web.
• First-class partial and sparse clones everywhere, including Bitbucket Pipelines, without hand-written clone scripts.
• Merge queues built for agent volume. Anthropic found a classic one-at-a-time merge queue didn't hold up at agent scale.

Until then, the Git features in this post are the workaround. They're not glamorous, but they're free and they work today.