---
title: "Your AI Agents Don't Need the Whole Repo: Fixing the Full Clone Problem on GitHub and Bitbucket (2026)"
description: "Anthropic outgrew Git and built its own source control. The rest of us are still cloning entire repos for every agent. Here's why that hurts, and the Git features that fix most of it today."
date: "2026-10-17"
tags: [ai, ai-agents, git, github, bitbucket, claude-code, source-control, devops]
agent_task: "Look at how my agents and CI jobs currently clone my repo, then recommend the leanest checkout for each one (shallow, blobless, sparse, worktree, or no clone at all) and write the exact commands or pipeline config to switch."
skill_name: "lean-repo-checkouts-for-agents"
sources:
  - title: "Redesigning source control at Anthropic"
    url: "https://www.anthropic.com/engineering/redesigning-source-control"
  - title: "Git partial clone documentation"
    url: "https://git-scm.com/docs/partial-clone"
  - title: "git sparse-checkout documentation"
    url: "https://git-scm.com/docs/git-sparse-checkout"
  - title: "Git clone behavior (Bitbucket Cloud)"
    url: "https://support.atlassian.com/bitbucket-cloud/docs/git-clone-behavior"
  - title: "Run parallel sessions with worktrees (Claude Code docs)"
    url: "https://code.claude.com/docs/en/worktrees"
---
Here's a problem almost everyone running agents hits sooner or later: **every agent wants its own full copy of the repo.** Spin up five agents, get five clones. Each one waits on a download, eats disk, and pulls years of history it will never look at, just to change three files.

Anthropic hit the extreme version of this. Agents pushed its main repo from about 2,500 merges a day to over 30,000, and Git couldn't keep up. So it built its own source control system, Scion, from scratch. Cool for Anthropic. The rest of us are on GitHub or Bitbucket, and we're not building a new version control system this quarter.

The good news: Git already has most of the tools to stop cloning the world. Most of us just never needed them before agents showed up.

## Table of Contents
1. [Why Git Hands Everyone Everything](#why-git)
2. [What Anthropic Did About It](#scion)
3. [What Full Clones Cost You](#cost)
4. [Six Ways to Clone Less](#fixes)
5. [GitHub and Bitbucket Specifics](#platforms)
6. [Gotchas](#gotchas)
7. [Pick by Job](#pick-by-job)
8. [What I Want From GitHub and Bitbucket](#wishlist)
9. [TL;DR](#tldr)

---

## Why Git Hands Everyone Everything {#why-git}

Git was built in 2005 for people working on the Linux kernel. Its big idea was **offline-first**: every developer gets the entire history on their own machine, so you can commit, branch, blame, and search without a server.

For a human, that's great. You clone once, then work in that copy for months. The download cost gets spread across a long time.

Agents flip that completely:

| | A human developer | An AI agent |
|---|---|---|
| **How long the copy lives** | Months or years | Minutes to hours |
| **How many at once** | One per laptop | As many as you can afford |
| **Where it runs** | A machine that stays around | A fresh container that gets thrown away |
| **How much of the repo it touches** | Over time, a lot | Usually a handful of files |
| **Is it online?** | Sometimes | Always |

That last row is the key. Git assumes you might be offline, so it gives you everything up front. Agents are never offline. They'd be totally happy asking a server for files as they need them. Git just wasn't built to work that way by default.

---

## What Anthropic Did About It {#scion}

Anthropic's engineering post on this is worth reading in full, but here's the short version:

- **The breaking point:** merges went from about 2,500 a day in May 2026 to more than 30,000 by September. Its automated jobs were checking out the repo roughly 25 million times a day.
- **The design:** Scion assumes clients are always online. The server handles history, blame, merges, and rebases. Clients keep only a small local cache and fetch the files they actually need.
- **API access:** a job that just needs to ask "who last changed this file?" can query an API instead of cloning anything.
- **The results:** about 200 times less CPU and memory than its old Git mirror setup, checkouts in seconds instead of minutes, and a quarter of its CI capacity freed up.

The catch for us: **Scion isn't public, and Anthropic says it isn't meant as a general Git replacement.** The company shared the design hoping the big Git platforms pick up the ideas.

So until they do, we work with what Git gives us. Which, honestly, is more than most people realize.

---

## What Full Clones Cost You {#cost}

If your repo is small, a full clone is fine and you can skip this post. Once it's big, every agent pays:

**1. Startup time.** An agent can't start working until the clone finishes. On a large repo, that's minutes of waiting before the first useful thing happens, and you pay it every single time.

**2. Disk.** Cloud agent sessions usually have a fixed disk allowance. A big repo's history can eat a real chunk of it before the agent installs a single dependency.

**3. Rate limits and bandwidth.** Ten agents cloning at once looks a lot like ten very enthusiastic people hammering your Git host. Hosts throttle that.

**4. Parallel agents fight over one copy.** If you try to save clones by sharing one, two agents editing the same folder at once is a fast way to get chaos.

**5. Security.** This one gets overlooked. An agent with a full clone can read *everything*: every folder, every old commit, and anything someone accidentally committed and "deleted" three years ago. If an agent only needs one part of the repo, giving it the whole thing breaks the least-access rule we'd apply to any human.

---

## Six Ways to Clone Less {#fixes}

All of these are standard Git and work the same on GitHub, Bitbucket, or anywhere else.

### 1. Shallow clone: skip the history

```bash
git clone --depth 1 <repo-url>
```

You get the latest version of every file and only the most recent commit. It's the fastest, simplest option.

**The tradeoff:** no history. `git log` and `git blame` are mostly useless, and some hosts reject pushes from a shallow clone. Great for "build and test this" jobs, rough for agents that need to understand why code looks the way it does.

### 2. Blobless partial clone: history now, file contents later

```bash
git clone --filter=blob:none <repo-url>
```

You get the full commit history and folder structure, but Git only downloads the actual file contents (the "blobs") for what you check out. Anything else gets fetched the first time something asks for it.

This is my favorite default for agents. The agent can still run `git log`, look at history, and switch branches, but you're not downloading every version of every file ever written.

There's also a **treeless** version (`--filter=tree:0`) that skips even more. It's best for throwaway CI jobs, and slow if the agent starts digging through history.

### 3. Sparse checkout: only the folders that matter

```bash
git clone --filter=blob:none --sparse <repo-url>
cd <repo>
git sparse-checkout set apps/web packages/ui
```

Now the working folder only contains the paths you listed. Pair it with a blobless clone and you're downloading close to only what the agent will actually use.

This is the closest thing to Scion's "fetch what you need" idea that you can use today. It's also a security win: if the agent's working copy only has `apps/web`, that's mostly what it sees.

### 4. Worktrees: one clone, many agents

```bash
git worktree add ../agent-task-a -b agent/task-a
git worktree add ../agent-task-b -b agent/task-b
```

A worktree is an extra working folder attached to the same clone. Each agent gets its own folder and its own branch, but they all share one copy of the history on disk.

Claude Code has this built in. `claude --worktree` starts a session in its own worktree, and you can set `isolation: worktree` on a custom subagent so parallel subagents don't trip over each other's edits. Worktrees with no changes get cleaned up automatically.

**The catch:** worktrees help when your agents run on the same machine. They don't help when every agent gets its own fresh container.

### 5. A shared local mirror: clone once, copy cheaply

```bash
# once, and on a schedule to keep it fresh
git clone --mirror <repo-url> /cache/repo.git

# per agent
git clone --reference /cache/repo.git --dissociate <repo-url> work
```

Keep one up-to-date mirror on the machine (or a shared volume), and have each agent clone using it as a reference. Git copies objects from the local mirror instead of downloading them again, and only fetches what's new.

`--dissociate` makes each agent's clone independent of the mirror afterward, so cleaning up the cache doesn't break anyone. This is a tiny, homemade version of the mirror setup Anthropic ran before Scion.

### 6. No clone at all

Plenty of agent tasks don't need a working copy. "Summarize what changed in this PR," "find where this function is defined," "read the config file" can all be answered from the host's API.

GitHub and Bitbucket both let you read file contents, diffs, and commit history over their APIs, and an [MCP server](/blog/mcp-vs-api-whats-the-difference-claude) can give an agent those as tools. Zero clone time, zero disk.

**The catch:** API rate limits, and it falls apart the moment the agent needs to run tests or build something.

---

## GitHub and Bitbucket Specifics {#platforms}

Most of the pain shows up in CI, because that's where agents and automated jobs clone the most.

| | GitHub Actions (`actions/checkout`) | Bitbucket Pipelines |
|---|---|---|
| **Default depth** | 1 commit | 50 commits |
| **Full history** | `fetch-depth: 0` | `clone: depth: full` |
| **Sparse checkout** | Built in (`sparse-checkout` input) | Do it yourself in a script |
| **Partial clone** | Built in (`filter` input) | Do it yourself in a script |
| **Skip the clone** | Don't add the checkout step | `clone: enabled: false` |

**GitHub** makes this easy. The checkout action already does a shallow clone by default and has inputs for sparse checkout and partial clone filters.

**Bitbucket** is a little more manual. Pipelines does a 50-commit shallow clone by default, and the `clone` options cover depth, LFS, and turning the clone off entirely. For a blobless or sparse clone, the usual trick is to turn off the automatic clone with `enabled: false` and run your own `git clone` commands in the step's script.

One more Bitbucket reality check: a lot of agent tooling is built GitHub-first. (Claude's managed Code Review, for example, is GitHub only, as I covered in my [Code Review post](/blog/claude-code-review-cost-pros-cons).) If you're on Bitbucket, expect to wire more of this up yourself.

---

## Gotchas {#gotchas}

Every one of these fixes has a sharp edge:

- **Shallow clones lose tags.** Tags older than your depth won't be there. If a job reads version tags, give it more depth or full history.
- **Shallow clones can't always push.** Some hosts reject pushes from shallow clones. If an agent needs to push, test it before relying on it.
- **Blobless clones go back to the server.** Running `git blame` or `git log -p` on old files triggers downloads. That needs network access and working credentials, and it can be slow if an agent does a lot of it.
- **Sparse checkout hides callers.** If the agent only sees `apps/web`, it can't see the code in `packages/api` that calls what it's changing. Tell it what it *can't* see, in your `CLAUDE.md` or task prompt, so it doesn't assume nothing else exists.
- **Worktrees share one Git database.** Two worktrees can't check out the same branch at once, and deleting the main clone breaks every worktree attached to it.
- **Large files are their own problem.** If your repo uses Git LFS, decide whether agents need those files at all. Skipping LFS downloads is often the biggest single speedup.

---

## Pick by Job {#pick-by-job}

Here's how I'd match the checkout to the job:

| The agent needs to... | Use |
|---|---|
| Read a few files or a diff | **No clone.** API or an MCP server. |
| Build and run tests, no history needed | **Shallow clone** (`--depth 1`) |
| Edit code in one part of a big repo | **Blobless + sparse** |
| Understand history (blame, log) | **Blobless**, not shallow |
| Run in parallel with other agents on one machine | **Worktrees** |
| Start fast, many times a day, on the same machine | **Shared mirror + `--reference`** |
| Work across the whole codebase | **Full or blobless clone.** Sometimes you really do need it all. |

My rule of thumb: **start every agent with the least repo that could work, and only widen it when the agent proves it needs more.** That's the same least-access idea we use for permissions, applied to code.

---

## What I Want From GitHub and Bitbucket {#wishlist}

Anthropic's post reads a lot like a wishlist for the platforms the rest of us use. Here's mine:

- **History and blame over the API, fast enough to replace a clone.** Most agent questions about history shouldn't need a local copy.
- **Agent-friendly rate limits.** Limits built for humans clicking around don't fit 20 agents working in parallel.
- **Access scoped to folders, not just whole repos.** Let me give an agent a token that can only read and write `apps/web`.
- **First-class partial and sparse clones everywhere,** including Bitbucket Pipelines, without hand-written clone scripts.
- **Merge queues built for agent volume.** Anthropic found a classic one-at-a-time merge queue didn't hold up at agent scale.

Until then, the Git features in this post are the workaround. They're not glamorous, but they're free and they work today.

---

## TL;DR {#tldr}

| Fix | What you get | Watch out for |
|---|---|---|
| Shallow (`--depth 1`) | Fastest, latest files only | No history, missing tags, push issues |
| Blobless (`--filter=blob:none`) | Full history, files on demand | Old files fetched from server when used |
| Sparse checkout | Only the folders you list | Agent can't see code outside them |
| Worktrees | Many agents, one clone | Same machine only |
| Shared mirror (`--reference`) | Fast repeat clones | You maintain the mirror |
| No clone (API / MCP) | Instant, zero disk | Rate limits, can't build or test |

- **The problem:** Git gives everyone the whole repo because it assumes you might be offline. Agents are always online and usually need a few files.
- **What Anthropic did:** built Scion, an always-online version control system. It isn't public.
- **What you can do today:** blobless + sparse clones, worktrees, a shared mirror, or skip the clone for read-only tasks.
- **Bitbucket note:** Pipelines defaults to a 50-commit shallow clone; anything fancier means your own clone script.
- **Rule of thumb:** give each agent the least repo that could work.

---

**More posts:**
- [MCP vs API: What's the Difference, and Why Claude Uses MCP](/blog/mcp-vs-api-whats-the-difference-claude)
- [Claude Code Review: What It Does, What It Costs, and Whether It's Worth $15-25 a PR](/blog/claude-code-review-cost-pros-cons)
- [Claude Code Mods, Explained: I Built One That Watches My Own Agent](/blog/claude-code-mods-2026)

---

*// hereshecodes.com*