git archive — what a tarball leaves behind

A tarball is the artifact a repository produces for everyone who will never clone it. A release asset, a vendored dependency, a source drop for a packager, the thing CI attaches to a build. The command is one word — git archive — and what comes out looks like the repository: same file names, same directory shape, same README. It is not the repository, and the distance between the two is where the incidents live. A submodule arrives as an empty directory. A build fails on a machine that has the source and not the ignored file the source needs. A checksum changes on every run, so nothing downstream can be verified. A tarball has no line to the commit it claims to represent, and the person who has to prove otherwise is you.

This is a post about that distance. The git behaviour below is version-pinned to 2.56.0, and every command and number is output I ran rather than behaviour I remembered. One correction to the usual framing, though: the interesting variables are not all in git. Some live in your configuration, and one is the compressor on whichever machine does the archiving — so where a result depends on the machine, I say which machine produced it.

The scope is the export path: what leaves a repository when you ship it, and what the resulting artifact can prove about itself. The working copy is a different post, running a historical commit another, and automating that in a pipeline a third. This is about the archive you hand out, not the tree you build in. Examples are shell throughout.

What git archive actually exports

The first thing to internalise is that git archive reads the object store, not your working directory. It walks a tree and streams the blobs it finds. Files that are not in the tree — untracked files, ignored files, anything you have not committed — do not exist as far as the command is concerned. The .git directory is never part of the output; there is no history, no refs, no configuration, no hooks.

I built a small repository to make the boundary visible. Tracked: README.md, src/app.sh, docs/notes.md, version.txt, and a .gitignore that ignores build/ and .env. Present in the working directory but never committed: an untracked wip.txt, plus the ignored .env and build/out.txt. There is deliberately no .gitattributes yet — it arrives two sections down, and the order matters.

Eleven paths exist in the working directory. Five of them are tracked. The archive lists seven entries, because the two directories that contain tracked files appear as entries of their own:

Mapped back to the working directory, the omissions sort themselves into exactly three reasons, and none of them is "git decided to skip it":

Path in the working directory	Tracked	In the archive
README.md, src/app.sh, docs/notes.md, version.txt	yes	yes
.gitignore (tracked, and a surprise to some)	yes	yes
wip.txt	no — never committed	no
.env, build/, build/out.txt	no — ignored	no
.git/	—	no

Two entries in that table deserve a second look. The first is that the ignored files are absent for the ordinary reason — they were never added — and not because git archive consulted .gitignore at all. It does not read .gitignore, and it does not need to: an ignored file is simply not in the tree. The second is the opposite surprise: the file whose whole job is to keep things out is itself tracked, so it ships. The archive documents the rules that shaped it.

The output format is tar unless you say otherwise, and zip, tar.gz and tgz are built in. If you pass -o with a recognised extension, the format is inferred from the filename; --prefix prepends a directory name, which is what you want for a source drop that should not explode into the current directory.

One useful property falls out of the design: because the command reads trees and not a working directory, it runs happily in a bare repository. A server can hand out a tarball for a commit it has never checked out, which is the basis of the --remote form and of git upload-archive. That does not make the remote form unrestricted, though, and the restrictions are worth knowing before you build a pipeline on it. The manual's SECURITY section is explicit:

• a client may request a commit or tree pointed to directly by a ref — git archive --remote=origin v1.0;
• a client may request a sub-tree with the ref:path syntax — v1.0:Documentation;
• a client may not use other SHA expressions, "even if the end result is reachable" — so master^ and a literal abcd1234 are both refused.

Those rules protect objects that were removed from history but not yet pruned, and they can be relaxed with uploadArchive.allowUnreachable, which defaults to false. The consequence for this article is not trivia: in remote mode the ref is the only thing you are allowed to name, which is precisely the mutable thing the later sections argue against pinning to. Hold that thought until the section on branches. There is one more asymmetry worth stating here — GitHub does not implement upload-archive at all. The tarball behind "Source code (tar.gz)" is served by a separate HTTP endpoint, which means its stability is a GitHub policy question rather than a protocol one. That turns out to matter a great deal.

A commit and a tree are not the same artifact

Here is the sentence from the git archive manual that decides everything downstream, quoted from the version this post was written against:

When a tree ID is provided, the current time is used as the modification time of each file in the archive. On the other hand, when a commit ID or tag ID is provided, the commit time as recorded in the referenced commit object is used instead.

Read that twice, because the difference is not cosmetic. HEAD is a commit. HEAD^{tree} is a tree, and so is v1.4.0^{tree}. Archiving the commit is deterministic; archiving the tree is not, because every run stamps the current wall-clock time into the archive. The same commit, archived twice, produces identical bytes. The same tree, archived twice, does not:

Two seconds of wall clock is the only difference between those last two files, and it is enough to break every checksum, cache key and signature that expected a stable artifact.

Losing determinism is the smaller half of the problem. The larger half is provenance, and the manual is as explicit about it as it is about time: the commit ID is stored in a global extended pax header in the tar format, and in the ZIP format it is stored as the archive comment. When you archive a tree, there is no commit ID to store, and the header is simply absent. git ships a command to read it back — git get-tar-commit-id — and the contrast between the two forms is a single command:

The first prints the commit the archive was cut from. The second prints nothing and exits non-zero, because a tree archive carries no such claim. The commit ID is in the file even where a listing will not show it — the tar header is a pax global entry, which tar -tf hides and which you have to go looking for in the bytes:

The zip form is friendlier about it: the same hash comes back as the archive comment, which unzip -z prints on its own, and which unzip -l shows as a stray forty-character line above the file table rather than in it:

There is a third consequence of the same choice, and it is the one that reaches furthest downstream: a tree archive also turns off export-subst, the attribute that substitutes placeholders in a file as it is added. It needs the export rules to demonstrate, so it is shown in the next section — where the lab finally commits a .gitattributes.

Put the timestamps and the provenance together and the rule writes itself: archive the commit or the tag, never the tree. The trap is not exotic. The git manual's own examples show v1.4.0^{tree} used specifically to produce an archive "without a global extended pax header" — which is exactly the provenance you spent the release process creating. --mtime will pin the timestamp of a tree archive and give you determinism back, but it cannot put the commit ID back, because a tree does not have one.

A detail of the timestamps is worth stating here because it is counter-intuitive and because it sets up the reproducibility section later. Every entry in the archive gets the same modification time — the commit's — regardless of when each file last changed. A file untouched for four years and a file edited in the last commit carry the same stamp, because the value comes from the commit being archived and not from each path's history. Archive an older commit and all the stamps move back together. The consequence is a build hazard rather than a curiosity. Unpack that archive over an existing tree that still holds build outputs and the sources arrive older than the artefacts they ought to invalidate, so make — or anything else that compares mtimes — concludes that everything is up to date and keeps the stale output. Extract into a clean directory, or touch what you unpacked.

The export rules live in the tree you archive

What goes into the archive is shaped by two attributes, and both are read from the tree being archived rather than from your working tree. export-ignore keeps paths out; export-subst expands placeholders in the ones that stay. That is the whole vocabulary, defined in the gitattributes manual, and it is worth knowing precisely because the source of the rules is where people get caught.

Back to the lab: commit the rules and the listing changes. This is the same repository as before, with only .gitattributes added and committed:

Compare that with the seven entries earlier: docs/notes.md has disappeared, and docs/ has stayed. Filtering out the only file in a directory removes the file but leaves the directory entry behind, as an empty folder — zero occurrences of the path, one of its parent:

This is also why the lab in the first section had to exist without export rules: had the export-ignore rule been in place before that first listing, that listing would have been this one, and the "everything tracked ships" baseline would have been a claim the reader could not reproduce.

The second rule in that file, version.txt export-subst, is where the export path meets the commit-versus-tree distinction from the previous section. The manual states the dependency plainly: the expansion requires a commit ID, and if git archive has been given a tree instead of a commit or a tag, no replacement is done. So a version.txt containing $Format:%H$ becomes the commit hash under the commit form, and stays a literal placeholder under the tree form:

That is the third thing the tree form gives up, and the most public one: the archive that cannot name its commit hands the consumer a file that says the same thing, in the format the consumer's build is most likely to read.

The consequence that catches people out is that an uncommitted .gitattributes does nothing. The two experiments below run in a separate scratch repository — the main lab is finished with its attributes — which also carries two extra files, drop.txt and keep.txt, so the effect is visible in a path list. If you decide on Friday that drop.txt should not ship and edit .gitattributes without committing it, the default archive still contains the file, because the rules come from the committed tree:

The --worktree-attributes flag is the escape hatch: it tells git to consider the .gitattributes files in the working tree as well, so you can tweak the output after the fact without another commit. The other escape hatch is $GIT_DIR/info/attributes, which applies to every archive from that repository and is not itself part of any tree — the right home for a rule that is about your packaging and not about the project:

What silently does not leave

The omissions that hurt are the ones that do not announce themselves, because the archive extracts cleanly and fails later. The clearest example is a submodule. A submodule is a gitlink — a recorded commit, not content — so the parent tree contains a directory entry and a .gitmodules file, and nothing else. Add a submodule with a single tracked file in it, archive the parent, and the tarball tells the consumer that a submodule exists while shipping none of it:

The directory is there, the file inside it is not, and the extraction succeeds. What the consumer can do is one command — git submodule update --init — and it works only if the URL in .gitmodules is still reachable from where they are standing. Anyone who builds from that tarball without that step gets whatever the submodule's absence produces: a missing header, a failed make, a test that skips instead of failing.

The same shape recurs wherever content lives outside the tree: generated files that are ignored, secrets and configuration that are deliberately not committed, dependencies that are installed rather than vendored, and any tooling that populates the working directory after a checkout. A worktree is a fresh checkout of tracked files, and an archive is a fresh checkout of tracked files with even less around it — this is the same property that makes a linked worktree arrive without your .env and node_modules, applied to a file you send to a stranger.

Since git 2.39 there is a direct answer for the cases where the missing file is small and generated: --add-file copies a file that is not in the tree into the archive, and --add-virtual-file writes content you supply on the command line. Both are repeatable. Their path handling is the part that surprises people, so the three commands below are worth reading as much for where the entries land as for what they contain:

Two cautions the manual does not spell out. --add-file does not check whether the path is already in the tree; pointing it at a tracked file appends a second entry with the same name, and the last one wins on extraction, so keep it for files that are genuinely outside the tree. And --add-virtual-file can inject the commit ID — --add-virtual-file=COMMIT:"$sha" does exactly that — but it injects the value your script passed in, not the value of the object being archived. That is the difference from export-subst, and the reason to prefer the attribute when the value must not drift: export-subst reads the object it is describing, so it cannot disagree with it.

For a submodule that must travel inside the artifact, there is no flag; the pattern is to archive the parent and then append each submodule's own archive — with git submodule foreach, which runs a command in every submodule, or with a wrapper that already does it, such as git-archive-all. Appending is where the naive version fails: every tar stream ends with zero blocks that mark end-of-archive, so cat-ing two archives together gives you a file whose reader stops at the first one and never sees the rest. GNU tar reads past the boundary with --ignore-zeros, or joins the two properly with tar -A -f a.tar b.tar. bsdtar has no -A, but it does have the @ syntax, which reads an archive as an input: tar -cf joined.tar @parent.tar @sub.tar, tested here on macOS with bsdtar 3.5.3, produced all nine entries from a pair of four- and five-entry archives, without unpacking anything. Either way the result stops being a single archive of a single commit, and its commit ID means what it means for the parent only.

The quiet inverse is also worth naming: the archive ships the files whose job is to exclude things. .gitignore, .gitattributes, the CI configuration, the linter config — all tracked, all present. A consumer who knows to look gets a full account of how the project intended to be built and packed. That is usually a feature. It is only a problem when the archive was also carrying something those files were supposed to keep out, which is a different failure and a worse one.

Is the tarball reproducible

The short answer is that git builds determinism into its own compressor, and that four other things can still change the bytes of the result: the version of git doing the compressing, the gzip you may wrap the stream with, your own git configuration, and the attributes that rewrite content on the way out. The tree-versus-commit mistake from the previous section is a fifth, and by now the obvious one.

The long answer starts with a version number. Since git 2.38, git archive --format=tar.gz does not spawn an external gzip; it compresses in-process, through zlib. Before that, it shelled out. That one change is the loudest reproducibility incident in this subject's history, and it is worth walking through because it is the exact failure mode a private release pipeline can reproduce every day without noticing.

The built-in path is visible in the first ten bytes of its output. The modification-time field at offset four is zero, and the operating-system byte is 03, Unix — the gzip wrapper from zlib, not from your system's gzip:

Two runs agree, and so does a second machine. I archived the same commit — a repository with about a megabyte of generated source, so the compressor has real work to do — on macOS with git 2.56.0 and on Debian with git 2.47.3, and got the same 49,124 bytes:

That is the property people mean when they say a tarball is reproducible, and it is the first of the four variables: the version of git decides the compressor, and the compressor decides the bytes. It is also where the usual article stops, and where the more useful part begins, because the same commit archived by hand does not come out the same on both machines. Wrapping the stream yourself is the next source, and how badly it goes depends on which gzip you have. On macOS, with Apple gzip 479, gzip -n reproduces git's built-in bytes exactly — which is a pleasant coincidence and not a guarantee, because Apple's gzip is a zlib client just as git is. On Linux, with GNU gzip, it does not:

Same commit, same tar stream, same flag, different bytes, purely because GNU gzip's deflate produces a smaller stream than zlib's at the default level. A third implementation makes the same point: busybox gzip, in an Alpine container, matched GNU gzip byte for byte on the same input and disagreed with git. So the honest form of the reproducibility claim is narrower than the folk version. The tar payload is machine-independent — that is git's doing — but the gzip container is the business of whichever deflate implementation the archiver happens to use, and the built-in one is the only one git guarantees. If you wrap, you have added a second implementation to your reproducibility contract and you should pin it.

Wrapping has a cruder failure mode too, and this one costs nothing to avoid: the bare gzip that the manual's own examples reach for, which stamps the current time into the container it creates and reinstates exactly the non-determinism git had carefully avoided. Two runs two seconds apart:

Bytes four through seven are the timestamp, and it moved. gzip -n suppresses the field — which is why the earlier comparison used it — but as the Linux result showed, suppressing the field makes two runs agree, not two implementations. The two questions are separate, and only the built-in path answers both.

Next is your own configuration, the source most likely to be invisible because it lives in a dotfile that CI sets differently from your laptop. git archive sends each blob through the same working-tree conversion that a checkout does, so anything that rewrites content on the way out rewrites the archive too. Set core.autocrlf=true and the text files inside the tarball gain CRLF bytes — the commands below name the commit explicitly, and the first digest is the one c1.tar has:

core.autocrlf=input leaves the same archive unchanged, which is a useful reminder that this is not about the platform but about the value. tar.umask belongs in the same drawer: it rewrites the permission bits of every entry, which is enough to change the bytes on its own. With tar.umask=0077 every file entry in the same archive drops from rw-rw-r-- to rw-------, and every directory from rwxrwxr-x to rwx------.

The last of the four is the set of attributes that own content: ident expands $Id$ on the way out, text eol=crlf converts line endings, a filter driver's smudge side runs, and a repository using Git LFS will put either a pointer or the real file in the archive depending on which filters are installed on the machine doing the archiving. None of it shows up in a tar -tf listing, and all of it changes the bytes.

Which brings the section back to the incident. git 2.38 shipped in October 2022 with the switch to the built-in compressor — for tar.gz and tgz only; the zip writer has always compressed in-process. On 30 January 2023 GitHub deployed that version to the service that generates source downloads, and every "Source code (tar.gz)" archive on the site changed its checksum without a single byte of content changing. The damage was not in GitHub's release assets; it was in the derivative pipelines — build systems and package managers that had pinned the checksums of the tarballs they download. Bazel, Homebrew and Nix all broke. GitHub reverted the change the same week and, on 21 February 2023, published a commitment: source archives would stay byte-for-byte stable for at least a year, in both the .tar.gz and the .zip form, with six months' notice before any future change. The Bazel post-mortem is the clearest account of the mechanism, and the maintainer thread on GitHub is the view from the receiving end.

Both documents are also the reason this section has been careful to say tar.gz and not "archives", because the zip form has a machine-dependence of its own, and it is not the compressor. Each zip entry carries a timestamp in local time, so the same commit, archived by the same git, comes out as different bytes on machines in different timezones: the lab's archive is 990 bytes either way, with a digest of 1cdd3dd4… under TZ=Europe/Warsaw — identical to the macOS build — and c5b439ba… under TZ=UTC. The tar stream, by contrast, does not care at all: the same commit archived as --format=tar yields one digest under UTC, under Europe/Warsaw and under Asia/Tokyo alike. GitHub's own post names that field in its list of things it wishes it could fix — "timestamps embedded in zipballs; dependency on system gzip for tarballs" — which is the same pair of problems this section has just measured. The fix on your side has the shape of everything else here: pin the timezone in the image that builds the release, next to the git version.

The lesson generalises past GitHub, because the mechanism is available to anyone. git archive --format=tar.gz is reproducible across machines for a fixed git version, and the version number is part of the artifact's identity whether or not it appears in the bytes. An upgrade from 2.37 to 2.38 changes nothing about the tree and everything about the file. Pin the git version in the image that builds your release, note it next to the checksum, and the claim stops being folklore.

Archive the SHA, not the branch

Everything above assumes you archived the same thing twice. In a pipeline, the interesting failure is that you archived two different things while believing they were one — because branches move, and an archive is a snapshot of whatever the ref pointed at when the command ran. The window is small and entirely real:

Both commands are correct, and that is the point: the first asked for a commit and got the file as that commit had it; the second asked for whatever main was at the moment it ran, after a further commit had moved it. A build that reads a branch name, then archives it, is racing the people pushing to that branch — and the failure is silent, because both tarballs are valid and both extract cleanly. The discipline is boring and it is the whole fix: resolve the ref once, archive the immutable object, and record which one you archived.

rev-parse --verify plus a ^{commit} is what makes the line safe rather than merely tidy. The verify fails if the name does not resolve, and the peel guarantees you are holding a commit ID — so a lightweight tag, an annotated tag or a branch all arrive as the same kind of thing, and a piece of shell that reaches for $sha cannot accidentally be handed a tree. The branch is consulted exactly once, and after that the artifact is named by a value that cannot move.

The advice that "a tag is as good as a SHA" is common and half right, and the two halves are worth separating. A tag is a mutable reference in exactly the way a branch is: it can be deleted, and it can be force-updated to point somewhere else. What makes a tag acceptable for a source archive is not immutability but the pax header — whatever the tag pointed at when the archive was built is recorded inside the file, so the artifact still names the commit even if the tag later moves. For pinning a build you want a value with no name at all; for a human-facing label you want the tag; and the reason to prefer the SHA in the pipeline is that it removes a failure mode instead of documenting one.

That is not only my preference. GitHub's guidance on archive stability reaches the same conclusion from the other end: fetch the archive through the REST API with "a commit ID for the :ref parameter", because "there is no need to record the hash, since the commit ID ensures you'll always get the same file contents inside the archive" — and because a commit ID makes the request "immune to repositories rewriting tags or moving branch heads", which is precisely the hazard this section is about. Read that promise precisely, though, because it stops one word short of this post: GitHub guarantees the same contents, not the same bytes, and the distance between those two words is the whole subject. The commit ID fixes what is inside the archive; the checksum in the checklist is for the file itself.

Then there is the tension the remote section left open. In --remote mode a bare SHA is refused by default — rule 3 again — so the one form of the command where the ref is the only thing you may name is also the form where naming a ref is least safe. The resolution is to do the pinning out of band: resolve the ref locally, archive the ref remotely, and confirm that the commit ID the archive reports is the one you resolved. That is the check the next section builds, and it is why the check has to exist at all — it is the only thing standing between a moving ref and a release nobody can point at.

What the artifact can prove about itself

None of the provenance machinery is useful unless the consumer uses it, and using it means three steps rather than one, because the commit ID inside the archive is a claim rather than a proof. The first step reads the claim; the second asks whether the commit exists; the third — the one usually skipped — rebuilds the archive and compares it byte for byte. git supplies all three, and c1.tar from the commit-versus-tree section is the guinea pig:

That last line is the whole verification, and it retires the archive's own header as evidence. The header is content, not a signature: it can be edited, and an edited header that names a different commit still reads back as a perfectly plausible forty-character hash:

The forged archive passes the first step — it reports the commit I told it to report — and fails the third. cmp stops at the first byte that differs, which in this pair is the pax header's own timestamp, the two commits having been made minutes apart; everything behind it differs too, because behind it are two different trees. This is the difference between reading a claim and checking it, and it is why the checklist below records the SHA and the archive's flags alongside the artifact: the check is cheap, it needs only the repository and the file, and it is the only step that notices.

One honest limit, and it is the one worth stating plainly: all of this proves which commit a tarball corresponds to. It does not prove who produced the file. Anyone who can rebuild the archive from the same commit writes the same bytes and the same ID — which is the desired property for reproducibility and a useless one for authenticity. If the question is "did this come from you", the archive is not the layer that answers it; that is what a signature on the tag, or on the artifact itself, is for. The archive's contribution is still real, but it is a hint that ties the pieces together rather than a proof, and treating it as proof is how a tampered file gets a clean bill of health.

When a tarball is the wrong artifact

A tarball is the right answer when the consumer needs the files at one point in time and nothing else: a packager building a package, a vendoring step, a source drop to a mirror. It is the wrong answer in several cases, and the cost of discovering that late is high, because the failure surfaces at the consumer's build and not at your release.

• The consumer needs history. An archive carries a tree, not a repository; there is no log to bisect, no parent to diff against, no blame. When the recipient must work in the history rather than build it once, the right primitive is git bundle, which packages objects and refs into a single file that a clone or fetch can read, and which can be verified with git bundle verify before it is used. Reach for a bundle for air-gapped transfers, backups and handoffs; reach for the archive for artifacts.
• The consumer needs a working checkout. Hooks, .git state, the ability to commit and then push — none of it is in the archive, because none of it is in a tree. If the deliverable is "a checkout they can keep working in", that is a clone, or a bundle, and no amount of --prefix changes it.
• The build depends on things that are not tracked. This is the submodule case and the ignored-file case and the generated-file case, and they share a shape: the archive is a faithful copy of the tree, and the tree was never self-sufficient. The fix is not in git archive. It is either committing what the build needs, or adding it with --add-file, or shipping a checkout, or documenting the un-archive step and making the consumer's build fail loudly when it is skipped.
• Integrity must be provable, not just asserted. An archive is an unsigned derivative of a commit. If you need a chain that survives an adversary rather than an accident, the archive is the wrong last link — sign the tag, ship a bundle, or sign the artifact as its own object, and let the commit ID inside the tarball be the hint that ties the three together rather than the proof.

The common thread is that a tarball answers exactly one question — what did the tracked files look like at this commit — and each failure above is a question it was never built to answer. The discipline is to say which question the release is answering, and to check that the recipient is asking the same one.

Release checklist

If you keep one thing from this post, keep the six lines:

1. Resolve the ref once. sha=$(git rev-parse --verify "$REF^{commit}") — and never let a branch name reach a later command.
2. Archive the commit, with the built-in compressor. git archive --format=tar.gz -o "$name.tgz" "$sha". No | gzip; if you must wrap, use -n and pin the gzip implementation too.
3. Pin the git version in the image that builds the release. 2.38 changed the bytes of every .tar.gz for the same commit, and nothing in the file says so.
4. Archive under a known configuration. core.autocrlf, tar.umask and any ident, eol or filter attributes rewrite the bytes; record them or neutralise them. If you ship zip as well, pin the timezone too, because every entry's timestamp is written in local time.
5. Record the identities and the recipe next to the artifact: the commit ID from git get-tar-commit-id, the sha256 of the file, and the exact flags you archived with — --format, --prefix, --add-file, --add-virtual-file. Verification compares bytes, so the re-archive has to repeat them.
6. Verify, don't trust. Three commands below, and note the first one: get-tar-commit-id reads a raw tar, so a .tar.gz has to have its gzip layer stripped first or you get fatal: EOF before reading tar header.

A forged header passes the first two lines and fails the third. The --prefix is not decoration: an archive built with a prefix and rebuilt without one fails cmp at char 13, which is a false alarm on an honest artifact, and the same goes for the flags that follow. The two --add-* options are a sharper version of it, because they inject content that is not in the repository at all: an archive that uses them cannot be re-derived from the commit, so record the injected files next to the checksum, or verify against a stored copy of the archive instead of regenerating it. For a plain .tar rather than a .tar.gz, the first line loses its pipeline and becomes git get-tar-commit-id < "$name.tar".

None of the six is novel, and that is the point: the archive's determinism, its provenance and its verifiability are all things you opt into, one line at a time, against defaults that are permissive rather than safe. Do them and the tarball becomes a claim you can check. Skip them and it stays a file you have to take on faith — which is exactly what everyone downstream will do, because the archive extracts cleanly either way.