Git-Annex from scratch#

I found I missed an under-the-hood explanation of what Git-Annex added on top of an ordinary Git repository.

I’m partly starting from Joey Hess’ talk.

Here is a set of fundamental objects for Git-Annex, from that talk:

Git-Annex objects (GAOs)#

GAOs are objects in the same sense that Git stores objects in its object store.

For Git (not Git-Annex), when we do git add myfile, Git takes the contents of myfile (well, in fact, a compressed version of myfile) and copies it into the .git/objects directory, with a filename generated from the SHA1 hash of the object.

However, Git-Annex is designed to work with very large files that should not be compressed or copied. So when you create a Git-Annex object (with git annex add mybigfile):

The file is moved, not copied.
Within the working tree, Git-Annex leaves behind a pointer (usually a symbolic link) to the now-moved file. See below.
The moved file is not compressed.
The contents go into .git/annex/objects instead of .git/objects.
Git-Annex removes write permission on the moved file, to prevent you overwriting the file accidentally (see lock/unlock).
There are different rules for making the hash filename that we don’t need to worry about for now.

Git-Annex pointers#

Above, we said that when we make a file into a Git-Annex object, we leave a pointer to the now-moved file, in the working tree. The pointer is (on a Unix system) a symlink to the now-moved file in the .git/annex directory.

(Some systems, such as Windows by default, can’t use symbolic links; in this case Git-Annex can use pointer files).

It is these pointers (usually symbolic links) that we check into the Git repository.

Git-Annex remotes#

Git-Annex has remotes. Remotes are stores for GAOs.

This can be confusing, because:

Git-Annex can re-use ordinary Git repositories for this purpose. In this case the remote in the Git-Annex sense can also be a remote in the Git sense.
But Git-Annex also has the concept of a special remote, that is not a Git repository, and therefore cannot be a remote in the Git sense. Special remotes have various types of storage backing and interfaces, such as rclone pointing to Google Drive and so on.

I’ll call both Git re-used repositories, and special remotes — Git-Annex Remotes (GARs) — because they have the specific purpose of storing the GAOs. GARs are pointers to particular data stores that can store GAOs.

Each GAR has a Universally Unique Identifier (UUID), so that Git-Annex can record which of its remotes has each of the potential GAOs.

As this implies, any one individual GAR need not store all possible GAOs - it might have a subset. Git-Annex has to keep track of this information, so it can git annex get any files that the user asks for, that they do not already have in their .git/annex GAO object store.

Here is the confusion — your local Git repository also functions as a GAR — and has its own UUID. That is because, on your machine, after you git annex add (or, as you see later git annex get) a large file, your own local .git directory will have a copy of the GAO — in .git/annex/objects. So Git-Annex has to know that you also have a copy of the file, in your .git/annex/objects directory, so you or your collaborators can ask for a copy of the file from you, if they need to (using git annex get etc).

To say it again - a GAR is a different thing from a standard Git remote. It is specifically a place that Git-Annex can store or retrieve GAOs. Your local .git repository store is one such place, so it also functions as a GAR, from which you, or other people connected to your network, can get GAOs.

Git-Annex metadata#

Git-Annex makes and uses its own branch named git-annex to store metadata about the location of GAOs within the known remotes. We’ll come across that branch in the walkthrough below.

Walkthrough#

Start with standard Git#

Make a new directory for the Git repository, that will soon also be Git-Annex-enhanced.

mkdir my-repo
cd my-repo

When we do git init we have the usual .git subdirectories — this is, so far, a normal Git repository.

git init
ls .git

Initialized empty Git repository in /tmp/tmp_pf0j390/annex-repos/my-repo/.git/
HEAD  config  description  hooks  info	objects  refs

tree .git/objects

.git/objects
├── info
└── pack
3 directories, 0 files

echo "Some text" > my_small_file
# Makes my_small_file into a Git object, stored in .git/objects
git add my_small_file
tree .git/objects

.git/objects
├── 01
│   └── b51e6ac5e39d1eb2fec3143437c6f117a58f03
├── info
└── pack
4 directories, 1 file

Make an ordinary Git commit. This generates a new directory-listing object, and a new commit object — the usual Git behavior.[1]

git commit -m "Added first file"
tree .git/objects

[main (root-commit) fb62273] Added first file
 1 file changed, 1 insertion(+)
 create mode 100644 my_small_file
.git/objects
├── 01
│   └── b51e6ac5e39d1eb2fec3143437c6f117a58f03
├── 7b
│   └── 527389ed9d0c64db733e80fadb582e211b5b7c
├── fb
│   └── 62273823002a788458e21e398bc0e5a88406dc
├── info
└── pack
6 directories, 3 files

We only have one branch - the default main branch:

git branch -a

* main

Here’s the current (default) .git/config file. Remember, this file is local to the repository, and does not get pushed with a git push. Notice here there is no [annex] section.

cat .git/config

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true

By default, Git doesn’t have a pre-commit file in hooks. We’ll see later that Git-Annex adds one.

ls .git/hooks

applypatch-msg.sample	   pre-commit.sample	    prepare-commit-msg.sample
commit-msg.sample	   pre-merge-commit.sample  push-to-checkout.sample
fsmonitor-watchman.sample  pre-push.sample	    sendemail-validate.sample
post-update.sample	   pre-rebase.sample	    update.sample
pre-applypatch.sample	   pre-receive.sample

Add Git-Annex features with `git annex init`#

Now we overlay the Git-Annex stuff on the normal Git repository:

git annex init

init  ok
(recording state in git...)

Notice that we now have a new annex directory in the .git directory.

ls .git

COMMIT_EDITMSG	annex	description  index  logs     refs
HEAD		config	hooks	     info   objects

.git/annex doesn’t have any objects yet, just some housekeeping files.

tree .git/annex

.git/annex
├── fsck
│   ├── fsck.lck
│   └── fsckdb
│       └── db
├── index
├── index.lck
├── journal
├── journal.lck
├── mergedrefs
├── othertmp
├── othertmp.lck
├── sentinal
└── sentinal.cache
5 directories, 9 files

We also have a new git-annex branch:

git branch -a

  git-annex
* main

Next we look at git annex info to show the Git-Annex-Remotes:

git annex info

trusted repositories: 0
semitrusted repositories: 3
	00000000-0000-0000-0000-000000000001 -- web
 	00000000-0000-0000-0000-000000000002 -- bittorrent
 	93a51ec7-3dc0-4438-b553-0e76636779f4 -- runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo [here]
untrusted repositories: 0
transfers in progress: none
available local disk space: 91.43 gigabytes (+100 megabytes reserved)
local annex keys: 0
local annex size: 0 bytes
annexed files in working tree: 0
size of annexed files in working tree: 0 bytes
combined annex size of all repositories: 0 bytes
annex sizes of repositories:
backend usage:
bloom filter size: 32 mebibytes (0% full)

For the moment, notice the “semitrusted repositories”. These are GARs - places that Git-Annex knows can be sources or destinations for the Git-Annex-Object files. The first two (web and bittorrent) are generic sources corresponding to URLs, and bittorrent files, but we’ll ignore these for now. The third points to the local .git directory. This is a “remote” in a rather confusing sense - that is - it’s a place that can serve as a source for the files that Git-Annex will store (the GAOs). It’s a remote in the sense that other Git-Annex overlaid repositories may be able to use to get those files. You can think of it as a remote to other repositories, and a local store to this one.

After git annex init, the .git/config file has an [annex] section that identifies this repository store (set of files on disk) with its new UUID.

cat .git/config

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[annex]
	uuid = 93a51ec7-3dc0-4438-b553-0e76636779f4
	version = 10
[filter "annex"]
	smudge = git-annex smudge -- %f
	clean = git-annex smudge --clean -- %f
	process = git-annex filter-process

Git-Annex has added a pre-commit and other hooks to allow it to intercept and modify Git commands.

ls .git/hooks

applypatch-msg.sample	   post-update.sample	    pre-rebase.sample
commit-msg.sample	   pre-applypatch.sample    pre-receive.sample
fsmonitor-watchman.sample  pre-commit		    prepare-commit-msg.sample
post-checkout		   pre-commit.sample	    push-to-checkout.sample
post-merge		   pre-merge-commit.sample  sendemail-validate.sample
post-receive		   pre-push.sample	    update.sample

Adding files with `git annex add`#

Now we’ll add some potentially large file, using git annex add. That is going to make a Git-Annex Object (GAO), and leave a pointer (symlink) behind in the working directory.

First we make a file that we will pretend is large. We’ll also calculate its SHA1 sum - you’ll see why later.

echo "Something really large" > my_large_file
shasum my_large_file

eee0481dabe361a026281b3260a37732fa79881d  my_large_file

Now we make that large file into a Git-Annex Object with git annex add:

git annex add my_large_file

add my_large_file
100%  23 B             83 KiB/s 0s
ok
(recording state in git...)

Notice now that:

The file has moved to .git/annex/objects.
The moved file is now read-only (to prevent you accidentally overwriting it — see lock/unlock).
There’s a symlink to that file in the working directory.
Git-Annex added the symlink (and not the file itself) to the ordinary Git staging area.

In order:

# It's the same file exactly as the one previously in the working directory.
shasum .git/annex/objects/*/*/*/*

eee0481dabe361a026281b3260a37732fa79881d  .git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720

# The moved file is read-only to prevent accidental overwrites.
ls -al .git/annex/objects/*/*/*/*

-r--r--r-- 1 runner runner 23 Sep  5 16:27 .git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720

# The file in the working tree has become a symlink.
ls -al my_large_file

lrwxrwxrwx 1 runner runner 180 Sep  5 16:27 my_large_file -> .git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720

# The symlink has been added to the Git staging area.
git status

On branch main
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   my_large_file

git annex list tells us which GAR has each GAO. here is the name for the local GAR, in .git/annex:

git annex list

here
|web
||bittorrent
|||
X__ my_large_file

We can now do a commit - but notice - the thing that gets added to the standard Git history, and stays in .git/objects, is nothing but the symlink:

git commit -m "Add link to my_large_file"

[main f41d3c7] Add link to my_large_file
 1 file changed, 1 insertion(+)
 create mode 120000 my_large_file

git show below identifies the symlink with the file mode 120000:

# Show contents of last commit.
git show main

commit f41d3c791defe41b528f43ba661c9681718f75f0 (HEAD -> main)
Author: ci <ci@example.com>
Date:   Sat Sep 5 16:27:23 2026 +0000
    Add link to my_large_file
diff --git a/my_large_file b/my_large_file
new file mode 120000
index 0000000..aa09c8c
--- /dev/null
+++ b/my_large_file
@@ -0,0 +1 @@
+.git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720
\ No newline at end of file

To recap - the Git .git/objects directory never got a copy of my_large_file. The content of my_large_file belongs to Git-Annex, and is stored in the .git/annex/objects directory, that does not get transferred when you git push, or you git clone the repository. Git itself only has the symbolic link to the file. But as we’ll see soon, it does know where copies of my_large_file live, through the information in the git-annex branch.

Clones and Git-Annex#

To illustrate, if we do a typical clone of this my-repo repository, we do not have the contents of my_large_file; we only have the symlink:

cd ..
# Simulate pushing to some upstream service such as Github.
git clone --bare my-repo my-repo-upstream.git
# Simulate cloning from there.
git clone my-repo-upstream.git my-repo-clone
cd my-repo-clone

Cloning into bare repository 'my-repo-upstream.git'...
done.
Cloning into 'my-repo-clone'...
done.

We have put ourselves into the usual situation, where we’ve done a git push to a remote service, and then done a git clone from the remote service, to another computer. After doing that:

# No .git/annex directory in the clone.
ls .git

HEAD  config  description  hooks  index  info  logs  objects  packed-refs  refs

# But we do have access to the remote git-annex branch, which maps
# repositories to GAOs.
git branch -a

* main
  remotes/origin/HEAD -> origin/main
  remotes/origin/git-annex
  remotes/origin/main

# my_large_file is a broken symlink.
file my_large_file

my_large_file: broken symbolic link to .git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720

If we try and fetch the file to which the symlink points, with git annex get, Git-Annex can’t do it, because it has no record (in the git-annex remote branch) that the origin (upstream) repository has that file in its .git/annex filesystem.

git annex list

here
|origin
||web
|||bittorrent
||||
____ my_large_file

Fetching files with `git annex get`#

However, the git-annex remote branch has told it where we might find the file (see Git-Annex internals). Notice the suggestion generated during the failure:

git annex get my_large_file

get my_large_file (not available)
  Maybe add some of these git remotes (git remote add ...):
  	93a51ec7-3dc0-4438-b553-0e76636779f4 -- runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo
failed
get: 1 failed

This message is telling us that we need to point directly to the original repository as a source for the GAO files (in its .git/annex folder).

# Add the original repository as an ordinary Git remote.
git remote add original-repo ../my-repo --fetch

Updating original-repo
From ../my-repo
 * [new branch]      git-annex  -> original-repo/git-annex
 * [new branch]      main       -> original-repo/main

# Git-Annex now knows it can get the file.
git annex list

here
|origin
||original-repo
|||web
||||bittorrent
|||||
__X__ my_large_file

We ask Git-Annex to fetch the file.

git annex get my_large_file

get my_large_file (from original-repo...)
100%  23 B             10 KiB/s 0s
ok
(recording state in git...)

# There are two repositories that Git-Annex knows have
# the file (in `.git/annex`) - this repo, and the original.
git annex list

here
|origin
||original-repo
|||web
||||bittorrent
|||||
X_X__ my_large_file

# After git annex get, the symlink is fixed, because the file exists.
file my_large_file

my_large_file: symbolic link to .git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720

Copying annex files with `git annex copy`#

# Make another pretend large file.
echo "More large" > another_large_file
# Calculate, store, show hash of file.
alf_hash=$(shasum another_large_file)
echo $alf_hash

9d1363628fdb6b44fe2c632d371ffb62d190dd6c another_large_file

# Add it to the Annex
git annex add another_large_file
# It's a symlink now.
ls -al another_large_file

add another_large_file
100%  11 B             49 KiB/s 0s
ok
(recording state in git...)
lrwxrwxrwx 1 runner runner 180 Sep  5 16:27 another_large_file -> .git/annex/objects/gQ/q1/SHA256E-s11--509997cd7487f362f7f72733f63c5b7bc9e73adfa9df45e53cbf749b6c69c6c3/SHA256E-s11--509997cd7487f362f7f72733f63c5b7bc9e73adfa9df45e53cbf749b6c69c6c3

git commit -m "Add another large file"

[main 7da91e9] Add another large file
 1 file changed, 1 insertion(+)
 create mode 120000 another_large_file

We have this file in my-repo-clone, but the original my-repo does not:

git annex list

here
|origin
||original-repo
|||web
||||bittorrent
|||||
X____ another_large_file
X_X__ my_large_file

# Notice we have the bytes of another_large_file
# (identified by the shasum, run above), as well as
# the original my_large_file
echo "SHA of another_large_file: $alf_hash"
echo "SHA of files in git annex objects directories:"
shasum .git/annex/objects/*/*/*/*

SHA of another_large_file: 9d1363628fdb6b44fe2c632d371ffb62d190dd6c  another_large_file
SHA of files in git annex objects directories:
eee0481dabe361a026281b3260a37732fa79881d  .git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720
9d1363628fdb6b44fe2c632d371ffb62d190dd6c  .git/annex/objects/gQ/q1/SHA256E-s11--509997cd7487f362f7f72733f63c5b7bc9e73adfa9df45e53cbf749b6c69c6c3/SHA256E-s11--509997cd7487f362f7f72733f63c5b7bc9e73adfa9df45e53cbf749b6c69c6c3

We can send our file to the original my-repo:

# Not there at the moment:
shasum ../my-repo/.git/annex/objects/*/*/*/*

eee0481dabe361a026281b3260a37732fa79881d  ../my-repo/.git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720

git annex copy another_large_file --to original-repo

copy another_large_file (to original-repo...)
100%  11 B              5 KiB/s 0s
ok
(recording state in git...)

# It arrived.
shasum ../my-repo/.git/annex/objects/*/*/*/*

eee0481dabe361a026281b3260a37732fa79881d  ../my-repo/.git/annex/objects/4k/p7/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720/SHA256E-s23--5533fe49b4ab945dc8c4b9d878a8c6e74232c8e00fe9a2b2e5f8d88a70406720
9d1363628fdb6b44fe2c632d371ffb62d190dd6c  ../my-repo/.git/annex/objects/gQ/q1/SHA256E-s11--509997cd7487f362f7f72733f63c5b7bc9e73adfa9df45e53cbf749b6c69c6c3/SHA256E-s11--509997cd7487f362f7f72733f63c5b7bc9e73adfa9df45e53cbf749b6c69c6c3

The awe-inspiring power of `git annex sync`#

Let’s add another file.

echo "More larger still" > yet_another_large_file
git annex add yet_another_large_file
git commit -m "Add yet another large file"

add yet_another_large_file
100%  18 B             64 KiB/s 0s
ok
(recording state in git...)
[main b53b9d6] Add yet another large file
 1 file changed, 1 insertion(+)
 create mode 120000 yet_another_large_file

# This repo is the only one with this file.
git annex list

here
|origin
||original-repo
|||web
||||bittorrent
|||||
X_X__ another_large_file
X_X__ my_large_file
X____ yet_another_large_file

# The current known git branches
git fetch original-repo  # In fact this has not changed.
git branch -av

remote: Enumerating objects: 6, done.
remote: Counting objects:  16% (1/6)
remote: Counting objects:  33% (2/6)
remote: Counting objects:  50% (3/6)
remote: Counting objects:  66% (4/6)
remote: Counting objects:  83% (5/6)
remote: Counting objects: 100% (6/6)
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects:  25% (1/4)
remote: Compressing objects:  50% (2/4)
remote: Compressing objects:  75% (3/4)
remote: Compressing objects: 100% (4/4)
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Unpacking objects:  20% (1/5)
Unpacking objects:  40% (2/5)
Unpacking objects:  60% (3/5)
Unpacking objects:  80% (4/5)
Unpacking objects: 100% (5/5)
Unpacking objects: 100% (5/5), 472 bytes | 472.00 KiB/s, done.
From ../my-repo
   72bbc39..826b9d4  git-annex  -> original-repo/git-annex
  git-annex                       ab27826 update
* main                            b53b9d6 [ahead 2] Add yet another large file
  remotes/origin/HEAD             -> origin/main
  remotes/origin/git-annex        72bbc39 update
  remotes/origin/main             f41d3c7 Add link to my_large_file
  remotes/original-repo/HEAD      -> original-repo/main
  remotes/original-repo/git-annex 826b9d4 update
  remotes/original-repo/main      f41d3c7 Add link to my_large_file

I’m now going to run git annex sync without qualification. This does a rather extreme Git and Git-Annex synchronization of this repository with the other repositories known to Git-Annex.

Read the linked documentation page. Usually, with Git, you do not push directly to remotes, other than a single source-of-truth remote, and that remote is typically a --bare remote - it does not have a working tree. If it does have a working tree, Git will, by default, warn you about this and decline, because it is so easy for the branch to get out of sync with the working tree.

However, Git-Annex sync, by default, takes a different approach - as it is so common to want to keep both Git history and the Git-Annex files in sync between repositories.

As you’ll see in the linked documentation, by default, Git-Annex applies a synchronization scheme devised by Joachim Breitner, involving a special set of branches with names starting synced/. As we’re currently on the main branch, the corresponding synced/ branch is synced/main.

In particular git annex sync, by default, does the following:

Automatically commits any changes in the working tree (configure with the annex.autocommit setting, see below).
Merges the synced/main branch (in our case) (if we have one) into main. This pulls in any Git history and files that a previous git annex sync pushed into our repository. In our case, we do not have such a branch, because no-one has done a git annex sync to us.
Fetches from each remote, and merges in any changes from other remotes.
Pushes (in our case) main directly to any remotes. Where Git prevents pushing to a branch with a working tree, it instead pushes to synced/main (in our case).
Copies any annexed files that the remote repository “wants”. “Wanting” is something you need to configure per repository. We haven’t done that yet, so no content gets copied

git annex sync

(merging original-repo/git-annex into git-annex...)
(recording state in git...)
commit
On branch main
Your branch is ahead of 'origin/main' by 2 commits.
  (use "git push" to publish your local commits)
nothing to commit, working tree clean
ok
pull original-repo
ok
pull origin
remote: Enumerating objects: 5, done.
remote: Counting objects:  20% (1/5)
remote: Counting objects:  40% (2/5)
remote: Counting objects:  60% (3/5)
remote: Counting objects:  80% (4/5)
remote: Counting objects: 100% (5/5)
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects:  33% (1/3)
remote: Compressing objects:  66% (2/3)
remote: Compressing objects: 100% (3/3)
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Unpacking objects:  33% (1/3)
Unpacking objects:  66% (2/3)
Unpacking objects: 100% (3/3)
Unpacking objects: 100% (3/3), 383 bytes | 383.00 KiB/s, done.
From /tmp/tmp_pf0j390/annex-repos/my-repo-upstream
   72bbc39..3639ecb  git-annex  -> origin/git-annex
ok
(merging origin/git-annex into git-annex...)
(recording state in git...)
push original-repo
Enumerating objects: 50, done.
Counting objects:   2% (1/46)
Counting objects:   4% (2/46)
Counting objects:   6% (3/46)
Counting objects:   8% (4/46)
Counting objects:  10% (5/46)
Counting objects:  13% (6/46)
Counting objects:  15% (7/46)
Counting objects:  17% (8/46)
Counting objects:  19% (9/46)
Counting objects:  21% (10/46)
Counting objects:  23% (11/46)
Counting objects:  26% (12/46)
Counting objects:  28% (13/46)
Counting objects:  30% (14/46)
Counting objects:  32% (15/46)
Counting objects:  34% (16/46)
Counting objects:  36% (17/46)
Counting objects:  39% (18/46)
Counting objects:  41% (19/46)
Counting objects:  43% (20/46)
Counting objects:  45% (21/46)
Counting objects:  47% (22/46)
Counting objects:  50% (23/46)
Counting objects:  52% (24/46)
Counting objects:  54% (25/46)
Counting objects:  56% (26/46)
Counting objects:  58% (27/46)
Counting objects:  60% (28/46)
Counting objects:  63% (29/46)
Counting objects:  65% (30/46)
Counting objects:  67% (31/46)
Counting objects:  69% (32/46)
Counting objects:  71% (33/46)
Counting objects:  73% (34/46)
Counting objects:  76% (35/46)
Counting objects:  78% (36/46)
Counting objects:  80% (37/46)
Counting objects:  82% (38/46)
Counting objects:  84% (39/46)
Counting objects:  86% (40/46)
Counting objects:  89% (41/46)
Counting objects:  91% (42/46)
Counting objects:  93% (43/46)
Counting objects:  95% (44/46)
Counting objects:  97% (45/46)
Counting objects: 100% (46/46)
Counting objects: 100% (46/46), done.
Delta compression using up to 4 threads
Compressing objects:   3% (1/32)
Compressing objects:   6% (2/32)
Compressing objects:   9% (3/32)
Compressing objects:  12% (4/32)
Compressing objects:  15% (5/32)
Compressing objects:  18% (6/32)
Compressing objects:  21% (7/32)
Compressing objects:  25% (8/32)
Compressing objects:  28% (9/32)
Compressing objects:  31% (10/32)
Compressing objects:  34% (11/32)
Compressing objects:  37% (12/32)
Compressing objects:  40% (13/32)
Compressing objects:  43% (14/32)
Compressing objects:  46% (15/32)
Compressing objects:  50% (16/32)
Compressing objects:  53% (17/32)
Compressing objects:  56% (18/32)
Compressing objects:  59% (19/32)
Compressing objects:  62% (20/32)
Compressing objects:  65% (21/32)
Compressing objects:  68% (22/32)
Compressing objects:  71% (23/32)
Compressing objects:  75% (24/32)
Compressing objects:  78% (25/32)
Compressing objects:  81% (26/32)
Compressing objects:  84% (27/32)
Compressing objects:  87% (28/32)
Compressing objects:  90% (29/32)
Compressing objects:  93% (30/32)
Compressing objects:  96% (31/32)
Compressing objects: 100% (32/32)
Compressing objects: 100% (32/32), done.
Writing objects:   2% (1/36)
Writing objects:   5% (2/36)
Writing objects:   8% (3/36)
Writing objects:  11% (4/36)
Writing objects:  13% (5/36)
Writing objects:  16% (6/36)
Writing objects:  19% (7/36)
Writing objects:  22% (8/36)
Writing objects:  25% (9/36)
Writing objects:  27% (10/36)
Writing objects:  30% (11/36)
Writing objects:  33% (12/36)
Writing objects:  36% (13/36)
Writing objects:  38% (14/36)
Writing objects:  41% (15/36)
Writing objects:  44% (16/36)
Writing objects:  47% (17/36)
Writing objects:  50% (18/36)
Writing objects:  52% (19/36)
Writing objects:  55% (20/36)
Writing objects:  58% (21/36)
Writing objects:  61% (22/36)
Writing objects:  63% (23/36)
Writing objects:  66% (24/36)
Writing objects:  69% (25/36)
Writing objects:  72% (26/36)
Writing objects:  75% (27/36)
Writing objects:  77% (28/36)
Writing objects:  80% (29/36)
Writing objects:  83% (30/36)
Writing objects:  86% (31/36)
Writing objects:  88% (32/36)
Writing objects:  91% (33/36)
Writing objects:  94% (34/36)
Writing objects:  97% (35/36)
Writing objects: 100% (36/36)
Writing objects: 100% (36/36), 3.25 KiB | 831.00 KiB/s, done.
Total 36 (delta 9), reused 0 (delta 0), pack-reused 0 (from 0)
To ../my-repo
 * [new branch]      main -> synced/main
 * [new branch]      git-annex -> synced/git-annex
ok
push origin
Enumerating objects: 49, done.
Counting objects:   2% (1/46)
Counting objects:   4% (2/46)
Counting objects:   6% (3/46)
Counting objects:   8% (4/46)
Counting objects:  10% (5/46)
Counting objects:  13% (6/46)
Counting objects:  15% (7/46)
Counting objects:  17% (8/46)
Counting objects:  19% (9/46)
Counting objects:  21% (10/46)
Counting objects:  23% (11/46)
Counting objects:  26% (12/46)
Counting objects:  28% (13/46)
Counting objects:  30% (14/46)
Counting objects:  32% (15/46)
Counting objects:  34% (16/46)
Counting objects:  36% (17/46)
Counting objects:  39% (18/46)
Counting objects:  41% (19/46)
Counting objects:  43% (20/46)
Counting objects:  45% (21/46)
Counting objects:  47% (22/46)
Counting objects:  50% (23/46)
Counting objects:  52% (24/46)
Counting objects:  54% (25/46)
Counting objects:  56% (26/46)
Counting objects:  58% (27/46)
Counting objects:  60% (28/46)
Counting objects:  63% (29/46)
Counting objects:  65% (30/46)
Counting objects:  67% (31/46)
Counting objects:  69% (32/46)
Counting objects:  71% (33/46)
Counting objects:  73% (34/46)
Counting objects:  76% (35/46)
Counting objects:  78% (36/46)
Counting objects:  80% (37/46)
Counting objects:  82% (38/46)
Counting objects:  84% (39/46)
Counting objects:  86% (40/46)
Counting objects:  89% (41/46)
Counting objects:  91% (42/46)
Counting objects:  93% (43/46)
Counting objects:  95% (44/46)
Counting objects:  97% (45/46)
Counting objects: 100% (46/46)
Counting objects: 100% (46/46), done.
Delta compression using up to 4 threads
Compressing objects:   3% (1/33)
Compressing objects:   6% (2/33)
Compressing objects:   9% (3/33)
Compressing objects:  12% (4/33)
Compressing objects:  15% (5/33)
Compressing objects:  18% (6/33)
Compressing objects:  21% (7/33)
Compressing objects:  24% (8/33)
Compressing objects:  27% (9/33)
Compressing objects:  30% (10/33)
Compressing objects:  33% (11/33)
Compressing objects:  36% (12/33)
Compressing objects:  39% (13/33)
Compressing objects:  42% (14/33)
Compressing objects:  45% (15/33)
Compressing objects:  48% (16/33)
Compressing objects:  51% (17/33)
Compressing objects:  54% (18/33)
Compressing objects:  57% (19/33)
Compressing objects:  60% (20/33)
Compressing objects:  63% (21/33)
Compressing objects:  66% (22/33)
Compressing objects:  69% (23/33)
Compressing objects:  72% (24/33)
Compressing objects:  75% (25/33)
Compressing objects:  78% (26/33)
Compressing objects:  81% (27/33)
Compressing objects:  84% (28/33)
Compressing objects:  87% (29/33)
Compressing objects:  90% (30/33)
Compressing objects:  93% (31/33)
Compressing objects:  96% (32/33)
Compressing objects: 100% (33/33)
Compressing objects: 100% (33/33), done.
Writing objects:   2% (1/38)
Writing objects:   5% (2/38)
Writing objects:   7% (3/38)
Writing objects:  10% (4/38)
Writing objects:  13% (5/38)
Writing objects:  15% (6/38)
Writing objects:  18% (7/38)
Writing objects:  21% (8/38)
Writing objects:  23% (9/38)
Writing objects:  26% (10/38)
Writing objects:  28% (11/38)
Writing objects:  31% (12/38)
Writing objects:  34% (13/38)
Writing objects:  36% (14/38)
Writing objects:  39% (15/38)
Writing objects:  42% (16/38)
Writing objects:  44% (17/38)
Writing objects:  47% (18/38)
Writing objects:  50% (19/38)
Writing objects:  52% (20/38)
Writing objects:  55% (21/38)
Writing objects:  57% (22/38)
Writing objects:  60% (23/38)
Writing objects:  63% (24/38)
Writing objects:  65% (25/38)
Writing objects:  68% (26/38)
Writing objects:  71% (27/38)
Writing objects:  73% (28/38)
Writing objects:  76% (29/38)
Writing objects:  78% (30/38)
Writing objects:  81% (31/38)
Writing objects:  84% (32/38)
Writing objects:  86% (33/38)
Writing objects:  89% (34/38)
Writing objects:  92% (35/38)
Writing objects:  94% (36/38)
Writing objects:  97% (37/38)
Writing objects: 100% (38/38)
Writing objects: 100% (38/38), 3.37 KiB | 861.00 KiB/s, done.
Total 38 (delta 9), reused 0 (delta 0), pack-reused 0 (from 0)
To /tmp/tmp_pf0j390/annex-repos/my-repo-upstream.git
 * [new branch]      main -> synced/main
 * [new branch]      git-annex -> synced/git-annex
ok

We note, and then ignore, that the repos to which we have synced now have synced/git-annex branches. This is not important for our purposes.

Notice too that the original repo (but not the upstream origin repo) has a new branch synced/main, because here, as is the default, Git did not allow a push directly to the current branch of a non-bare repository (a repository with a working tree).

git branch -av

  git-annex                              48c6f6a merging origin/git-annex into git-annex
* main                                   b53b9d6 Add yet another large file
  synced/main                            b53b9d6 Add yet another large file
  remotes/origin/HEAD                    -> origin/main
  remotes/origin/git-annex               48c6f6a merging origin/git-annex into git-annex
  remotes/origin/main                    b53b9d6 Add yet another large file
  remotes/origin/synced/git-annex        48c6f6a merging origin/git-annex into git-annex
  remotes/origin/synced/main             b53b9d6 Add yet another large file
  remotes/original-repo/HEAD             -> original-repo/main
  remotes/original-repo/git-annex        48c6f6a merging origin/git-annex into git-annex
  remotes/original-repo/main             f41d3c7 Add link to my_large_file
  remotes/original-repo/synced/git-annex 48c6f6a merging origin/git-annex into git-annex
  remotes/original-repo/synced/main      b53b9d6 Add yet another large file

In our case, sync does not copy the annex content to the original repository, because we haven’t told Git-Annex that that remote “wants” these files yet.

git annex list

here
|origin
||original-repo
|||web
||||bittorrent
|||||
X_X__ another_large_file
X_X__ my_large_file
X____ yet_another_large_file

Let’s configure the wanted setting for the original repository:

git annex wanted original-repo "include=*_large_file"

wanted original-repo ok
(recording state in git...)

Re-run git annex sync, now Git-Annex knows the file is wanted:

git annex sync
git annex list

git-annex sync will change default behavior in the future to send content to repositories that have preferred content configured. If you do not want this to send any content, use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent)
commit
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
ok
pull original-repo
ok
pull origin
ok
push original-repo
Enumerating objects: 4, done.
Counting objects:  25% (1/4)
Counting objects:  50% (2/4)
Counting objects:  75% (3/4)
Counting objects: 100% (4/4)
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects:  33% (1/3)
Compressing objects:  66% (2/3)
Compressing objects: 100% (3/3)
Compressing objects: 100% (3/3), done.
Writing objects:  33% (1/3)
Writing objects:  66% (2/3)
Writing objects: 100% (3/3)
Writing objects: 100% (3/3), 429 bytes | 429.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
To ../my-repo
   48c6f6a..4f4865f  git-annex -> synced/git-annex
ok
push origin
Enumerating objects: 4, done.
Counting objects:  25% (1/4)
Counting objects:  50% (2/4)
Counting objects:  75% (3/4)
Counting objects: 100% (4/4)
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects:  33% (1/3)
Compressing objects:  66% (2/3)
Compressing objects: 100% (3/3)
Compressing objects: 100% (3/3), done.
Writing objects:  33% (1/3)
Writing objects:  66% (2/3)
Writing objects: 100% (3/3)
Writing objects: 100% (3/3), 429 bytes | 429.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
To /tmp/tmp_pf0j390/annex-repos/my-repo-upstream.git
   48c6f6a..4f4865f  git-annex -> synced/git-annex
ok
here
|origin
||original-repo
|||web
||||bittorrent
|||||
X_X__ another_large_file
X_X__ my_large_file
X____ yet_another_large_file

You might want to constrain `git annex sync`#

First, you might have noted the first step in the sync above; git annex sync, by default, automatically does a commit of any changes to the working tree before it starts.

We make some staged and unstaged changes to the working tree:

# Show the last commit
git log -1

commit b53b9d6491c7800f0120c5cb427b277abdce35bb (HEAD -> main, original-repo/synced/main, origin/synced/main, origin/main, origin/HEAD, synced/main)
Author: ci <ci@example.com>
Date:   Sat Sep 5 16:27:27 2026 +0000
    Add yet another large file

# Make a new file
echo "A file" > a_file
# Stage it.
git add a_file
# Make some more unstaged changes to file.
echo "Another line" >> a_file
git status

On branch main
Your branch is up to date with 'origin/main'.
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   a_file
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   a_file

Now:

git annex sync

git-annex sync will change default behavior in the future to send content to repositories that have preferred content configured. If you do not want this to send any content, use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent)
commit
[main 2e739a7] git-annex in runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo-clone
 1 file changed, 2 insertions(+)
 create mode 100644 a_file
ok
pull original-repo
ok
pull origin
ok
push original-repo
Enumerating objects: 4, done.
Counting objects:  25% (1/4)
Counting objects:  50% (2/4)
Counting objects:  75% (3/4)
Counting objects: 100% (4/4)
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects:  50% (1/2)
Compressing objects: 100% (2/2)
Compressing objects: 100% (2/2), done.
Writing objects:  33% (1/3)
Writing objects:  66% (2/3)
Writing objects: 100% (3/3)
Writing objects: 100% (3/3), 323 bytes | 323.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
To ../my-repo
   b53b9d6..2e739a7  main -> synced/main
ok
push origin
Enumerating objects: 4, done.
Counting objects:  25% (1/4)
Counting objects:  50% (2/4)
Counting objects:  75% (3/4)
Counting objects: 100% (4/4)
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects:  50% (1/2)
Compressing objects: 100% (2/2)
Compressing objects: 100% (2/2), done.
Writing objects:  33% (1/3)
Writing objects:  66% (2/3)
Writing objects: 100% (3/3)
Writing objects: 100% (3/3), 323 bytes | 323.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
To /tmp/tmp_pf0j390/annex-repos/my-repo-upstream.git
   b53b9d6..2e739a7  main -> synced/main
ok

# Sync committed the staged and unstaged changes.
git status

On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean

# We have a new automated commit.
git log -1

commit 2e739a7da803e87bec17f53050e0ee2d38530aa9 (HEAD -> main, original-repo/synced/main, origin/synced/main, origin/main, origin/HEAD, synced/main)
Author: ci <ci@example.com>
Date:   Sat Sep 5 16:27:30 2026 +0000
    git-annex in runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo-clone

You will see that sync made a new commit for you. You might want that, but I do not, and I turn it off thus:

# Don't do automatic worktree commits.
git annex config --set annex.autocommit false

annex.autocommit false ok
(recording state in git...)

Now back to the merges. As you can imagine, whenever you push directly into another branch, with a history that is not fast-forward, you can get merge conflicts. That means that, if you are not careful, git annex sync will generate merge conflicts, either pulling into our repo, or pushing into another. As usual, merge conflicts can be obscure and difficult to resolve. These forced merges also make it harder to think about what Git is doing, and for many of us, that makes the process more obscure. Combining the possibility of merge conflicts with Git-Annex as an extra layer on top of Git, makes things even more difficult.

Therefore, I suggest that you, like me, tell Git-Annex not to apply this synced merge strategy, and instead, do Git pushes manually.

We make another commit just for the illustration:

echo "Yet another another" > yet_yet_another_large_file
git annex add yet_yet_another_large_file
git commit -m "Another another large file"

add yet_yet_another_large_file
100%  20 B             81 KiB/s 0s
ok
(recording state in git...)
[main 81b577a] Another another large file
 1 file changed, 1 insertion(+)
 create mode 120000 yet_yet_another_large_file

You can turn off Git-Annex’ synced merge behavior for your repository with:

# Sync content only, not Git branches.
git annex config --set annex.synconlyannex true

annex.synconlyannex true ok
(recording state in git...)

Now:

git annex sync

git-annex sync will change default behavior in the future to send content to repositories that have preferred content configured. If you do not want this to send any content, use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent)
pull original-repo
From ../my-repo
 * branch            git-annex  -> FETCH_HEAD
 * branch            synced/git-annex -> FETCH_HEAD
ok
pull origin
From /tmp/tmp_pf0j390/annex-repos/my-repo-upstream
 * branch            git-annex  -> FETCH_HEAD
 * branch            synced/git-annex -> FETCH_HEAD
ok
copy another_large_file (to origin...)
100%  11 B              5 KiB/s 0s
ok
copy my_large_file (to origin...)
100%  23 B            237 KiB/s 0s
ok
copy yet_another_large_file (to original-repo...)
100%  18 B              9 KiB/s 0s
ok
copy yet_another_large_file (to origin...)
100%  18 B            195 KiB/s 0s
ok
copy yet_yet_another_large_file (to original-repo...)
100%  20 B            244 KiB/s 0s
ok
copy yet_yet_another_large_file (to origin...)
100%  20 B            167 KiB/s 0s
ok
pull original-repo
From ../my-repo
 * branch            git-annex  -> FETCH_HEAD
 * branch            synced/git-annex -> FETCH_HEAD
ok
pull origin
From /tmp/tmp_pf0j390/annex-repos/my-repo-upstream
 * branch            git-annex  -> FETCH_HEAD
 * branch            synced/git-annex -> FETCH_HEAD
ok
(recording state in git...)
push original-repo
Enumerating objects: 35, done.
Counting objects:   2% (1/35)
Counting objects:   5% (2/35)
Counting objects:   8% (3/35)
Counting objects:  11% (4/35)
Counting objects:  14% (5/35)
Counting objects:  17% (6/35)
Counting objects:  20% (7/35)
Counting objects:  22% (8/35)
Counting objects:  25% (9/35)
Counting objects:  28% (10/35)
Counting objects:  31% (11/35)
Counting objects:  34% (12/35)
Counting objects:  37% (13/35)
Counting objects:  40% (14/35)
Counting objects:  42% (15/35)
Counting objects:  45% (16/35)
Counting objects:  48% (17/35)
Counting objects:  51% (18/35)
Counting objects:  54% (19/35)
Counting objects:  57% (20/35)
Counting objects:  60% (21/35)
Counting objects:  62% (22/35)
Counting objects:  65% (23/35)
Counting objects:  68% (24/35)
Counting objects:  71% (25/35)
Counting objects:  74% (26/35)
Counting objects:  77% (27/35)
Counting objects:  80% (28/35)
Counting objects:  82% (29/35)
Counting objects:  85% (30/35)
Counting objects:  88% (31/35)
Counting objects:  91% (32/35)
Counting objects:  94% (33/35)
Counting objects:  97% (34/35)
Counting objects: 100% (35/35)
Counting objects: 100% (35/35), done.
Delta compression using up to 4 threads
Compressing objects:   5% (1/19)
Compressing objects:  10% (2/19)
Compressing objects:  15% (3/19)
Compressing objects:  21% (4/19)
Compressing objects:  26% (5/19)
Compressing objects:  31% (6/19)
Compressing objects:  36% (7/19)
Compressing objects:  42% (8/19)
Compressing objects:  47% (9/19)
Compressing objects:  52% (10/19)
Compressing objects:  57% (11/19)
Compressing objects:  63% (12/19)
Compressing objects:  68% (13/19)
Compressing objects:  73% (14/19)
Compressing objects:  78% (15/19)
Compressing objects:  84% (16/19)
Compressing objects:  89% (17/19)
Compressing objects:  94% (18/19)
Compressing objects: 100% (19/19)
Compressing objects: 100% (19/19), done.
Writing objects:   4% (1/25)
Writing objects:   8% (2/25)
Writing objects:  12% (3/25)
Writing objects:  16% (4/25)
Writing objects:  20% (5/25)
Writing objects:  24% (6/25)
Writing objects:  28% (7/25)
Writing objects:  36% (9/25)
Writing objects:  40% (10/25)
Writing objects:  44% (11/25)
Writing objects:  48% (12/25)
Writing objects:  52% (13/25)
Writing objects:  56% (14/25)
Writing objects:  60% (15/25)
Writing objects:  64% (16/25)
Writing objects:  68% (17/25)
Writing objects:  72% (18/25)
Writing objects:  76% (19/25)
Writing objects:  80% (20/25)
Writing objects:  84% (21/25)
Writing objects:  88% (22/25)
Writing objects:  92% (23/25)
Writing objects:  96% (24/25)
Writing objects: 100% (25/25)
Writing objects: 100% (25/25), 1.82 KiB | 929.00 KiB/s, done.
Total 25 (delta 10), reused 0 (delta 0), pack-reused 0 (from 0)
To ../my-repo
   4f4865f..649903b  git-annex -> synced/git-annex
ok
push origin
Enumerating objects: 35, done.
Counting objects:   2% (1/35)
Counting objects:   5% (2/35)
Counting objects:   8% (3/35)
Counting objects:  11% (4/35)
Counting objects:  14% (5/35)
Counting objects:  17% (6/35)
Counting objects:  20% (7/35)
Counting objects:  22% (8/35)
Counting objects:  25% (9/35)
Counting objects:  28% (10/35)
Counting objects:  31% (11/35)
Counting objects:  34% (12/35)
Counting objects:  37% (13/35)
Counting objects:  40% (14/35)
Counting objects:  42% (15/35)
Counting objects:  45% (16/35)
Counting objects:  48% (17/35)
Counting objects:  51% (18/35)
Counting objects:  54% (19/35)
Counting objects:  57% (20/35)
Counting objects:  60% (21/35)
Counting objects:  62% (22/35)
Counting objects:  65% (23/35)
Counting objects:  68% (24/35)
Counting objects:  71% (25/35)
Counting objects:  74% (26/35)
Counting objects:  77% (27/35)
Counting objects:  80% (28/35)
Counting objects:  82% (29/35)
Counting objects:  85% (30/35)
Counting objects:  88% (31/35)
Counting objects:  91% (32/35)
Counting objects:  94% (33/35)
Counting objects:  97% (34/35)
Counting objects: 100% (35/35)
Counting objects: 100% (35/35), done.
Delta compression using up to 4 threads
Compressing objects:   5% (1/19)
Compressing objects:  10% (2/19)
Compressing objects:  15% (3/19)
Compressing objects:  21% (4/19)
Compressing objects:  26% (5/19)
Compressing objects:  31% (6/19)
Compressing objects:  36% (7/19)
Compressing objects:  42% (8/19)
Compressing objects:  47% (9/19)
Compressing objects:  52% (10/19)
Compressing objects:  57% (11/19)
Compressing objects:  63% (12/19)
Compressing objects:  68% (13/19)
Compressing objects:  73% (14/19)
Compressing objects:  78% (15/19)
Compressing objects:  84% (16/19)
Compressing objects:  89% (17/19)
Compressing objects:  94% (18/19)
Compressing objects: 100% (19/19)
Compressing objects: 100% (19/19), done.
Writing objects:   4% (1/25)
Writing objects:   8% (2/25)
Writing objects:  12% (3/25)
Writing objects:  16% (4/25)
Writing objects:  20% (5/25)
Writing objects:  24% (6/25)
Writing objects:  28% (7/25)
Writing objects:  36% (9/25)
Writing objects:  40% (10/25)
Writing objects:  44% (11/25)
Writing objects:  48% (12/25)
Writing objects:  52% (13/25)
Writing objects:  56% (14/25)
Writing objects:  60% (15/25)
Writing objects:  64% (16/25)
Writing objects:  68% (17/25)
Writing objects:  72% (18/25)
Writing objects:  76% (19/25)
Writing objects:  80% (20/25)
Writing objects:  84% (21/25)
Writing objects:  88% (22/25)
Writing objects:  92% (23/25)
Writing objects:  96% (24/25)
Writing objects: 100% (25/25)
Writing objects: 100% (25/25), 1.82 KiB | 929.00 KiB/s, done.
Total 25 (delta 10), reused 0 (delta 0), pack-reused 0 (from 0)
To /tmp/tmp_pf0j390/annex-repos/my-repo-upstream.git
   4f4865f..649903b  git-annex -> synced/git-annex
ok

Notice that the commit didn’t get sent to the remotes, but the large file did:

git branch -av

  git-annex                              649903b update
* main                                   81b577a [ahead 1] Another another large file
  synced/main                            2e739a7 git-annex in runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo-clone
  remotes/origin/HEAD                    -> origin/main
  remotes/origin/git-annex               649903b update
  remotes/origin/main                    2e739a7 git-annex in runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo-clone
  remotes/origin/synced/git-annex        649903b update
  remotes/origin/synced/main             2e739a7 git-annex in runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo-clone
  remotes/original-repo/HEAD             -> original-repo/main
  remotes/original-repo/git-annex        649903b update
  remotes/original-repo/main             f41d3c7 Add link to my_large_file
  remotes/original-repo/synced/git-annex 649903b update
  remotes/original-repo/synced/main      2e739a7 git-annex in runner@runnervmejwal:/tmp/tmp_pf0j390/annex-repos/my-repo-clone

git annex list

here
|origin
||original-repo
|||web
||||bittorrent
|||||
XXX__ another_large_file
XXX__ my_large_file
XXX__ yet_another_large_file
XXX__ yet_yet_another_large_file

We send the commit manually:

git push origin main

Enumerating objects: 4, done.
Counting objects:  25% (1/4)
Counting objects:  50% (2/4)
Counting objects:  75% (3/4)
Counting objects: 100% (4/4)
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects:  33% (1/3)
Compressing objects:  66% (2/3)
Compressing objects: 100% (3/3)
Compressing objects: 100% (3/3), done.
Writing objects:  33% (1/3)
Writing objects:  66% (2/3)
Writing objects: 100% (3/3)
Writing objects: 100% (3/3), 377 bytes | 377.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
To /tmp/tmp_pf0j390/annex-repos/my-repo-upstream.git
   2e739a7..81b577a  main -> main

The special remotes#

As you have seen, Git-Annex can use Git repositories on filesystems as storage for GAOs — in the repository .git/annex directory.

But Git-Annex can also have special remotes, that have no necessary relationship to Git repositories. For example, they can work with directories on file-sharing systems such as Dropbox and Google Drive.

You create special remotes with the git annex initremote command.

I say special remotes have no necessary relationship to Git repositories, but you can also label a Git repository as a special remote, using the --type git flag to git annex initremote. Read the note below for more detail.

Note

By the way, I hate to be confusing, but you will have seen above (with ../my-repo), that Git-Annex can identify repositories as capable of storing annex files — a sort of special-remote-in-practice — and it can do this when it has some way to read the .git/config of the repository, as it can, for example when the repository is on the local file-system. It uses .git/config to work out what the UUID is of the remote repository. Adding the repository explicitly as a special remote, with git annex initrepo --type git, stores the UUID in the git-annex branch, along with some useful location (such as SSH path, filesystem path, etc). When other repositories clone or otherwise update their git-annex branch with this information, they can see the stored UUID, and therefore use that remote as a Git-Annex special remote.

You can enable known and already-configured special remotes with git annex enableremote <remotename>. This tells the local repository that you want to be able to use that remote to fetch and store annex files, with sync and other commands.

I don’t cover special remotes further here, but see the Git-Annex page on special remotes, and the doc page on special remotes.

Which files go where#

Consider setting annex.largefiles entries with git annex config, or in your .gitattributes file, to tell Git-Annex which files it should handle (as GAOs) and which Git should handle. See the annex.largefiles page for details. Note that annex.largefiles just identifies files that Git-Annex should handle. You can use values for annex.largefiles to make Git-Annex always operate on files larger than a particular size, but you can also use that setting to configure Git / Git-Annex to select files by path name.

Consider using git annex wanted commands (see above) to tell Git-Annex which remotes should house which files. These rules get stored in the git-annex branch. See the Git-Annex wanted page for more.

Whither Git-Annex#

That was a tour of the basics. You might now want to have a look at the primary Git-Annex pages.

Git-Annex from scratch

Contents

Git-Annex from scratch#

Git-Annex objects (GAOs)#

Git-Annex pointers#

Git-Annex remotes#

Git-Annex metadata#

Walkthrough#

Start with standard Git#

Add Git-Annex features with git annex init#

Adding files with git annex add#

Clones and Git-Annex#

Fetching files with git annex get#

Copying annex files with git annex copy#

The awe-inspiring power of git annex sync#

You might want to constrain git annex sync#

The special remotes#

Which files go where#

Whither Git-Annex#

Add Git-Annex features with `git annex init`#

Adding files with `git annex add`#

Fetching files with `git annex get`#

Copying annex files with `git annex copy`#

The awe-inspiring power of `git annex sync`#

You might want to constrain `git annex sync`#