As of about two days ago, my self-hosted Git forge runs Gitlab. This is great - it's a big quality of life improvement over Gitea, even if it is more complex to install and maintain.

One feature of Gitea that I made considerable use of was pull mirroring . While Gitlab does also support this, it's only available in the premium and ultimate editions , not the community edition that I use. ¹

So, as a way around this, I decided to setup my own simple pull mirroring - I'll describe the how that works and rationale behind some of the decisions I made here.

1. Clone a copy of the source repository

Git provides a useful useful flag to clone a repository with the explicit intent of mirroring - that's --mirror . From Git's docs on the topic :

[ --mirror sets] up a mirror of the source repository. This implies --bare . Compared to --bare , --mirror not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.) and sets up a refspec configuration such that all these refs are overwritten by a git remote update in the target repository.

I'd be lying if I said I understood completely what this meant and what its implications were, but it seems useful so we'll roll with it. Running git clone --mirror <src url> will get you a copy of your repository checked out on disk without a worktree and with all the references we need from the source repository pointing at the right things.

2. Push the clone to the destination repository

2a. Pushing using git push --mirror

Just as there's a --mirror flag for git clone , there's a --mirror flag for git push too. If we run git push --mirror <dest url> ...

Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 280 bytes | 280.00 KiB/s, done.
Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
To <dest url>
 ! [remote rejected] refs/merge-requests/12/head -> refs/merge-requests/12/head (deny updating a hidden ref)
 ! [remote rejected] refs/merge-requests/12/merge -> refs/merge-requests/12/merge (deny updating a hidden ref)
 ! [remote rejected] refs/pipelines/34499 -> refs/pipelines/34499 (deny updating a hidden ref)
error: failed to push some refs to '<dest url>'
Ah, whoops.
In this example, the source repository is in another Gitlab instance. For every merge request and every pipeline run, Gitlab creates "hidden refs" that points to relevant commits for that thing. However, our Gitlab instance will reject those refs when we try to push them as they're not really supposed to be manually updated.
We can't control what is and isn't pushed when we use --mirror, so we'll have to look at other options.
2b. Pushing using custom refspecs
This article suggests a different method to push our repository to the destination. If we use git push --prune along with some specifically crafted refspecs, we can tell Git to only push refs pertaining to branches and tags to the remote, along with removing any from the remote that don't exist locally.
The full command to do that is git push --prune <dest url> +refs/remotes/origin/*:refs/heads/* +refs/tags/*:refs/tags/*, which kind of works...?
remote: GitLab: The default branch of a project cannot be deleted.
To <dest url>
 ! [remote rejected] dev (pre-receive hook declined)
 ! [remote rejected] m2/leaderboard1 (pre-receive hook declined)
 ! [remote rejected] m2/nav-bar (pre-receive hook declined)
 ! [remote rejected] m2/rating (pre-receive hook declined)
 ! [remote rejected] m3/darkmode (pre-receive hook declined)
 ! [remote rejected] m3/explore-folders (pre-receive hook declined)
 ! [remote rejected] m3/recently-listened (pre-receive hook declined)
 ! [remote rejected] m3/recently-reviewed (pre-receive hook declined)
 ! [remote rejected] main (pre-receive hook declined)
 ! [remote rejected] s3/discovery (pre-receive hook declined)
 ! [remote rejected] s3/folder (pre-receive hook declined)
 ! [remote rejected] s3/leaderboard (pre-receive hook declined)
 ! [remote rejected] s3/profile-page (pre-receive hook declined)
 ! [remote rejected] s3/rating (pre-receive hook declined)
 ! [remote rejected] s3/want-to-listen (pre-receive hook declined)
error: failed to push some refs to '<dest url>'
Okay, it doesn't work at all.
It would appear that git push --prune works by nuking everything on the remote (including the default branch) then pushing everything we have locally to the remote - the net result being removing any branches that aren't present on the remote. It would also appear that Gitlab refuses to let this happen since it involves deleting the default branch, which you cannot do.
2c. Clobbering the local repo before we push
Okay, fine - if we can't easily use custom refspecs and git push --mirror looks like it will work in the absence of the merge-requests and pipelines refs, why not just delete those refs locally and then run git push --mirror?
It's fairly straight forwards to list these refs using git show-ref and grep:
$ git show-ref | grep -P 'refs\/(?:merge-requests|pipelines)\/.*'
e29bf226e38928adfc38cd417b53326234748861 refs/merge-requests/12/head
8adb25a0454080e66097129da60b14f8a1c0cf16 refs/merge-requests/12/merge
354deca470d398b9c49a031c070b1b8bd86d6014 refs/pipelines/34499
These can be deleted with git update-ref. Since we're making a batch update, we can make use of update-ref's abiilty to take commands from stdin, meaning our final magic ref deletion command looks like this:
git show-ref | grep -P 'refs\/(?:merge-requests|pipelines)\/.*' | awk '{print "delete " $2}' | git update-ref --stdin
Once run, we can finally push our new, clobbered repo to its destination:
$ git push --mirror <dest url>
Everything up-to-date
Hurrah!
A small Go program
To deploy this, I encoded the clone-clobber-push procecure in a small Go program that was also outfitted with some health checks and config file loading. This got dropped on my server, along with a Cron job to run it every 15 minutes (*/15 * * * *).
You can grab the program here: https://git.tdpain.net/-/snippets/1
It works pleasingly well, and I have something that screams at me if it doesn't. Perfect.
Update 2024-04-16: About 6 hours after this was initially published, I discovered the existence of refs/pipelines refs through repeated and intermittent crashes of the script. The article has been updated to reflect this.
Before going to the trouble of setting up my own mirroring stuff as described in this article, I did consider just paying for Gitlab, but at USD$29/user/month, it was way out of my student budget.↩
2a. Pushing using git push --mirror
2b. Pushing using custom refspecs
2c. Clobbering the local repo before we push
A small Go program
        © 2019 - 2025 Abigail Pain. Written content licensed as CC BY-NC-ND 4.0.

        No AI was used in the creation of this content.