diff options
| -rw-r--r-- | content/weblog/2020-12-31_minimal-git-server/index.md | 431 |
1 files changed, 431 insertions, 0 deletions
diff --git a/content/weblog/2020-12-31_minimal-git-server/index.md b/content/weblog/2020-12-31_minimal-git-server/index.md new file mode 100644 index 0000000..9f55c20 --- /dev/null +++ b/content/weblog/2020-12-31_minimal-git-server/index.md @@ -0,0 +1,431 @@ ++++ +title = "Simple and Minimal Git Server Setup" +date = 2020-12-31T16:39:56Z ++++ + +The New Year is just around the corner, but there's still a little time to write +one last post before the year ends. This time I want to write about my +experience setting up a git server. I had already set up before a git server +using a free and open source kind of clone of Github called Gitea, but there's a +better way to setup a git server, especially if you just want to setup a +personal Git server. + +<!-- more --> + +About a year ago I setup a git server using Gitea[^1] server together with a +CI/CD system called Drone[^2]. Back then it made sense, because I was working on +some projects with a couple of friends, and I needed easy access for them to be +able to work together with me. However, time went on, we stopped working on +those projects, and I left that git server running for my own needs, i.e. to +store the source of my website and deploy it automatically on each push to +master. + +It worked fine all around, but it was too bloated for my tastes, and I didn't +really need most of the features that both Gitea and Drone had to offer. The +containerized way of working using Docker containers for the servers themselves, +and for the post-push hooks used, in my opinion, too many resources. And it +turns out I was right. + +I wanted to get rid of all of that, one, because I didn't really needed all of +that; two, because I like minimalism, and try to avoid web interfaces and their +clunky ways of working as much as possible; and three, because I wanted to get +rid of the VPS in which they were hosted, and wanted to have it all on one VPS, +the one where my website is being hosted, among many other things, such as my +email server. + +I already knew that there were some light-weight web front-ends for git, such as +cgit[^3], or even git's own web interface; I didn't know what exactly they had +though, so I started digging deeper into git itself, by reading the +documentation, and looking in the internet for tutorials and explanations of how +cgit and similar tools worked. + +After learning more about git, and finding myself with more time than usual all +thanks to COVID-19[^4] and the government, I finally decided to just trying out +setting up my minimal Git server. + +## Prelude, or some things to know before setting up a git server + +Before trying to setup a git server this way, you should know some basic stuff +about how git works. I didn't know a lot of these details when I embarked myself +upon this venture, so I ended having to read a good deal of information on the +way git works, which one of the reasons I really enjoyed this. Turns out git is +a much better piece of software than I thought before, and I already had a +really high opinion of it. + +First, all of the git hosting solutions out there, Github, Gitlab, Gitea etc. +are using none other that git, and some other like gitolite, as their main +backend. Every time you push something to, say Github, be it by ssh or http, +git gets called on their server. Github just shows you what git is telling them. + +Second, you don't really need a web interface to have a git server. The web +interface is just a nice way of showing your repositories to other people. A +computer with ssh access is more than enough, if all you want to do is host your +repositories. + +Third, there are, in practice, two types of repositories: + +1. Standard repositories that contain all the information and actual files in the + `.git` directory, together with a "working directory" at the root, which is the + representation of the state of your projects in the ref you are currently, and + where you actually, well, work on your files by editing them. This are not the + actual files stored by git. +2. Bare repositories, which forgo the "working directory" and just contain all + the git files and information at the root of the directory. This types of + repositories are the ones meant for the server side of things. + +## Setting up the git backend + +Now that you know a little bit more about how git works (or maybe you already +knew that), it's time to set up the actual git server. All you need is a +computer with access to the internet, preferably with a static IP (like a VPS), +and optionally a domain name, just to make things prettier and easier. To guide +myself through this process, I mostly followed the Arch Linux wiki guide on +it[^5], and the official Git book chapter on Git on the server[^6]. + +First thing that you need to do, is setup ssh access. This is pretty easy to do, +you most probably already have it setup, and it is out the scope of this +tutorial, so I am not going to explain how to do it in this post. + +Next, you need to make a user specially for git. This is quite important, since +you don't want people with access (even just read access) to your repositories +to have access to other parts of your server. For simplicity, we'll just call it +git. + +Each Linux distro (or Unix-like OS), has its own defaults, and sometimes even +alternate ways to create users. For simplicity's sake I decided to use Debian's +defaults when creating the git user: + +``` +# useradd git +# passwd git +``` + +These two commands, at least in Debian Buster, should create a normal user with +a home directory at `/home/git/` and prompt you for a password for the git user. +Some people like to have the git home directory in a location such as +`/srv/git/`, but I just decided to go with the defaults for simplicity's sake. + +Now we need to setup some things for the git user. First we should switch to it: + +``` +$ su -l git +``` + +And once we are the git user, we should make a `.ssh` folder with a +`authorized_keys` file, which should contain your public key(s), and also the +keys of any people you plan on having write (push) access to your repositories. +Keep in mind, that any people with push access, will have push access to **all** +of the repositories under the git user. + +Now we should think about where we are going to store the git repositories. You +could just store them at the root of the git home directory. However, I decided +to store them in a separate folder, since I also wanted to keep some (mainly +uninteresting) repositories private. So I created two folders `private`, and +`public`. Public, as its name implies, is going to be the directory with the +repositories that are going to be shown on the web interface for all the people +to see. I could have also created a separate web interface protected by password +for the `private` repositories, but the web interface is mainly just for people +on the internet to take a glance at my repositories. + +Now for a test, you can create your first repository under the public folder, +and try pushing into it. So we should cd into our `public` directory (or +whatever directory you plan to store your repositories at) and init a **bare** +repository. By convention bare repository names end with a `.git`, so it should +look something like this: + +``` +$ cd ~/public +$ git init --bare my-repo.git +``` + +Any time that you want to add a repo to your server, you'll basically need to +login to your git user and repeat the steps above. Or, if you already have a +non-empty server, you could clone it this way: + +``` +git clone --bare <path-or-url-to-my-repo> my-repo.git +``` + +Now you can push from your local computer to that directory using ssh this way: + +``` +$ git remote set-url origin git@your-domain-or-ip:public/my-repo.git +$ git push -u origin master +``` + +Or clone it, once you have something there: + +``` +$ git clone git@your-domain-or-ip:public/my-repo.git +``` + +Now you have working git server. It's that simple! However, right now there's +now read-only access for other people to clone your repos. There's two ways you +could solve that. You could use git's own smart HTTP, or the git daemon. In my +case I didn't end up using git's smart HTTP, since I used cgit's functionality +for that, but if you are interested in any of them, you can check out the +official git book chapter that I mentioned earlier[^6]. + +Before we go further, though, we need to harden our git user a little bit, +especially if you intend on giving other people push access. Right now, anybody +with push access, basically has ssh access to your git user, since basically +when you are pushing using this method, the only thing that is happening is that +git is ssh'ing into your server and executing git (the one on your server) in +your server. That's a big no-no. For that, we are going to change the standard +login shell of git, to one that will allow git to function properly, but won't +allow people to login to an full-fledged interactive shell into your git user +using ssh. + +Fortunately, git provides use with a limited shell for this specific case, +called the git-shell. First, we need to know where it is located: + +``` +$ which git-shell +/usr/bin/git-shell +``` + +Now that we know its actual location, we need to change the default shell for +our git user: + +``` +# chsh -s /usr/bin/git-shell git +``` + +Now you might think, well, how do I log into the git user to add/edit +repositories? Just log into your normal user, and then use `su` to login with a +shell like sh, bash, or zsh: + +``` +$ su -s /bin/bash git +``` + +## Setting up the web interface + +Git provides its own web interface, which, if you want the minimalest of the +minimalest, should be fine. I, however, decided to use cgit[^3], since I wanted +a little more flexibility and more room for configuration. It's even the +preferred web interface for the kernel.org git repositories. + +To use it, we'll need not only it, but also a reverse proxy for it to work with +it. My preferred program for that is nginx, I have used it for several years, +and it love it for its ease of use and its performance. You could also use other +software, like Apache, if that's what you prefer. + +Even though I'm using Debian on my VPS, I mostly followed the Arch Linux Wiki +article on cgit[^7]. If you're not yet familiar with the Arch Wiki, you should +definitely check it out. It's arguably the best resource on Linux out there. + +So basically we need the following packages: + +``` +# apt install nginx fcgiwrap cgit +``` + +After that, we should go on and configure nginx. I won't go into big detail on +how to configure nginx itself, but the configuration file for cgit should be +located here `/etc/nginx/sites-available/cgit` and look something like this: + +``` +server { + listen 80; + listen [::]:80; + server_name <your-git-domain>; + return 301 https://$host$request_uri; +} + +server { + listen 443 ssl; + + server_name <your-git-domain>; + ssl_certificate <your-ssl-cert-path>; + ssl_certificate_key <your-ssl-cert-key-path>; + + root /usr/share/cgit; + try_files $uri @cgit; + + location @cgit { + include fastcgi_params; + fastcgi_param SCRIPT_FILENAME /usr/lib/cgit/cgit.cgi; + fastcgi_param PATH_INFO $uri; + fastcgi_param QUERY_STRING $args; + fastcgi_param HTTP_HOST $server_name; + fastcgi_pass unix:/run/fcgiwrap.socket; + } +} +``` + +This configuration assumes that you have setup an SSL certificate. You can +configure it just use plain HTTP, but I really recommend you get an SSL +certificate. + +Now that we have nginx configured we should start and enable both nginx and +fcgiwrap. + +At this point, you should be able to see cgit by going to the domain or url that +you pointed at in the nginx config. However, it probably won't be showing you +your repositories just yet, and that's because we need configure cgit, in part +so that we can tell it where to look for our repos. + +I won't go into much detail on how to configure cgit, but I'll show you my +configuration file, so you have an idea of how to configure yours. This file +should be located at `/etc/cgitrc`: + +``` +# +# cgit config +# see cgitrc(5) for details + +css=/cgit.css +logo=/cgit.png +virtual-root=/ +max-repodesc-length=100 +repository-sort=age + +root-title=Yaroslav's git repositories +root-desc=Public repos for some of my projects, and mirrors of other projects. + +# Possible readme files for about page +readme=:README.md +readme=:README.txt +readme=:README.rst +readme=:README + +# Syntax highlighting +source-filter=/usr/local/lib/cgit/filters/syntax-highlighting.py + +# About page rendering script +about-filter=/usr/local/lib/cgit/filters/about-formatting.sh + +# Some common mimetypes for serving files raw +mimetype.gif=image/gif +mimetype.html=text/html +mimetype.jpg=image/jpeg +mimetype.jpeg=image/jpeg +mimetype.pdf=application/pdf +mimetype.png=image/png +mimetype.svg=image/svg+xml + +# Set of snapshot formats for people to download +snapshots=tar.gz zip + +# Git configuration +enable-http-clone=1 +enable-commit-graph=1 +remove-suffix=1 +section-from-path=1 +scan-path=/home/git/public +clone-prefix=https://git.yaroslavps.com git://git.yaroslavps.com +``` + +The most important bits here are: + +``` +virtual-root=/ +enable-http-clone=1 +scan-path=/home/git/public +clone-prefix=https://git.yaroslavps.com +``` + +You can read more about cgit's configuration in the Arch Wiki article I +mentioned earlier[^8]. I'll just say, that you should keep in mind that the +order of the configuration variables is important. If something is not working +as it should, its probably because you put some variable before another one, and +their order should be changed. This gave a big headache when I was trying to +configure cgit for myself. + +One last thing worth mentioning, regarding cgit, is that you can configure each +repo for cgit by adding a `cgitrc` file inside the bare repository in your +server. You can then add some more information about it like this: + +``` +owner=Yaroslav de la Peña Smirnov <yps@yaroslavps.com> +desc=Personal site and blog. +``` + +For more information, read cgitrc's manual: + +``` +$ man 5 cgitrc +``` + +## Conclusion + +This is one possible way of having your own hosting solution for git +repositories, in a very minimal way. Since the cgit frontend is not running +constantly, it's written in C, and is very minimal, it barely consumes any +resources. This is also a great way to be more independent of centralized, +proprietary services like GitHub. Of course, services like those are always +great for visibility, but you could always just have mirror of your open source +projects on them that points to your personal server. + +There many other great things that you can do with a setup like this, like +setting up git hooks, without having to set up a complex and complicated CI/CD +system, for simple tasks, like in my case, pushing, generating and deploying my +site on each push to master. For an example of a post-receive you can checkout +mine here: +[https://git.yaroslavps.com/yaroslavps.com/tree/post-receive](https://git.yaroslavps.com/yaroslavps.com/tree/post-receive) + +For a little more information on git, I recommend you to read the following +manual pages: + +``` +$ man git +$ man git-init +$ man git-clone +$ man git-config +$ man githooks +$ man git-send-email +$ man git-format-patch +``` + +The last are especially useful if you want to know how to collaborate with other +people on other projects, or want other people to collaborate on yours, without +having to resort to the centralized way of doing things of GitHub and similar +sites. Git is especially well suited for collaborating through email, which is +in and of itself a decentralized service. There's a guide on collaborating +through email using git[^8], written by the great Drew DeVault, which is, in my +opinion, one of the best FOSS programmers out there. I recommend also to check +out his blog[^9], which I must say is much more interesting than mine. + + +## A little unrelated note + +As I write this post, this strange year is (finally) coming to an end. It wasn't +a completely terrible year per se, but it did present us with a lot of nasty +surprises. I have quite a lot of personal plans for this coming 2021, some of +which involve this site. Some of them I'll keep a secret for now, but most +importantly I want to diversify my writing a little bit more. I kind of already +did that this year, with my "recipe book"[^10], but I also want to start writing +about some other things on my weblog. So far it's been mostly about software +development and computer-related stuff, but there are many other things not +related to that on my mind, which I'd like to write about. + +Here's to a New Year, that's not as Orwellian as this one, and may you +accomplish as much as you set yourself to accomplish. + +Happy New Year and Merry Christmas[^11] to all! + +[^1]: [https://gitea.com/](https://gitea.com/) + +[^2]: [https://www.drone.io/](https://www.drone.io/) + +[^3]: [https://git.zx2c4.com/cgit/about/](https://git.zx2c4.com/cgit/about/) + +[^4]: I'll write a little bit about my experience with the corona, and my + thoughts on it next year. + +[^5]: [https://wiki.archlinux.org/index.php/Git_server](https://wiki.archlinux.org/index.php/Git_server) + +[^6]: + [https://git-scm.com/book/en/v2/Git-on-the-Server-Getting-Git-on-a-Server](https://git-scm.com/book/en/v2/Git-on-the-Server-Getting-Git-on-a-Server) + +[^7]: [https://wiki.archlinux.org/index.php/Cgit](https://wiki.archlinux.org/index.php/Cgit) + +[^8]: [https://git-send-email.io/](https://git-send-email.io/) + +[^9]: [https://drewdevault.com/](https://drewdevault.com/) + +[^10]: [https://www.yaroslavps.com/food/](https://www.yaroslavps.com/food/) + +[^11]: I know, a little bit late for Catholic or Protestant folks out there. But + just in time for (Russian) Orthodox Christmas ;) |