New post: Minimal git server

1 files changed, 431 insertions, 0 deletions

diff --git a/content/weblog/2020-12-31_minimal-git-server/index.md b/content/weblog/2020-12-31_minimal-git-server/index.md
new file mode 100644
index 0000000..9f55c20
--- /dev/null
+++ b/content/weblog/2020-12-31_minimal-git-server/index.md
@@ -0,0 +1,431 @@
++++
+title = "Simple and Minimal Git Server Setup"
+date = 2020-12-31T16:39:56Z
++++
+
+The New Year is just around the corner, but there's still a little time to write
+one last post before the year ends. This time I want to write about my
+experience setting up a git server. I had already set up before a git server
+using a free and open source kind of clone of Github called Gitea, but there's a
+better way to setup a git server, especially if you just want to setup a
+personal Git server.
+
+<!-- more -->
+
+About a year ago I setup a git server using Gitea[^1] server together with a
+CI/CD system called Drone[^2]. Back then it made sense, because I was working on
+some projects with a couple of friends, and I needed easy access for them to be
+able to work together with me. However, time went on, we stopped working on
+those projects, and I left that git server running for my own needs, i.e. to
+store the source of my website and deploy it automatically on each push to
+master. 
+
+It worked fine all around, but it was too bloated for my tastes, and I didn't
+really need most of the features that both Gitea and Drone had to offer. The
+containerized way of working using Docker containers for the servers themselves,
+and for the post-push hooks used, in my opinion, too many resources. And it
+turns out I was right.
+
+I wanted to get rid of all of that, one, because I didn't really needed all of
+that; two, because I like minimalism, and try to avoid web interfaces and their
+clunky ways of working as much as possible; and three, because I wanted to get
+rid of the VPS in which they were hosted, and wanted to have it all on one VPS,
+the one where my website is being hosted, among many other things, such as my
+email server.
+
+I already knew that there were some light-weight web front-ends for git, such as
+cgit[^3], or even git's own web interface; I didn't know what exactly they had
+though, so I started digging deeper into git itself, by reading the
+documentation, and looking in the internet for tutorials and explanations of how
+cgit and similar tools worked.
+
+After learning more about git, and finding myself with more time than usual all
+thanks to COVID-19[^4] and the government, I finally decided to just trying out
+setting up my minimal Git server.
+
+## Prelude, or some things to know before setting up a git server
+
+Before trying to setup a git server this way, you should know some basic stuff
+about how git works. I didn't know a lot of these details when I embarked myself
+upon this venture, so I ended having to read a good deal of information on the
+way git works, which one of the reasons I really enjoyed this. Turns out git is
+a much better piece of software than I thought before, and I already had a
+really high opinion of it.
+
+First, all of the git hosting solutions out there, Github, Gitlab, Gitea etc.
+are using none other that git, and some other like gitolite, as their main
+backend.  Every time you push something to, say Github, be it by ssh or http,
+git gets called on their server. Github just shows you what git is telling them.
+
+Second, you don't really need a web interface to have a git server. The web
+interface is just a nice way of showing your repositories to other people. A
+computer with ssh access is more than enough, if all you want to do is host your
+repositories.
+
+Third, there are, in practice, two types of repositories: 
+
+1. Standard repositories that contain all the information and actual files in the
+   `.git` directory, together with a "working directory" at the root, which is the
+   representation of the state of your projects in the ref you are currently, and
+   where you actually, well, work on your files by editing them. This are not the
+   actual files stored by git.
+2. Bare repositories, which forgo the "working directory" and just contain all
+   the git files and information at the root of the directory. This types of
+   repositories are the ones meant for the server side of things.
+
+## Setting up the git backend
+
+Now that you know a little bit more about how git works (or maybe you already
+knew that), it's time to set up the actual git server. All you need is a
+computer with access to the internet, preferably with a static IP (like a VPS),
+and optionally a domain name, just to make things prettier and easier. To guide
+myself through this process, I mostly followed the Arch Linux wiki guide on
+it[^5], and the official Git book chapter on Git on the server[^6].
+
+First thing that you need to do, is setup ssh access. This is pretty easy to do,
+you most probably already have it setup, and it is out the scope of this
+tutorial, so I am not going to explain how to do it in this post.
+
+Next, you need to make a user specially for git. This is quite important, since
+you don't want people with access (even just read access) to your repositories
+to have access to other parts of your server. For simplicity, we'll just call it
+git.
+
+Each Linux distro (or Unix-like OS), has its own defaults, and sometimes even
+alternate ways to create users. For simplicity's sake I decided to use Debian's
+defaults when creating the git user:
+
+```
+# useradd git
+# passwd git
+```
+
+These two commands, at least in Debian Buster, should create a normal user with
+a home directory at `/home/git/` and prompt you for a password for the git user.
+Some people like to have the git home directory in a location such as
+`/srv/git/`, but I just decided to go with the defaults for simplicity's sake.
+
+Now we need to setup some things for the git user. First we should switch to it:
+
+```
+$ su -l git
+```
+
+And once we are the git user, we should make a `.ssh` folder with a
+`authorized_keys` file, which should contain your public key(s), and also the
+keys of any people you plan on having write (push) access to your repositories.
+Keep in mind, that any people with push access, will have push access to **all**
+of the repositories under the git user.
+
+Now we should think about where we are going to store the git repositories. You
+could just store them at the root of the git home directory. However, I decided
+to store them in a separate folder, since I also wanted to keep some (mainly
+uninteresting) repositories private. So I created two folders `private`, and
+`public`. Public, as its name implies, is going to be the directory with the
+repositories that are going to be shown on the web interface for all the people
+to see. I could have also created a separate web interface protected by password
+for the `private` repositories, but the web interface is mainly just for people
+on the internet to take a glance at my repositories.
+
+Now for a test, you can create your first repository under the public folder,
+and try pushing into it. So we should cd into our `public` directory (or
+whatever directory you plan to store your repositories at) and init a **bare**
+repository. By convention bare repository names end with a `.git`, so it should
+look something like this:
+
+```
+$ cd ~/public
+$ git init --bare my-repo.git
+```
+
+Any time that you want to add a repo to your server, you'll basically need to
+login to your git user and repeat the steps above. Or, if you already have a
+non-empty server, you could clone it this way:
+
+```
+git clone --bare <path-or-url-to-my-repo> my-repo.git
+```
+
+Now you can push from your local computer to that directory using ssh this way:
+
+```
+$ git remote set-url origin git@your-domain-or-ip:public/my-repo.git
+$ git push -u origin master
+```
+
+Or clone it, once you have something there:
+
+```
+$ git clone git@your-domain-or-ip:public/my-repo.git
+```
+
+Now you have working git server. It's that simple! However, right now there's
+now read-only access for other people to clone your repos. There's two ways you
+could solve that. You could use git's own smart HTTP, or the git daemon. In my
+case I didn't end up using git's smart HTTP, since I used cgit's functionality
+for that, but if you are interested in any of them, you can check out the
+official git book chapter that I mentioned earlier[^6].
+
+Before we go further, though, we need to harden our git user a little bit,
+especially if you intend on giving other people push access. Right now, anybody
+with push access, basically has ssh access to your git user, since basically
+when you are pushing using this method, the only thing that is happening is that
+git is ssh'ing into your server and executing git (the one on your server) in
+your server. That's a big no-no. For that, we are going to change the standard
+login shell of git, to one that will allow git to function properly, but won't
+allow people to login to an full-fledged interactive shell into your git user
+using ssh.
+
+Fortunately, git provides use with a limited shell for this specific case,
+called the git-shell. First, we need to know where it is located:
+
+```
+$ which git-shell
+/usr/bin/git-shell
+```
+
+Now that we know its actual location, we need to change the default shell for
+our git user:
+
+```
+# chsh -s /usr/bin/git-shell git
+```
+
+Now you might think, well, how do I log into the git user to add/edit
+repositories? Just log into your normal user, and then use `su` to login with a
+shell like sh, bash, or zsh:
+
+```
+$ su -s /bin/bash git
+```
+
+## Setting up the web interface
+
+Git provides its own web interface, which, if you want the minimalest of the
+minimalest, should be fine. I, however, decided to use cgit[^3], since I wanted
+a little more flexibility and more room for configuration. It's even the
+preferred web interface for the kernel.org git repositories.
+
+To use it, we'll need not only it, but also a reverse proxy for it to work with
+it. My preferred program for that is nginx, I have used it for several years,
+and it love it for its ease of use and its performance. You could also use other
+software, like Apache, if that's what you prefer.
+
+Even though I'm using Debian on my VPS, I mostly followed the Arch Linux Wiki
+article on cgit[^7]. If you're not yet familiar with the Arch Wiki, you should
+definitely check it out. It's arguably the best resource on Linux out there.
+
+So basically we need the following packages:
+
+```
+# apt install nginx fcgiwrap cgit
+```
+
+After that, we should go on and configure nginx. I won't go into big detail on
+how to configure nginx itself, but the configuration file for cgit should be
+located here `/etc/nginx/sites-available/cgit` and look something like this:
+
+```
+server {
+    listen 80;
+    listen [::]:80;
+    server_name <your-git-domain>;
+    return 301 https://$host$request_uri;
+}
+
+server {
+    listen 443 ssl;
+
+    server_name <your-git-domain>;
+    ssl_certificate <your-ssl-cert-path>;
+    ssl_certificate_key <your-ssl-cert-key-path>;
+
+    root                  /usr/share/cgit;
+    try_files             $uri @cgit;
+
+    location @cgit {
+                include             fastcgi_params;
+                fastcgi_param       SCRIPT_FILENAME /usr/lib/cgit/cgit.cgi;
+                fastcgi_param       PATH_INFO       $uri;
+                fastcgi_param       QUERY_STRING    $args;
+                fastcgi_param       HTTP_HOST       $server_name;
+                fastcgi_pass        unix:/run/fcgiwrap.socket;
+    }
+}
+```
+
+This configuration assumes that you have setup an SSL certificate. You can
+configure it just use plain HTTP, but I really recommend you get an SSL
+certificate.
+
+Now that we have nginx configured we should start and enable both nginx and
+fcgiwrap.
+
+At this point, you should be able to see cgit by going to the domain or url that
+you pointed at in the nginx config. However, it probably won't be showing you
+your repositories just yet, and that's because we need configure cgit, in part
+so that we can tell it where to look for our repos.
+
+I won't go into much detail on how to configure cgit, but I'll show you my
+configuration file, so you have an idea of how to configure yours. This file
+should be located at `/etc/cgitrc`:
+
+```
+#
+# cgit config
+# see cgitrc(5) for details
+
+css=/cgit.css
+logo=/cgit.png
+virtual-root=/
+max-repodesc-length=100
+repository-sort=age
+
+root-title=Yaroslav's git repositories
+root-desc=Public repos for some of my projects, and mirrors of other projects.
+
+# Possible readme files for about page
+readme=:README.md
+readme=:README.txt
+readme=:README.rst
+readme=:README
+
+# Syntax highlighting
+source-filter=/usr/local/lib/cgit/filters/syntax-highlighting.py
+
+# About page rendering script
+about-filter=/usr/local/lib/cgit/filters/about-formatting.sh
+
+# Some common mimetypes for serving files raw
+mimetype.gif=image/gif
+mimetype.html=text/html
+mimetype.jpg=image/jpeg
+mimetype.jpeg=image/jpeg
+mimetype.pdf=application/pdf
+mimetype.png=image/png
+mimetype.svg=image/svg+xml
+
+# Set of snapshot formats for people to download
+snapshots=tar.gz zip
+
+# Git configuration
+enable-http-clone=1
+enable-commit-graph=1
+remove-suffix=1
+section-from-path=1
+scan-path=/home/git/public
+clone-prefix=https://git.yaroslavps.com git://git.yaroslavps.com
+```
+
+The most important bits here are:
+
+```
+virtual-root=/
+enable-http-clone=1
+scan-path=/home/git/public
+clone-prefix=https://git.yaroslavps.com
+```
+
+You can read more about cgit's configuration in the Arch Wiki article I
+mentioned earlier[^8]. I'll just say, that you should keep in mind that the
+order of the configuration variables is important. If something is not working
+as it should, its probably because you put some variable before another one, and
+their order should be changed. This gave a big headache when I was trying to
+configure cgit for myself.
+
+One last thing worth mentioning, regarding cgit, is that you can configure each
+repo for cgit by adding a `cgitrc` file inside the bare repository in your
+server. You can then add some more information about it like this:
+
+```
+owner=Yaroslav de la Peña Smirnov <yps@yaroslavps.com>
+desc=Personal site and blog.
+```
+
+For more information, read cgitrc's manual:
+
+```
+$ man 5 cgitrc
+```
+
+## Conclusion
+
+This is one possible way of having your own hosting solution for git
+repositories, in a very minimal way. Since the cgit frontend is not running
+constantly, it's written in C, and is very minimal, it barely consumes any
+resources. This is also a great way to be more independent of centralized,
+proprietary services like GitHub. Of course, services like those are always
+great for visibility, but you could always just have mirror of your open source
+projects on them that points to your personal server.
+
+There many other great things that you can do with a setup like this, like
+setting up git hooks, without having to set up a complex and complicated CI/CD
+system, for simple tasks, like in my case, pushing, generating and deploying my
+site on each push to master. For an example of a post-receive you can checkout
+mine here:
+[https://git.yaroslavps.com/yaroslavps.com/tree/post-receive](https://git.yaroslavps.com/yaroslavps.com/tree/post-receive)
+
+For a little more information on git, I recommend you to read the following
+manual pages:
+
+```
+$ man git
+$ man git-init
+$ man git-clone
+$ man git-config
+$ man githooks
+$ man git-send-email
+$ man git-format-patch
+```
+
+The last are especially useful if you want to know how to collaborate with other
+people on other projects, or want other people to collaborate on yours, without
+having to resort to the centralized way of doing things of GitHub and similar
+sites. Git is especially well suited for collaborating through email, which is
+in and of itself a decentralized service. There's a guide on collaborating
+through email using git[^8], written by the great Drew DeVault, which is, in my
+opinion, one of the best FOSS programmers out there. I recommend also to check
+out his blog[^9], which I must say is much more interesting than mine.
+
+
+## A little unrelated note
+
+As I write this post, this strange year is (finally) coming to an end. It wasn't
+a completely terrible year per se, but it did present us with a lot of nasty
+surprises. I have quite a lot of personal plans for this coming 2021, some of
+which involve this site. Some of them I'll keep a secret for now, but most
+importantly I want to diversify my writing a little bit more. I kind of already
+did that this year, with my "recipe book"[^10], but I also want to start writing
+about some other things on my weblog. So far it's been mostly about software
+development and computer-related stuff, but there are many other things not
+related to that on my mind, which I'd like to write about.
+
+Here's to a New Year, that's not as Orwellian as this one, and may you
+accomplish as much as you set yourself to accomplish.
+
+Happy New Year and Merry Christmas[^11] to all!
+
+[^1]: [https://gitea.com/](https://gitea.com/)
+
+[^2]: [https://www.drone.io/](https://www.drone.io/)
+
+[^3]: [https://git.zx2c4.com/cgit/about/](https://git.zx2c4.com/cgit/about/)
+
+[^4]: I'll write a little bit about my experience with the corona, and my
+  thoughts on it next year.
+
+[^5]: [https://wiki.archlinux.org/index.php/Git_server](https://wiki.archlinux.org/index.php/Git_server)
+
+[^6]:
+  [https://git-scm.com/book/en/v2/Git-on-the-Server-Getting-Git-on-a-Server](https://git-scm.com/book/en/v2/Git-on-the-Server-Getting-Git-on-a-Server)
+
+[^7]: [https://wiki.archlinux.org/index.php/Cgit](https://wiki.archlinux.org/index.php/Cgit)
+
+[^8]: [https://git-send-email.io/](https://git-send-email.io/)
+
+[^9]: [https://drewdevault.com/](https://drewdevault.com/)
+
+[^10]: [https://www.yaroslavps.com/food/](https://www.yaroslavps.com/food/)
+
+[^11]: I know, a little bit late for Catholic or Protestant folks out there. But
+  just in time for (Russian) Orthodox Christmas ;)