We're still under construction!

And you just found out where. We're still working on this page. How did you get in here, anyway?

How to set up an gitserver and website in 2022

A complete and thorough guide to setting up a push-to-deploy gitserver and website on Ubuntu using SSH, git, and NGINX. It covers all of the essentials, including how to set up basic security.

---

Getting started

It has been a while since I last set up a web- server and site. I'm an old hand at that, and one truth about the internet is that as times change, the tools change - for better or worse.

I remember first cobbling together some astoundingly terrible early PHP with internet friends when we were all still high-school age. We were such fools; use of uninitialized variables was the bane of our site's codebase, and everything was coded live because source control didn't exist for us yet. Git had only been invented a few years before, so of course we didn't know about it. Web development was hard, and CSS was terrible, and life was great.

I've set up servers and sites since then, and I recently decided to revamp my server setup because I'd let it collect dust, unused due to how badly it was set up. It's time to fix my git- and webserver!

So how does one set up a git and webserver in 2022, assuming one has already built a local website?

Getting it on the server isn't too bad, as remote server setup is easier than ever, and a good source control setup means that deploying to live can be done with a single commit pushed to the server. HTML and CSS isn't too bad either these days, and it's much easier to slap together some responsive UI using semantic HTML with CSS3 supporting such things as var and calc.

We'll go over each of these in due turn. There's lots of little things that need care, so we'll try to be thorough and document them all.

I'll be swapping between users a bit, so I'll try to be clear when we ssh or log in one layer deeper, and when we logout back one layer. We've got the local user on the local machine, but also root me and git on the server.

If at any time you are confused by the shell commands, this site can help explain them. Simply paste the command in question there to get a nice breakdown of what it all means.

Step 1 - Get a server

You'll need a server to host your website. If you've got your own machine lying around - great! Plug that sucker in and slap Ubuntu on it!

We're going to assume that that isn't the case, though and so we are actually going to rent a remote server in the cloud.

If you are only thinking about hosting a static html website, you could pay for site hosting on github pages or wordpress, but that gives up a lot of fine control and power if you want to do literally anything else, and will cost the same or more. We can use our server for /anything/, we're just doing a git- and web- server as examples. The process can be used to deploy *any* of your repos / tools, such as game servers or other services.

I use DigitalOcean as my host - they're cheap, simple, give you lots of control, and have tons of help tutorials and optional tools. Renting a droplet for a webserver has been around 5$USD / mo for a decade, and gives you 25gb of space and plenty of bandwidth.

Step 2 - Install Ubuntu (or some other flavor of Linux) and dependencies

You'll need to install some flavor of Linux, and we're going with Ubuntu. It's popular, and mostly comes with everything we need. You totally can use another flavor of Linux, but the instructions may then need to vary depending on your distro. We assume that if you are savvy enough to already be running some distro, you are savvy enough to do this on your own, and so we will stick with the Ubuntu instructions. In general this just means another distro may use another shell, package manager, service controller, etc.

If you're doing this on your own machine, you'll have to deal with the usual images and booting, but you'll also likely have a keyboard and monitor, and so can directly log in to the server for access to things like its IP.

Otherwise, your host should give you your server's IP, plus any access credentials.

Either way, the first thing to do is to make sure we can log in, and that everything's fine.

shell - me@local:~

ssh root@MY_SERVER_IP

It should ask for the root password, because we haven't configured SSH yet, which we'll do in a moment but not just yet.

We're going to pause for a bit to set up a better way to access the machine than by IP address - we're going to point a domain name at it!

shell - root@MY_SERVER_IP:~

logout

Step - Configure your DNS

Unless you want to memorize IP addresses, get a domain name set up.

We're doing this even before configuring SSH, because we'd have configure SSH using IP addresses if we did it later, and we'd have to change them later when we configured DNS. It's just a bit easier to set up DNS first. In fact, you can set up the DNS before you even have a server, if you know what the IP is going to be, or if you don't mind updating the IP later.

If you're running your own machine, point the domain to the IP of the machine. Your registrar has its own nameservers, so you just need to add A and/or AAAA entries to its DNS table to associate your domain name with your server's IP. The instructions for this are registrar-specific, so you'll need to do a little legwork to figure out how.

Make sure you do both my.site and www.my.site because www is a subdomain that doesn't exist by default!

If you're following along in the cloud, things are a little different, because your registrar and host may not be the same entity. If they are, you just need to add A and/or AAAA entries to its DNS table, same as with running your own machine - its just that they are supplying both the machine, and the domain name.

If your registrar is not your host, then you will need to point the domain to the relevant host's nameservers. This is, again, registrar specific and requires legwork. Once you have done that, you need to add NS entries to the host's DNS table, associating your domain name to the nameservers that you just pointed your registrar at. That way, your registrar (which knows your domain name) knows about your host's nameservers, and your host (which knows about your server / IP) knows which registrar to expect. Together, they have the authority to tell all requests for your domain to point at your machine. Then you just need to add A and/or AAAA entries to the host's DNS table as in earlier cases.

Congratulations! You should now be able to SSH in using your domain name (give or take a little bit for DNS records to percolate).

shell - me@local:~

ssh root@my.site

It is still asking for a password though - we're going to set up SSH keys next.

shell - root@my.site:~

logout

If you're running your own machine, you could run your own name server. This is way more intensive (we're drifting into 'build your own registrar' territory here). For more info, see stuff like: https://ubuntu.com/server/docs/service-domain-name-service-dns

Step - Configure SSH

If you've got your own server with keyboard and mouse and monitor, you won't *need* to do this, but you'll still probably want to do it anyway so you can access it remotely without a password. If your machine is a remote cloud, you'll *need* to do this, because otherwise you'll lose access or forever be resetting the password.

If you haven't already, make sure your local ssh directory is set up, and that you have correctly set the permissions (otherwise SSH will complain at you and may refuse to continue).

shell - me@local:~

mkdir ~/.ssh
chmod 700 -R ~/.ssh

Now we will generate SSH keys. We'll use ed25519 instead of rsa.

shell - me@local:~

ssh-keygen -t ed25519 -C "me@my.site" -f ~/.ssh/my-key

It will still ask you to enter a passphrase - don't leave it unprotected.

You should then add the key to your local ~/.ssh/config file to include something like this:

ssh config - me@local:~/.ssh/config

Host *my.site
    User me
    IdentityFile ~/.ssh/my-key
    IdentitiesOnly yes

Now all you need to do is copy your public ssh key to the server, and put it in the authorized_keys file at /root/.ssh/authorized_keys (should resolve to /root/.ssh/authorized_keys).

There are two ways to do this - ssh-copy-id or ssh + cat.

With ssh-copy-id, instead of SSH'ing into the server, we call on a command-line tool to copy it over for us.

shell - me@local:~

ssh-copy-id -i ~/.ssh/my-key.pub root@my.site

In the -i argument for the identity file, "If the filename does not end in .pub this is added." We do this anyway to avoid confusion.

This will add the public key to the authorized_keys file, creating it if it doesn't exist.

Using ssh-copy-id has several advantages, as it does most of the work for us, but it is not always available. We can still do it with ssh + cat, we just have to do a tad bit more work ourselves.

First we ssh back into the server root.

shell - me@local:~

ssh root@my.site

Then we configure the server root SSH directory, and log back out.

shell - root@my.site:~

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
# Configure root SSH 
logout

Finally we copy over the SSH key.

shell - me@local:~

cat ~/.ssh/my-key.pub | ssh root@my.site "cat >> ~/.ssh/authorized_keys"

Note that we have to specify the .pub extension, unlike with ssh-copy-id, which will automatically add the extension if it is missing.

Now, we should be able to log in without typing our password.

shell - me@local:~

ssh root@my.site

We don't want to do too much with root though, we just want to make sure we can access it if needed.

Next we'll create a private user for actually doing things.

Setting up a private user means we will have to use sudo for some things, but that is fine. You can either do all of these as root if you want - feel free to use sudo or su -, whichever you are most comfortable with.

Step - Set up private user

Now, we're going to create a private user. We're going to do this for security purposes, but also convenience. It's best to log in to me@my.site and then sudo or su - as needed.

shell - root@my.site:~

adduser me

Do be careful of the difference between adduser and useradd. useradd is a low-level operating system primitive that by default only creates the user, whereas adduser is a script that performs other tasks like creating the home directory, or setting up groups. If you accidentally call useradd instead of adduser, you'll need to do those things yourself!

This will run an interactive script prompting you to fill out various details, starting with the password. The other details are optional, and the defaults are pretty sensible; it explains itself effectively enough for me to not record it here.

Give it sudo powers.

shell - root@my.site:~

usermod -aG sudo me

Now we can log into the new user

shell - root@my.site:~

su - me

Set up the private user's ssh directory next. We can use the same procedure as we did with root, except that now we can copy root's authorized_keys file instead, if we want.

shell - me@my.site:~

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
sudo cat >> ~/.ssh/authorized_keys /root/.ssh/authorized_keys
logout

Add any keys that this machine will need to itself ssh to, such as github. We would do this for our own git server if it were a separate machine, but we're going to make a git user on this machine instead. Still, it will be nice to know how to do it in case you need to ssh out for other reasons.

In the end, we should be able to ssh into our user on the server.

shell - me@local:~

ssh me@my.site

Step - Install the web server (nginx)

From this point, we may need to prefix some commands with sudo. The other option is to su - to log into root. We'll try to denote these commands with the sudo pre-command.

Ubuntu doesn't come with nginx, but its easy enough to install. This is our first install in this tutorial, so we should update apt first, to make sure we get the latest version of nginx.

shell - me@my.site:~

sudo apt update
sudo apt install nginx

This should install nginx and start the daemon automatically. We can check and see if it's running

shell - me@my.site:~

systemctl status nginx

This only tells us that it is working from the inside. To see if it is working from the outside, we have to request a page. We don't have a site set up yet, but luckily nginx provides a default placeholder.

We can use curl to request it.

shell - me@my.site:~

# curl may need to be installed
sudo apt install curl
# Request the site
curl http://my.site
# Request just the headers
curl -I http://my.site

Or you can use a browser to visit it at http://my.site. Either way, congratulations!

Nginx is nice enough to start itself, and also add itself to systemd; that way nginx will start itself automatically if we reboot. We can still control it manually using systemctl.

A quick refresher on systemctl.

If you need to start or stop:

shell - me@my.site:~

sudo systemctl stop nginx
sudo systemctl start nginx

A restart is the same as a stop and a start, while a reload re-reads configs without killing the server.

shell - me@my.site:~

sudo systemctl restart nginx
sudo systemctl reload nginx

Finally, we can disable and re-enable nginx automatically starting on boot

shell - me@my.site:~

sudo systemctl disable nginx
sudo systemctl enable nginx

We'll need these commands later when we start making changes to the nginx config. Now we can start hosting things, right?

Step - Setup the firewall (ufw)

Not so fast. Now, we'll want to set up even more security. Since we're using ubuntu, that means using ufw, the Uncomplicated Firewall. We'll have to let ssh and nginx through the firewall, but that's easy.

ufw should be preinstalled, but if not, sudo apt install ufw.

Then, see what application profiles we have available.

shell - me@my.site:~

sudo ufw app list
Available applications:
  Nginx Full
  Nginx HTTP
  Nginx HTTPS
  OpenSSH

We don't want to enable the firewall without allowing a few things through first. Imagine if we locked ourselves out of the box via firewall!

shell - me@my.site:~

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable

It is always good to check and see if we did it correctly.

shell - me@my.site:~

sudo ufw status
Status: active
To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
Nginx Full                 ALLOW       Anywhere

If you see something like that, all is well.

Step - Setup the git user

You could also just pay for github services, but they're making you pay a lot for something you can do yourself for free. Essentially they're making you pay for a wrapper around git, even though (or because) git is a free tool that does this already. This isn't the last time you'll see companies selling thin shells around the real software which they didn't write. They aren't better at software, they're just better at marketing to and capturing users.

git should be preinstalled, but if not, sudo apt install git.

To handle repos, we need to have a git user. It's going to own the site directory for deployment purposes anyway.

shell - me@my.site:~

sudo adduser git

We'll need to set up a bit more security with git-shell. Find where it is.

shell - me@my.site:~

which git-shell
/usr/bin/git-shell

Check /etc/shells to see if we need to add it.

shell - me@my.site:~

cat /etc/shells
# /etc/shells: valid login shells
/bin/sh
/bin/bash
...

If git-shell isn't in /etc/shells, add it.

shell - me@my.site:~

sudo echo "/usr/bin/git-shell" >> /etc/shells

Change the git user's shell from bash to git-shell.

shell - me@my.site:~

sudo chsh git -s $(which git-shell)

Now, the git user can't execute arbitrary things. We can still log into it if needed, which we do need to do to configure the git user's SSH.

shell - me@my.site:~

sudo su - git -s /bin/bash

You can use this any time you need shell access for the git user, you just have to log into root or a user with sudo first.

We can now configure the git user's SSH as we did before. Don't forget to copy the relevant keys!

shell - git@my.site:~

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
sudo cat >> ~/.ssh/authorized_keys /root/.ssh/authorized_keys

We're getting a lot of mileage out of that shell snippet aren't we?

Step - Create git repo

At long last, we can create an actual git repo. It is now worth spending a little time talking about your git user's repo directory structure. We'll set it up something like this, to make for easy parity between the server repos and checked out local repos.

Structure of git user home DIR

/home/git/
    org/
        group/
            /project.git

The group/ folder can be a subgroup of the org, or some target domain, language, or architecture. (This lets you organize third-party libraries clearly, as well)In our case, we're building a website, so we'll use www as our group-domain. My site is my.site, so I ended up with /home/git/myorg/www/my-site.git for the remote repo.

I use the same structure for checking out working repos to the local machine, except the path starts at a git subfolder in my home dir instead of at the git user's home dir, as we would do on the server. This means I end up with ~/git/myorg/www/my-site.git for my local repo.

Lets make that git repo!

shell - git@my.site:~

mkdir ~/myorg/www/
cd ~/myorg/www
git init --bare my-site.git

Make sure you create and edit these repos as the git user, or else you'll have to chown them later as root or a sudo user.

shell - root@my.site:~

sudo chown -R git:git /home/git/myorg/www/my-site.git/

We could use ~git as shorthand for /home/git but if we have stuff like command --path=~git/ it doesn't work because the tilde is in the middle. So we're stuck using /home/git for consistency.

We're done directly using the git user for a bit, so we log back out to our local machine now.

shell - git@my.site:~

logout

We should be able to access this repo now from ssh remotely.

shell - me@local:~

mkdir ~/git/myorg/www/
cd ~/git/myorg/www
git clone git@my.site:myorg/www/my-site.git
cd my-site

Wowee! we're going to have more things in here than just the final site, so we'll create some repo structure.

shell - me@local:~/git/myorg/www/my-site

mkdir public_html
touch public_html/index.html
touch public_html/404.html
mkdir git-hooks
touch git-hooks/post-receive

It should look like this:

Structure of site repo

./
    public_html/
        index.html
        404.html
    git-hooks/
        post-receive

Now, we fill it with content.

./public_html/index.html

<html>
    <head>
        <title>Welcome to my.site</title>
    </head>
    <body>
        <h1>Success!</h1>
        <p>Welcome to my.site</p>
    </body>
</html>

./public_html/404.html

<html>
    <head>
        <title>404 - Not Found</title>
    </head>
    <body>
        <h1>Error!</h1>
        <p>404 - Not Found</p>
    </body>
</html>

./git-hooks/post-receive

#!/bin/bash

# Vars
GIT_DIR="/home/git/myorg/www/my-site.git"
PROD_DIR="/var/www/my.site"
DATE=$(date)

# Loop through received branches
while read OLDREV NEWREV REF
do

    BRANCH=$(git rev-parse --symbolic --abbrev-ref $REF)

    # Note that I'm assuming the branch we want to deploy on is 'main'. You can call it prod or whatever.
    if [[ $BRANCH = "main" ]];
    then

        # Clean up prior site/html folder because -f is ignored outside of git repos
        rm -rf $PROD_DIR/public_html/*

        # Deploy to production
        git --work-tree=$PROD_DIR --git-dir=$GIT_DIR checkout -f -- public_html

        # Log success
        cat << EOF >> logs/post-receive.log
post-receive[$BRANCH,$GIT_DIR,$DATE]:
    Successfully deployed $BRANCH to $PROD_DIR
EOF

    else

        # Log no action
        cat << EOF >> logs/post-receive.log
post-receive[$BRANCH,$GIT_DIR,$DATE]:
    No action.
EOF

    fi

done

Add them all to the repo! Commit! Push to our remote repo on the host!

shell - me@local:~/git/myorg/www/my-site

git add .
git commit -m "First commit"
git push origin/main

We're getting somewhere now! It won't trigger the post-receive yet, but we can edit our site now. We still can't see it, just the nginx site placeholder, but let's fix that next.

Step - Set up nginx site configuration

We're in the home stretch now.

shell - me@local:~/git/myorg/www/my-site

ssh me@my.site.

First off, we don't want to serve the site directly from the git repo, because it's a bare repo, somewhere under the git user's home directory. We need to put it in a proper place, accessible to the web server. We'll put it in the traditional place.

shell - me@my.site:~

mkdir /var/www/my.site

We'll manually perform the result of the post-receive since it can't trigger yet.

shell - me@my.site:~

git --work-tree=/var/www/my.site --git-dir=~/git/myorg/www/my-site.git checkout -f -- public_html
sudo chown -R git:git /var/www/my.site
sudo chmod -R 755 /var/www/my.site

Now we have our site in place, but nginx can't find our website to serve it. How do we tell nginx that we have a site of our own? nginx locates its configuration files at /etc/nginx. There are a lot of folders, and there are two in particular that we need - sites-available, which contains configurations of all the sites you might want to host, and sites-enabled, which contains the configurations of all the sites that you are actually hosting.

We can create a config in sites-available.

shell - me@my.site:~

touch /etc/nginx/sites-available/my.site

Fill /etc/nginx/sites-available/my.site with this:

/etc/nginx/sites-available/my.site

server {

    listen 80;
    listen [::]:80;

    server_name my.site www.my.site;
    root /var/www/my.site/public_html;
    index index.html;

    location / {
        try_files $uri $uri/ =404;
    }

    error_page 404 /404.html;
        
    location = /404.html {
        root /var/www/my.site/public_html;
        internal;
    }

}

Then just symlink to it in sites-enabled.

shell - me@my.site:~

sudo ln -s /etc/nginx/sites-available/my.site /etc/nginx/sites-enabled/my.site

It is also recommended to ensure to edit /etc/nginx/nginx.conf and set the http.server_names_hash_bucket_size config variable to be enabled and set to at least 64. It looks something like this:

/etc/nginx/nginx.conf

http {
    ...
    server_names_hash_bucket_size 64;
    ...
}

The internet is litered with references to doing this, with little mention of why. If we look at the NGINX documentation, we find that this is the server name maximum length, and if the value is set to 32, then the server name too.long.server.name.example.org will cause nginx to fail.

We could reload or restart nginx now, but first we can test our config changes.

shell - me@my.site:~

sudo nginx -t

If it looks good, we can restart nginx

shell - me@my.site:~

sudo systemctl restart nginx

We should see our site pop up! Not the default one - the one we made!

Step - LetsEncrypt Cert

We need to enable HTTPS. Yes, half this tutorial is about good basic security. We'll use LetsEncrypt - its free and easy!

Letsencrypt has its own instructions., but I'll repeat them here for brevity and specificity.

Certbot is installed with snap. Ubuntu comes with snap but like apt it may need updating.

shell - me@my.site:~

sudo snap install core
sudo snap refresh core

Install certbot

shell - me@my.site:~

sudo snap install --classic certbot

Add it to our path

shell - me@my.site:~

sudo ln -s /snap/bin/certbot /usr/bin/certbot

Run certbot to generate our certs

shell - me@my.site:~

sudo certbot --nginx -d my.site -d www.my.site

Remember when we added the www subdomain forever ago? This is when we need it. We have to create the certs at the same time, or else LetsEncrypt won't know they are related.

These certificates do expire, but certbot will renew them automatically, periodically checking and renewing any certificates that are about to expire. We can do a dry-run test of certificate renewal to see if we've got everything done properly

shell - me@my.site:~

sudo certbot renew --dry-run

If all goes well, we should be able to see both https://my.site and https://www.my.site

Step - Enable server-side includes

This is a small step that greatly improves your site development quality of life. You can write your site as a series of small snippets, and include them using SSI directives, instead of copying and pasting the same boilerplate into every page. nginx is a bit picky in how this gets turned on, but I've found enabling it in the /etc/nginx/sites-available/my.site like this works best.

/etc/nginx/sites-available/my.site

server {
    ...
    index index.html index.shtml;
    ...
    location / {
        ...
        # SSI
        ssi on;
        ssi_silent_errors on;
        ssi_types text/html text/shtml;
        ...
    }
    ...
    location = /404.shtml {
        ...
        # SSI
        ssi on;
        ssi_silent_errors on;
        ssi_types text/html text/shtml;
        ...
    }
    ...
}

Note that we also edited the index field to include index.shtml.

Once you've turned it on (and tested nginx with sudo nginx -t && sudo systemctl reload nginx), we can now use server side includes in any *.shtml file.

For instance, this site uses <!--#include virtual="/includes/body/footer.html" --> to include the footer in every page.

Step - Setup git hooks for auto-deployment

The last thing is to hook up our auto-deployment trigger. Luckily, since we included it in our git repo, that's easy to copy out of the repo. Dont forget to set it as executable!

shell - me@my.site:~

sudo git --work-tree=/var/www/my.site --git-dir=/home/git/myorg/www/my-site.git checkout -f main -- git-hooks/post-receive
sudo mv git-hooks/post-receive hooks/post-receive
sudo chmod +x hooks/post-receive
sudo chown git:git hooks/post-receive
sudo rm -rf git-hooks

Now we should see this trigger whenever we push to main, doing the same deployment to /var/www/my.site that we did manually.

Step - Create your site

Congratulations! You're done!

This is where this very long how-to article ends. Thankfully we never really need to do any of it again. We can just commit to push to live now. So go edit your website, and set up other auto-deploying tools. In a future article, we will discuss building a lightweight, responsive website with semantic HTML5 and CS3, and when we do we'll link it here.

- Leo D., July, 2022

---