How To Run Your Own IPFS Gateway

IPFS has proven to be the decentralized storage protocol of choice by many blockchain developers, and one of the crucial tools used to access content on IPFS are Gateways. IPFS Gateways are like bridges between the IPFS protocol and the HTTP protocol that we use everyday to browse websites. There are lots of different options to choose from when it comes to IPFS Gateways, and in this post we’ll show you how to host and build your own!

⚠️Warning! This guide will show you how to make a public IPFS Gateway that can access any CID on IPFS, which means it has the potential to be abused. Please be cautious and look into ways you can secure your gateway.

Requirements

In order to follow this guide you’ll need a few things. First and foremost you’ll need a decent amount of experience using Linux servers and navigating around in the terminal, things like creating daemons or editing text files with vi or nano.  You’ll also need a cloud server provider, and there’s plenty to choose from. In this guide we’ll use Digital Ocean and get a simple droplet. Also if you want to have a custom domain instead of using an IP address you can get something through a domain provider like Namecheap.

Setting Up the Server

Before we rent a server for our IPFS node, you’ll want to create an SSH key to login with. This is the preferred secure way to SSH into your server versus a user name and password. You can check out this guide on how to create them.

It will prompt you to put in a passphrase to access the key so choose something secure, and once created it will make a file located in ~/.ssh/rsa.pub. We’ll use the contents in just a moment when we setup the server.

Since we’ll be using DigitalOcean you can head over there to create an account and buy a Droplet. You definitely don’t need anything crazy, I just got the following:

For the authorization select SSH Keys, then copy and paste the contents of ~/.ssh/rsa.pub and paste it in as a new key.

After the droplet has been created, you will actually want to turn it off, go to the Network settings, and enable IPV6. Once enabled turn it back on and try to SSH into it with the following command with the IPV4 address of the server:

ssh root@ipv4address

It should prompt you to enter in the passphrase for you SSH key, and after entering it you should be in!

While we are signed in, we are currently logged in as root, which is not the most secure practice. This next step is optional, but highly recommend. First we’ll create a new user with the command adduser steve, and of course you can use whatever username you want to. It will prompt you to make a new password and for some other information you can leave blank. Next we need to give the user the permissions necessary to run the IPFS node with sudo usermod -aG sudo steve (and of course from this point on replace steve with the username you chose).

Next we’ll need to run the following commands to create an .ssh directory for our new user and give proper permissions so we can edit it.

mkdir /home/steve/.ssh
touch /home/steve/.ssh/authorized_keys
sudo chown -R steve:steve /home/steve/.ssh
sudo chmod 700 /home/steve/.ssh
sudo chmod 600 /home/steve/.ssh/authorized_keys

With those commands completed you can now login as your user with su steve and then edit the SSH keys files to paste in your own that we used earlier with either vim or nano then /.ssh/authorized_keys. After pasting in your key you can run exit to log out of the user, then again to leave the SSH session. Now you should be able to SSH in with the new user ssh steve@ipv4address.

Install IPFS

Once you’re in your server you can run sudo apt update just to make sure all your packages are up to date. Then you will want to visit the release page for IPFS Kubo, the Go implementation of an IPFS node used pretty much everywhere. On that page you can choose the latest stable release, then locate the correct distribution for your OS. In my particular case it ended up being kubo_v0.22.0_linux-amd64.tar.gz. Copy the link to that file, then back in your terminal for the droplet run

wget "https://github.com/ipfs/kubo/releases/download/v0.22.0/kubo_v0.22.0_linux-amd64.tar.gz"

This will download the Kubo zip file to your home directory. You can unzip it with tar -xf kubo_v0.22.0_linux-amd64.tar.gz and then you should see a folder just called “kubo.” cd into that folder then run sudo ./install.sh and that will move the binary from the folder into your /usr/local/bin folder. To make sure it worked, try running ipfs --version, it should show the version number if successful.

With IPFS installed on our server the next thing we need to do is create a systemd service aka a daemon. This will make sure that IPFS is always running and will start up automatically if we ever reboot the server. To do this you will want to either use myApp `sudo` with either vim or nano and create a file called ipfs.service under /etc/systemd/user/, so altogether would look something like sudo vim /etc/systemd/user/ipfs.service. Once the editor is open you can paste in the following:

[Unit]
Description=InterPlanetary File System (IPFS) daemon
Documentation=https://docs.ipfs.io/
After=network.target

[Service]
Type=notify
ExecStart=/usr/local/bin/ipfs daemon --enable-gc=true --migrate=true
ExecStop=/usr/local/bin/ipfs shutdown
Restart=on-failure
KillSignal=SIGINT

[Install]
WantedBy=default.target

Save the file and exit the editor, then run the following commands to start up the daemon and make it persist between logins:

ipfs init --profile=server --empty-repo
systemctl --user enable ipfs
systemctl --user start ipfs
systemctl --user status ipfs
loginctl enable-linger $USER

All of this together should have the IPFS node running, and you can test it out by running the following command

curl -L http://localhost:8080/ipfs/QmPyCYfL5oF79cfXjbt5cyr5hAZcyNrPNV9ytvUPdk8KT9

💡Keep in mind that with a fresh node like this with zero configuration might be slow and take a while to pull content and be connected with other major IPFS networks

Setting Up Custom Domain

Now that your IPFS node is setup and we can use the gateway, these next steps will help you assign a domain to the gateway and make it public. First you will need to acquire a domain name which you can get from multiple providers like Namecheap. For this tutorial we’ll use the example domain.com (very original). After purchasing the domain you will want to go into the advance DNS settings through your domain provider, and there we’ll add some records so we can use ipfs.domain.com as our gateway domain. You can get the IPV4 and IPV6 addresses from your Digital Ocean console.

‍

You can use a DNS checker for ipfs.domain.com to make sure everything is propagating but it can take some time depending on your provider.

After assigning the domain to the IP addresses of our droplet, we need to go in and edit our IPFS config.

⚠️WARNING:The following command will open your gateway and IPFS node for anyone to use, do so with caution and do further research before releasing it to the while.

You can paste

ipfs config Addresses.Gateway /ip4/0.0.0.0/tcp/8080

into the terminal and it will change the IP from your local network to the outside network, allowing external traffic to use it. We stress caution here because IPFS gateways are known to be abused which we’ll get into later. After changing that setting on the IPFS config we need to make one additional edit with vim ~/.ipfs.config. Once the file is open navigate under the following and add your specific domain and configs for UseSubdomains and Paths.

"Gateway": {
    "PublicGateways": {
        "domain.com": {
                "UseSubdomains": true,
                "Paths": [
                        "/ipfs"
								]
				}
		}
}

After we have written and saved those changes we’ll need to restart the IPFS daemon with systemctl --user restart ipfs. Then we can test if our custom domain works in our own browser with a link like this:

http://ipfs.domain.com:8080/ipfs/QmVLwvmGehsrNEvhcCnnsw5RQNseohgEkFNN1848zNzdng

Now keep in mind that we did not use https as that will come later, and you’ll also notice we had to include that nasty port number which is not very smooth. So let’s fix that!

We’ll use nginx to help with some re-routing on our server so we can just leave out the port number in our urls. Run sudo apt install nginx to get started. Once installed we will want to rename the default configuration as a backup with

sudo mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default-back

then create a new one by running

sudo vim /etc/nginx/sites-available/default

In that file you can paste the following:

server {
	listen 80;
	server_name ipfs.domain.com;

	location / {
    proxy_pass http://127.0.0.1:8080;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
	}
}

Write and save that file, then run systemctl restart nginx. If successful we can now use a url like this:

http://ipfs.domain.com/ipfs/QmVLwvmGehsrNEvhcCnnsw5RQNseohgEkFNN1848zNzdng

Setting up SSL (HTTPS)

As you saw in the last url we used, we’re still using http which is a no go in today’s standards. In order to fix that we need to get an SSL certificate for our domain. Thankfully its pretty straight forward with a package called certbot. You can install it with sudo snap install certbot --classic then run the command sudo certbot --nginx -d ipfs.domain.com. It should walk you though some questions you can answer, then it should issue a certificate for your domain. Last step is to go back to your domain provider and add this DNS record:

‍

Now you can test it out with

https://ipfs.domain.com/ipfs/QmVLwvmGehsrNEvhcCnnsw5RQNseohgEkFNN1848zNzdng

Congrats!! You just setup your own public IPFS gateway. But, something isn’t quite right: its super slow isn’t it? Let’s talk about that.

Further Steps

You have your public gateway setup and it sorta works,  but its also super slow. There are some things you can do to help relieve this. One of those things is setting up a cache layer or CDN to help make fetching files a second time much faster. You can also look into peering your gateway with IPFS pinning services to tap into their network and get faster speeds, or configure your IPFS node to work with the Distributed Hash Table (DHT) to assist with finding files. Even with all those things, it can be tough to maintain good speeds.

Another thing you have to consider when hosting an IPFS gateway yourself is abuse. The unfortunately piece of a decentralized network is that there is plenty of people out there who want to abuse public gateways by hammering them with requests for files or use your gateway for phishing content. When that happens you have to keep up with a list of CIDs to block from your gateway or risk it have it taken down by domain registrars. You can check out a list of CIDs to block by IPFS here, however it is no longer being maintained making it even more difficult to keep up yourself.

Another Option: Pinata Dedicated IPFS Gateways

With any open source software endeavor, you have to ask yourself an important question: “Is this worth my time?” There are plenty of things you can do yourself when it comes to networking, like setting up your own custom email server, and doing those things will help you learn a lot. However if you’re trying to heavily use and depend on IPFS as a service for your decentralized applications, then it becomes a different question. Even if you got the speeds of your gateway up to a decent level, would it be worth the expense and upkeep to keep running it yourself and hope it does not get abused?

This is why IPFS pinning services like Pinata exist: to make IPFS easy and simple for developers. With Pinata you can not only upload files easily, but with Pinata Dedicated Gateways you get unmatched speeds thanks to a built in 200 location edge cache CDN. You get the benefits of being hooked up to a large network of nodes instead of just your solitary node. Plus, you get Gateway Access Controls so you can access content on IPFS with protection from spam and abuse. Setting up this public gateway was fun, but in a production environment, I’m thankful to have Pinata :)

Happy Pinning!