IPFS has proven to be the decentralized storage protocol of choice by many blockchain developers, and one of the crucial tools used to access content on IPFS are Gateways. IPFS Gateways are like bridges between the IPFS protocol and the HTTP protocol that we use everyday to browse websites. There are lots of different options to choose from when it comes to IPFS Gateways, and in this post we’ll show you how to host and build your own!
In order to follow this guide you’ll need a few things. First and foremost you’ll need a decent amount of experience using Linux servers and navigating around in the terminal, things like creating daemons or editing text files with vi or nano. You’ll also need a cloud server provider, and there’s plenty to choose from. In this guide we’ll use Digital Ocean and get a simple droplet. Also if you want to have a custom domain instead of using an IP address you can get something through a domain provider like Namecheap.
Setting Up the Server
Before we rent a server for our IPFS node, you’ll want to create an SSH key to login with. This is the preferred secure way to SSH into your server versus a user name and password. You can check out this guide on how to create them.
It will prompt you to put in a passphrase to access the key so choose something secure, and once created it will make a file located in <span class="code-inline">~/.ssh/rsa.pub</span>. We’ll use the contents in just a moment when we setup the server.
Since we’ll be using DigitalOcean you can head over there to create an account and buy a Droplet. You definitely don’t need anything crazy, I just got the following:
For the authorization select SSH Keys, then copy and paste the contents of <span class="code-inline">~/.ssh/rsa.pub</span> and paste it in as a new key.
After the droplet has been created, you will actually want to turn it off, go to the Network settings, and enable IPV6. Once enabled turn it back on and try to SSH into it with the following command with the IPV4 address of the server:
It should prompt you to enter in the passphrase for you SSH key, and after entering it you should be in!
While we are signed in, we are currently logged in as root, which is not the most secure practice. This next step is optional, but highly recommend. First we’ll create a new user with the command <span class="code-inline">adduser steve</span>, and of course you can use whatever username you want to. It will prompt you to make a new password and for some other information you can leave blank. Next we need to give the user the permissions necessary to run the IPFS node with <span class="code-inline">sudo usermod -aG sudo steve</span> (and of course from this point on replace <span class="code-inline">steve</span> with the username you chose).
Next we’ll need to run the following commands to create an <span class="code-inline">.ssh</span> directory for our new user and give proper permissions so we can edit it.
With those commands completed you can now login as your user with <span class="code-inline">su steve</span> and then edit the SSH keys files to paste in your own that we used earlier with either <span class="code-inline">vim</span> or <span class="code-inline">nano</span> then <span class="code-inline">/.ssh/authorized_keys</span>. After pasting in your key you can run <span class="code-inline">exit</span> to log out of the user, then again to leave the SSH session. Now you should be able to SSH in with the new user <span class="code-inline">ssh steve@ipv4address</span>.
Once you’re in your server you can run <span class="code-inline">sudo apt update</span> just to make sure all your packages are up to date. Then you will want to visit the release page for IPFS Kubo, the Go implementation of an IPFS node used pretty much everywhere. On that page you can choose the latest stable release, then locate the correct distribution for your OS. In my particular case it ended up being <span class="code-inline">kubo_v0.22.0_linux-amd64.tar.gz</span>. Copy the link to that file, then back in your terminal for the droplet run
This will download the Kubo zip file to your home directory. You can unzip it with <span class="code-inline">tar -xf kubo_v0.22.0_linux-amd64.tar.gz</span> and then you should see a folder just called “kubo.” <span class="code-inline">cd</span> into that folder then run <span class="code-inline">sudo ./install.sh</span> and that will move the binary from the folder into your <span class="code-inline">/usr/local/bin</span> folder. To make sure it worked, try running <span class="code-inline">ipfs --version</span>, it should show the version number if successful.
With IPFS installed on our server the next thing we need to do is create a <span class="code-inline">systemd</span> service aka a daemon. This will make sure that IPFS is always running and will start up automatically if we ever reboot the server. To do this you will want to either use <span class="code-inline">myApp</span> `sudo` with either <span class="code-inline">vim</span> or <span class="code-inline">nano</span> and create a file called <span class="code-inline">ipfs.service</span> under <span class="code-inline">/etc/systemd/user/</span>, so altogether would look something like <span class="code-inline">sudo vim /etc/systemd/user/ipfs.service</span>. Once the editor is open you can paste in the following:
Save the file and exit the editor, then run the following commands to start up the daemon and make it persist between logins:
All of this together should have the IPFS node running, and you can test it out by running the following command
Setting Up Custom Domain
Now that your IPFS node is setup and we can use the gateway, these next steps will help you assign a domain to the gateway and make it public. First you will need to acquire a domain name which you can get from multiple providers like Namecheap. For this tutorial we’ll use the example <span class="code-inline">domain.com</span> (very original). After purchasing the domain you will want to go into the advance DNS settings through your domain provider, and there we’ll add some records so we can use <span class="code-inline">ipfs.domain.com</span> as our gateway domain. You can get the IPV4 and IPV6 addresses from your Digital Ocean console.
You can use a DNS checker for <span class="code-inline">ipfs.domain.com</span> to make sure everything is propagating but it can take some time depending on your provider.
After assigning the domain to the IP addresses of our droplet, we need to go in and edit our IPFS config.
You can paste
into the terminal and it will change the IP from your local network to the outside network, allowing external traffic to use it. We stress caution here because IPFS gateways are known to be abused which we’ll get into later. After changing that setting on the IPFS config we need to make one additional edit with <span class="code-inline">vim ~/.ipfs.config</span>. Once the file is open navigate under the following and add your specific domain and configs for <span class="code-inline">UseSubdomains</span> and <span class="code-inline">Paths</span>.
After we have written and saved those changes we’ll need to restart the IPFS daemon with <span class="code-inline">systemctl --user restart ipfs</span>. Then we can test if our custom domain works in our own browser with a link like this:
Now keep in mind that we did not use <span class="code-inline">https</span> as that will come later, and you’ll also notice we had to include that nasty port number which is not very smooth. So let’s fix that!
We’ll use nginx to help with some re-routing on our server so we can just leave out the port number in our urls. Run <span class="code-inline">sudo apt install nginx</span> to get started. Once installed we will want to rename the default configuration as a backup with
then create a new one by running
In that file you can paste the following:
Write and save that file, then run <span class="code-inline">systemctl restart nginx</span>. If successful we can now use a url like this:
Setting up SSL (HTTPS)
As you saw in the last url we used, we’re still using http which is a no go in today’s standards. In order to fix that we need to get an SSL certificate for our domain. Thankfully its pretty straight forward with a package called certbot. You can install it with <span class="code-inline">sudo snap install certbot --classic</span> then run the command <span class="code-inline">sudo certbot --nginx -d ipfs.domain.com</span>. It should walk you though some questions you can answer, then it should issue a certificate for your domain. Last step is to go back to your domain provider and add this DNS record:
Now you can test it out with
Congrats!! You just setup your own public IPFS gateway. But, something isn’t quite right: its super slow isn’t it? Let’s talk about that.
You have your public gateway setup and it sorta works, but its also super slow. There are some things you can do to help relieve this. One of those things is setting up a cache layer or CDN to help make fetching files a second time much faster. You can also look into peering your gateway with IPFS pinning services to tap into their network and get faster speeds, or configure your IPFS node to work with the Distributed Hash Table (DHT) to assist with finding files. Even with all those things, it can be tough to maintain good speeds.
Another thing you have to consider when hosting an IPFS gateway yourself is abuse. The unfortunately piece of a decentralized network is that there is plenty of people out there who want to abuse public gateways by hammering them with requests for files or use your gateway for phishing content. When that happens you have to keep up with a list of CIDs to block from your gateway or risk it have it taken down by domain registrars. You can check out a list of CIDs to block by IPFS here, however it is no longer being maintained making it even more difficult to keep up yourself.
Another Option: Pinata Dedicated IPFS Gateways
With any open source software endeavor, you have to ask yourself an important question: “Is this worth my time?” There are plenty of things you can do yourself when it comes to networking, like setting up your own custom email server, and doing those things will help you learn a lot. However if you’re trying to heavily use and depend on IPFS as a service for your decentralized applications, then it becomes a different question. Even if you got the speeds of your gateway up to a decent level, would it be worth the expense and upkeep to keep running it yourself and hope it does not get abused?
This is why IPFS pinning services like Pinata exist: to make IPFS easy and simple for developers. With Pinata you can not only upload files easily, but with Pinata Dedicated Gateways you get unmatched speeds thanks to a built in 200 location edge cache CDN. You get the benefits of being hooked up to a large network of nodes instead of just your solitary node. Plus, you get Gateway Access Controls so you can access content on IPFS with protection from spam and abuse. Setting up this public gateway was fun, but in a production environment, I’m thankful to have Pinata :)