IPFS as an Archival Solution For Farcaster

In September of 2022, Dan Romero wrote a blog post titled RSS+. I didn’t realize it at the time, but RSS+ would become Farcaster, one of the fastest-growing and highest-quality alternative social networks to spring up in the last couple of years. Farcaster sets itself apart from networks like Twitter because all data is open and accessible by anyone. Posts on Farcaster, called Casts, are stored in a network of storage hubs. Much like a blockchain network, these hubs all contain copies of the entire network state. Because of this design, anyone can build their own client on top of Farcaster. The most popular client is Warpcast, built by Romero’s team, but dozens of alternate clients have been built and some of them are even outpacing Warpcast in generating new sign-ups for the network.

The Farcaster Protocol

The design of the protocol is especially attractive to me because of how similar it is to IPFS. Pinata builds on top of IPFS as a protocol because of the open data promises it makes. The immutability and verifiability of data on IPFS makes the network a perfect companion to blockchains. However, considering the design of Farcaster, and some of the constraints built into the network (which we will discuss shortly), IPFS is also a perfect companion to the nascent social network protocol.

To understand why you might need an additional storage layer on top of a protocol that has built-in storage through its hub system, you have to understand a bit more of how the storage hub system works within Farcaster. To ensure the data stored on Farcaster storage hubs doesn’t grow at an unmaintainable pace, there are two baked-in tools:

1. Storage units, which cost real money
2. Pruning of data

Every user on the Farcaster network, regardless of the client application they use must purchase storage units. These units tell the storage hubs how much data tied to a user to keep. The pruning of data happens at the CRDT level of the network which ties in directly with the storage hubs.

From the specification:

CRDTs also prune messages when they reach a certain size per user to prevent them from growing indefinitely. The sizes are measured in units per user. The number of units of storage a user has is determined by the Storage registry. When adding a message crosses the size limit, the message in the CRDT with the lowest timestamp-hash order is pruned. Pruning should be performed once every hour on the hour in UTC to minimize sync thrash between Hubs. If all storage units expire for a user, there is a 30 day grace period before hubs will prune all messages for the user.

So what does this mean in practice? Users either need to purchase enough storage on the protocol to store all of their casts (and other associated data) or they must accept that the data will be pruned and lost forever. But, there’s a third option.

Archival Services

When discussing the purchasing of storage units, Romero casted a message about the combined pruning mechanism and the potential for archival services to preserve past data as a reason for people to not be so concerned about the cost of storage.

If we think back to the NFT boom, there was early concern that NFT data would be lost forever if people (the NFT creators usually) stopped paying their AWS bills. The concern was valid and the market shifted almost fully to storing NFT media and metadata on IPFS. In effect, IPFS became both the primary storage mechanism and the archival solution all in one. In the case of Farcaster, IPFS can slide in as a very simple archival solution.

IPFS isn’t the only option for archival. Anyone could permissionlessly build an archival solution for Farcaster on Amazon AWS, Google Cloud, Azure, or any number of traditional cloud services. The problem with building an archival solution in this way is it breaks the open data model. Forgetting about ethos and principles, the open data model of Farcaster is practical. Breaking that model effectively breaks the promise the entire network was built upon and limits the potential of the archived data. What’s more is traditional cloud providers, because they lack the open data model, require a single entity to maintain and preserve the data. Remember the NFT analogy? Stop paying your AWS bill, and the archived data disappears, even if you were archiving it for someone else. IPFS provides an alternative.

Pinata operates as an IPFS pinning service. That means someone (or an entity) pays Pinata to simplify the process of adding and keeping data on the IPFS network. Should someone choose to build an archival service for Farcaster on IPFS and use Pinata, wouldn’t everyone leveraging that archival service face the same problem as if it was built on traditional cloud platforms? Said another way, wouldn’t everyone that relied on that archival service lose access to their archived Farcaster data if the archival service stopped paying Pinata? Because of IPFS, the answer to this is no.

Should an archival service stop paying their bills, everyone with archived Farcaster data would be able to use Pinata, use another IPFS service, or run their own IPFS node to preserve their own archived data. They can do this without needing to ask for an export from the archival service or from the cloud provider. Because IPFS is open, anyone can preserve data at any time.

We saw this happen during the NFT boom when a large NFT marketplace decided to stop paying to keep NFT media and metadata pinned to IPFS. An entire community of artists and collectors rose up to pin all of the content themselves, keeping everything alive and accessible without a minute of “downtime”.

Even if specific services do not pop up to support archiving solutions, individuals can archive their Farcaster data in creative ways that also make use of IPFS.

Minting Casts

One suggestion that came up during the Farcaster storage unit and pruning conversation was the idea of minting casts as a way of memorializing them and preserving them. A minted cast would become an NFT with metadata representing the cast and maybe a screenshot of the cast itself as the NFT image.

Such a solution is slightly less practical than the archival service solution, but it’s not completely impractical. The metadata and the image would be stored on IPFS and linked to the NFT. The NFT would be minted on the chain of the user’s preference. As a mechanism for discovery and organization, maybe the metadata could include details about who the user was that created the cast. And maybe the NFTs could be organized in a global collection via a marketplace of choice.

The cost of minting an NFT and managing a collection makes this solution less practical than archival services. But again, it’s not completely impractical and it still leverages the open data model of IPFS and blockchain.

Wrapping Up

Farcaster’s design is intelligent and well-reasoned. This is true even of the decision to prune cast data. The design also opens up opportunities to create not just alternative primary clients for the protocol but archival clients. Combined with IPFS, Farcaster can provide an open data model we haven’t experienced in the social media space outside of the fediverse that Mastodon runs on.

IPFS is open and matches the expectations of Farcaster’s storage hubs without having to plug directly into the protocol. The opportunity for archival services for Farcaster is as strong as you believe the Farcaster network is.

I’d love to hear from you if you’re interested in building an archival solution. And if you’re not a builder but you think an archiving solution is something you’d like to see Pinata build, reach out as well.

Happy casting and happy pinning!