This is the story of the Deep Dive sessions at IPFS Camp 2019 as told from the perspective of a DWeb historical researcher in the far future.
In 2019, the intergalactic dweb uprising began. The pillars of what had become the foundations of human civilisation started to crack under their own weight and the very beginnings of a new empire began emerging from the shadows. An empire forged in the flames of discontent of central servers and “do-no-evil” monopoly. Peer to peer communications - the heart of the dweb - slowly formed efficient and autonomous networks that expanded beyond the reach of the blue planet and far into the galaxy, the likes of which the old world order had never seen before.
The uprising began modestly as a “space camp” in the ALPHA region 5P4-1N. From fear of ambush the exact coordinates were only divulged to the heroes that enlisted. What happened at that camp laid the foundations for the world as we know it today, but between then and now, many battles took place and we will forever honour the unbeknown dragon slayers that emerged from camp to lead the uprising and shape our new world.
This story attempts to recount a small part of camp, what was referred to back then as “Deep Dives” but what we later came to know as The Great Calamity Circumvention Assembly.
At this time the uprising was in its infancy. Incredible progress had already been made expanding peer to peer communications, content addressing and information permanence, but as unbelievable as it sounds, the dweb had still not yet expanded beyond the reach of the blue planet (what we once referred to as “Earth”).
Networks in those days were siloed and inaccessible. Data was equally unavailable and fragile and temporary. The transition from analogue to digital was still in living memory and appropriate steps had not yet been taken to ensure human history was permanently recorded. The worldwide data erosion seen in the Global Electromagnetic Fire Disaster of 2027 (just a handful of years after camp) proved this point and some estimates suggest that over 83% of all human history for that period was wiped out.
As you can imagine, there were a great many problems the dweb uprising was facing at that time. The Calamity Circumvention Assembly (“Deep Dives”) was formed at camp to address as many of those issues as possible.
Around 160 space rangers formed somewhere in the region of 30 splinter groups to focus on specific open problems around IPFS. As you know, IPFS - the InterPlanetary File System is one of the biggest proponents of the dweb and is currently the most widely used file system across the universe, used by over 31,000 known alien races. It was one of just a handful of dweb technologies at the time and later went on to play a monumental part in the dweb uprising.
The plan was to come up with new and inventive solutions to these problems. The enlisted space rangers were vetted to ensure they were experts in the dweb field and were suitably well placed to solve the issues at hand. In just 1 hour and 30 minutes (that’s 1000 hexsecs in our time) the splinter groups intensely discussed their assigned topics and documented a wealth of knowledge, ideas and observations that are still referenced in academia today.
The outcomes from each splinter group’s discussions were “pull requested” to the camp repository (A “pull request” was a term used at that time to submit additions/deletions or changes to a set of files). The following day splinter groups presented their solutions to the assembly for intergalactic approval.
Transcripts and videos from The Great Calamity Circumvention Assembly were salvaged from the Global Electromagnetic Fire Disaster of 2027 and we’ve transcoded them into formats appropriate for today’s dweb browsers. Enjoy!
|Scaling up the DHT 📦||What can we do to optimise the performance of our DHT? Are there alternatives?||Video|
|IPID - Master key security and user-experience 📦||IDM (Identity Manager) aims to support several DIDs methods and IPID was chosen as the first DID method to be integrated.||Video|
|Package Managers: What Now, What Next? 📦||Community evaluation and ideation session to recap the current (as of late June) state of the IPFS-enabled package manager landscape, discuss happiness and pain points, and do some ideation/co-creation around the direction of our next rounds of package manager efforts.||Video|
|IPFS on Mobile||Increasingly, everyday computing is happening in mobile environments. At the same time, many features of IPFS and related technologies work best on an ‘always-connected’ network that simply doesn’t exist in the mobile world. In this deep dive, we will explore the issues inherent to p2p systems operating in mobile environments, discuss some existing solutions, and hopefully outline a research/development agenda that will help catalyze a wave of new IPFS-based mobile solutions.||Video|
|Private content on IPFS 📦||Can we have a private group chat implemented on IPFS?||Video|
|Building the IPFS Community||Community, evangelism, engagement, on ramp.||Video|
|IPFS for websites (static, dynamic, search, etc)||What are the best practices and open questions for using IPFS for static / dynamic websites? What about indexing and searching?|
|Revocation/Rotating of IPNS Keys||The idea of using a public key to identify mutable data is cool, but what happens when that key becomes compromised?||Presenter Opted Out of Recording|
|How can IPFS support sharding 📦||How to best pin an arbitrary part of a DAG.||Video|
|Replication protocol 📦||How can we more quickly sync a DAG or part of a DAG between two nodes? A compliment to bitswap, affectionately overloaded as GraphSync||Video|
|Do not replicate lists - DNR||How to stop your node from replicating content due to take down notices or other reasons||Video|
|Fast IPNS 📦||How to increase the IPNS resolution speed||Moved to Lightning Talk|
|IPFS Education and Documentation||What are the biggest improvements we can make to documentation for IPFS?||Video|
|Mounting an IPFS filesystem 📦||Is there a way to efficiently mount IPFS as a filesystem? FUSE support is here. What are current limitations, challenges? What needs to happen to enable mounting of MFS? WebDAV is often used in network storage drives, and mainstream operating systems come with native support for mounting WebDAV as drives. It could supersede Writable Gateway and create a surface for interop with apps that provide built-in client-side support for WebDAV.||Video|
|IPNS link rot||Right now, IPNS needs to be refreshed by original publisher every day, otherwise it stops working. There should be a way to pin IPNS address so that other peers keep refreshing the last valid entry when original node goes offline, keeping the last version alive even if the original publisher disappears forever.||Video|
|Deterministic CIDs! Reproducible File Imports! Verifable HTTP Gateways!||How to deterministically get the same CID for the same data, even if default parameters of
|IPFS in Web Workers||Where we are, limitations, known problems.||Video|
|UnixFS v2 📦||Latest spec iteration and implementation work||Video|
|Using CRDTs to build real-time collaborative DApps||What are the hurdles to using CRDTs in dapps with IPFS? How can we make peer-base easier to understand and use?||Video|
|Better NAT traversal so that Relay servers are a last (not first) resort||The current cost of running relay servers for IPFS is unsustainable. We need to implement better NAT hole punching mechanisms in libp2p to reduce the usage of relay servers by NAT’d hosts. TCP hole punching has a success rate of ~60%, so we should be preferentially hole punching. TCP simultaneous open is one example. STUN servers are another example; yet another is making every non-NATed host in a libp2p network a relay node.||Video|
|libp2p message orientation||libp2p is currently stream-oriented. We want to enable message-oriented communication (e.g. UDP). Help us envision what the APIs would look like, and how an application could simultaneously hold stream-based and message-based conversations with peers in the world via libp2p.||Did not Present|
|IPFS metrics and usage statistics 📦||How to collect telemetry like application metrics and usage in a distributed world||Video|
|Decentralized Data Processing||How could we process large amounts of data in permissionless networks? Use cases, verifiable state transitions, aggregations and bulk updates, etc.||Video|
|Building Dynamic P2P Applications||IPFS and its associated technologies are mostly static. Radicle is an attempt to build a dynamic P2P web on top of IPFS; one which can host applications with the same abilities of the centralized servers of today to receive inputs from various people or sources.||Video|
|Better file chunking 📦||Within the IPFS stack/ecosystem, just as within computing as a whole, an uncompressed stream of untagged octets is a fundamental unit of exchange. As a general-purpose data storage system IPFS needs to handle an unbounded variety of content represented by such streams. Handling the maximum amount of this variety efficiently ( ideally by default ) would likely have an outsized impact on the future adoption of IPFS as a long-term data interchange medium/format.||Video|
|Dynamic DAG traversal||Extend IPLD to cross between immutable and mutable references||Video|
|Design Patterns for DWeb Apps||Most IPFS apps have UX issues. We will pick a feature of an existing app, and improve its UX by simplifying how to interact with it, how it works. Textile.io lowered the permission and roles system from ~15 methods to 2 this way!||Video|
|IPFS and Guix 📦||Exploration into existing or potential issues with integrating IPFS with Guix||Video|
There are more to come, keep posted!
More content from IPFS Camp is being salvaged and transcoded.
If you are interested in receiving an update when the next batch of videos are available, you can do this in one of the following ways:
- Watch and Star the ipfs/camp repo, where all the content will live, in addition to other other locations (IPFS 😄) shielded from the Global Electromagnetic Fire Disaster of 2027
- Subscribe to the RSS feed of this blog.
- Subscribe to the IPFS Weekly Newsletter if you are interested in general IPFS updates.
- Subscribe to the IPFS Events Newsletter if you are interested in IPFS events.