Holochain’s 0.1.x series has a new release which fixes some breaking changes in a well-known dependency called serde. The Launcher has a new release for the unstable Holochain 0.2.1 release as well. After those release notes, I’ll give an update on the state of Holochain 0.2.x.

But first! A reminder about the upcoming dev training.

Holochain Developer Training Oct 2023

In case you missed it, we’re co-hosting another app dev training course with our education partner Mythosthesia, online, from 2 to 7 October. This is a very hands-on, very cooperative, and fairly intensive crash course on hApp development, led by Marcus Philips, a veteran of world-class dev school Hack Reactor, and Guillem Cordoba, a knowledgeable and friendly hApp developer. Registrations are closing very soon, so apply now so we can guide you through the intake process and get you prepared to take the course! If the registration cost is a challenge for you, please reach out to us on Discord — there are scholarships available.

Holochain 0.1.6: Fix serde issue

Release date: 28 August 2023
HDI compatibility: 0.2.x
HDK compatibility: 0.1.x

Recently the maintainers of serde, a crate that provides support for serialising and deserialising Rust types, accidentally introduced a breaking change to the way enums are serialised. Our core dev team has decided to pin serde at an earlier version to avoid that change from breaking our APIs as well. (We’ll be introducing that breaking change in the 0.3.x series.)

This issue isn’t likely to affect you unless you had run cargo update on your hApp in the time window between the change in serde and the release of HDI 0.1.4 and HDK 0.2.4 (both from the Holochain 0.1.6 release). If this is the case, you can run nix flake update on your project and then run cargo update again.

We discovered and patched the issue in the scaffolding templates pretty quickly after discovering the issue, so if you’ve used the scaffolding tool to generate a zome you’re probably safe.

Here are the full release notes:

  • Maintenance: Pin serde to 1.0.166 to roll back a breaking change to enum serialisation. (#2674)
  • Maintenance: Update lair to 0.3.0, which introduced no substantial changes but pinned serde_derive to 1.0.166. (#2668)
  • Removed: A compile-time feature to include support for tx5, Holochain’s new WebRTC-based networking protocol, has been removed.

Launcher 0.11.0: Holochain 0.2.1 support, new DevHub

Note: This Launcher is bundled with an unstable Holochain version. For those of you who are testing and deploying hApps at small scale, this is an opportunity to try your hApp with Holochain 0.2 and use the newest App Store and DevHub.

What’s new about these two features? Let’s back up a bit and talk about what DevHub is first. It’s a package manager hApp that lets devs distribute zomes, DNAs, hApps, and UIs. It comes preinstalled in the Launcher and powers both the App Store feature and the DevHub UI. And in the future it will be integrated into the Holo suite to allow ‘hApp managers’ — either developers or people forking a hApp for their group’s personal use — to get their hApps onto HoloPorts.

The original DevHub had a few DNAs that worked together as one to provide all package data — both blobs and metadata such as manifests. Everyone who installed the Launcher ran this DNA, which meant everyone had to store app packages — and in a small DHT where everyone stored everything, that meant that you had to wait until someone had gossiped a package or its metadata to you before you could install an app. This caused long load times and failures, making it seem like there were no hApps in the store and then making it seem hApps couldn’t be downloaded.

The new DevHub is broken into two hApps with multiple DNAs:

  • The original DevHub DNA(s) live in one hApp, which is only activated if you turn on the ‘developer mode’ switch in the Launcher settings.
  • An App Store DNA which only contains manifest files for hApps deemed ready for public use (sort of like a release tag on GitHub) is in another hApp, which is activated for all users.
  • A portal DNA is bundled with both of the above hApps, and it handles communication between the two of them.

If a peer belongs to the DevHub DHT, they’ll advertise that fact on the portal DHT. You could call them a package host, because they’re participating in the storage of package blobs. Anyone who wants to download a package can look up that list of hosts, choose a random one, and ask it for the package data.

This is a great example of a multi-DNA architecture that breaks up responsibility according to needs, abilities, and roles in the community. The portal DNA is a generic DNA that you can drop into your hApp to provide the bridging between a DHT of hosts and a DHT of clients. In the future, Holochain will support ‘deferred DNA provisioning’ which will eliminate the need to split this sort of functionality into two separate hApps, allowing you to bundle the host DNA and only activate it for those who want to be hosts.

There are a few things to keep in mind:

  • This release is bundled with Holochain 0.2.1, which is unstable.
  • Because of an upgrade to the Lair keystore, installing this Launcher will be similar to a factory reset. You’ll need to reinstall hApps. (This is a breaking change, not a bug.)
  • hApps may be visible in the App Store, but installation will fail if there are no available DevHub peers that are storing the requested package. This isn’t a bug but a side effect of small DHTs. We’re testing the deployment of persistent DevHub and App Store nodes on the Holo network, and that will help with this issue in the near future.
  • There are a few known issues, which you can read about in the release notes.

Download Launcher 0.11.0 (unstable) from GitHub.

Holochain 0.2 progress: building comprehensive diagnostic capacities

tl;dr: While Holochain 0.2 is working just fine for hApps used by small groups of people, we’re seeing bottlenecks when networks and their data loads grow. We’re nearing a stable 0.2.2 release which tackles some of the biggest performance issues, but further improvements will still come after that point.

As a developer, you’ve probably had the experience of polishing up your application, running it through CI tests one more time (all green!), cutting a release, and… getting a bug report from someone who hit some obscure corner of your code or tried to use it in OS or networking conditions you’d never dreamed of. Even the most well-tested code is vulnerable to the kind of creative challenges the real world likes to throw at it.

Holochain’s CI test suite has checks for performance regressions — it generally has to pass a certain level of responsiveness before a release can be cut. These tests exercise the conductor fairly aggressively. Even still, when we released it to the world, it struggled to keep up with real-world usage scenarios, as my colleague Eric wrote about recently.

Why did this happen? Simple: the real world is messy. It’s hard to write tests that anticipate the issues you’ll see out there. Many of them will be ‘black swans’ — situations that are inherently impossible to predict. For example, our automated tests didn’t anticipate the performance issues that could happen when network communication is thin and unreliable.

And that, of course, is exactly the kind of situation you encounter in the real world. It turned out that, in large networks with lots of data, network latency and bandwidth caused gossip bottlenecks at unexpected moments. Aborted and restarted gossip sessions were creating race conditions, causing network connection pool saturations that took a while to clear out.

So of course the core devs are tackling these specific bottlenecks. But they’re also building their capacity to respond to future issues of this kind. You can’t predict the next black swan, but you can work on your ability to respond to the unexpected. This is happening in two ways:

  • Application architecture is getting streamlined so that it’s easier to reason about the flow of control and data through a node. This will make it possible to find out exactly what’s happening during a bottleneck, capture the inputs, outputs, and system state, and replay the scenario.
  • As real-world applications encounter issues, the applications and the scenarios that cause the issues will be integrated into our CI suite to catch similar issues in the future.

This is just boring engineering stuff — stages two and three in Agile pioneer Kent Beck’s advice to “make it work, make it right, make it fast”. We’ve already made it work, from proof of concept to first stable beta. Holochain now correctly implements a very large portion of its roadmap. The task now is to make it right — which is not about fixing fundamental defects but instead refactoring the code to make it easier to test.

And that in turn will allow us to make it fast. Holochain is a framework for building everyday apps that serve real human needs through good times and bad. In order to do this, it needs to feel good to use. In ideal situations Holochain already feels fast enough — we’ve already been using hApps like Acorn, KanDo, and TalkingStickies for months in our daily work. Now it’s time to make it work well in all those non-ideal situations.

The core devs have already made a lot of progress. We’re anticipating a release of 0.2.2 soon, which we’ll declare the next stable release after 0.1.6. It will include fixes for a zome call timeout issue, along with various improvements they’ve already made to the networking layer. The first release candidate is already cut, and our next step is to stress-test it with a few flagship apps before deciding if it’s ready. 0.2.2 won’t fix all possible performance issues, but it will be an incremental-yet-noticeable improvement.

You can hear Eric and the core devs talk more about our findings and plans in the recording of our first Developer AMA:

Cover photo by ThisisEngineering RAEng on Unsplash

Install Holochain Learn about Holochain Join the Discussion