Summary

Well, this has been a week of big news for us. By now you’ve probably already heard that our core dev team has lifted the curtain on all the refactoring work we’ve been doing on Holochain. If you haven’t, read on for the highlights.

Highlights

  • Holochain RSM goes public
  • What are the next steps?
  • What does RSM mean for hApp developers?

Holochain RSM (Alpha) goes public

As regular readers know, Holo Host Alpha testing has held Holochain to the fire in ways that automated tests never could. We’ve learned about performance bottlenecks, architectural choices that impede debugging, and of course bugs in the code. Very early on in alpha testing, we were forced to consider whether to heavily refactor the current codebase or take our learnings into a complete rewrite.

We’ve decided on the latter, and we’re calling it Holochain RSM (Refactored State Model). In only six months of work the new codebase has surpassed the old one in terms of core features, while improving performance, maintainability, and correctness.

The biggest change, as the name suggests, is the removal of the internal state model inspired by React’s Redux pattern. The core dev team found that the extra abstraction layer over the base Holochain state model was making it hard to reason about correctness and hunt down bugs. This became especially plain over the past few months of HoloFuel testing as we identified many ‘holding list discrepancies’, AKA ‘consistency bugs’.

Now there’s no extra layer of abstraction between the conductor’s picture of application state and its representation on disk. Not only is this more straightforward (which means existing and future core contributors have less to think about), but it’s more robust in the face of software crashes and power outages. And we’re much more confident that our code correctly implements the formal specification.

RSM is a lot easier to hack on!

Other changes are the kind you’ll be able to feel:

  • Wasmer, the new WebAssembly engine, is two orders of magnitude faster in compilation and execution times than the previous one, and its different design also lets us simplify internal plumbing and save on memory usage.
  • The ‘cascade’, a new caching mechanism, hits local data first, favouring availability over freshness. This means that locally written and recently retrieved entries will load instantly without a network call, which is usually the best choice for human-facing applications.
  • We’ll be using QUIC rather than WebSocket for node-to-node communication; this will allow busy pairwise connections (such as between DHT neighbours) to multiplex their transfers and prevent parallel tasks from blocking each other. QUIC is also faster over unreliable connections because reconnecting takes less time.
  • We’re making much better use of LMDB to offload things from memory and retrieve data quickly.
  • Putting all of our changes together, we’re anticipating up to 2000× performance increases, 40× less memory usage, 2500× less network chatter, and faster compile times compared to the previous version of Holochain.

Speaking of networking, the new Holochain comes with a fully implemented DHT, built around the model of neighbourhoods of responsibility that distribute load and thwart collusion. While the QUIC transport part isn’t finished yet (presently instances can only talk to each other within one conductor), it’s already closer to full P2P than the previous version was. That means sim2h, the centralised switchboard server we’ve been using, goes away completely. (Note: At first all nodes will direct their traffic through a central proxy to reduce the number of variables to debug, but QUIC’s end-to end TLS encryption means the proxy won’t be able to inspect traffic. This wasn’t true of sim2h.)

All non-maintenance development effort has already been focused on RSM for quite a while now. Now that it’s public, our dev activity dashboard finally reflects what’s actually been happening in the org.

What are the next steps?

So far we’ve only made the RSM repo public, updated the Tryorama test framework, and written a new JavaScript client. There’s a lot we have to do now:

  1. We have one or two more bugfix releases of the old Holochain, which we’re now calling Holochain Redux. After all, we’re still using it in Holo Host Alpha testing.
  2. In the meantime, we’ll be porting our Holo hApps (HoloFuel, Service Logger, Host Console, and Publisher Portal) and HoloPort OS to use RSM.
  3. RSM still needs a few API features that were available in Redux, such as bridging, app-level data encryption, and signals.
  4. We have a bit more work to do to get developer tooling up to the same level as the previous version.
  5. Once Redux is officially deprecated, RSM will get a version number and be published to holochain.love (the Holonix dev tools distribution) and crates.io.
  6. Documentation and dev support are a work in progress. Currently it’s a bunch of READMEs, HackMDs, and notes from devs who’ve already been porting their hApps to RSM; we need to retire the current documentation and put up a basic site with just enough info to get you started. The API and HDK are already well documented; we’ll be publishing them to docs.rs once RSM is the officially supported version.
  7. When RSM is well settled in the developer community, we’ll move on to new features like built-in countersigning, extensibility, a user-friendly runtime, and core hApps.

What does RSM (Alpha) mean for hApp developers?


Note: This is not a release; it’s a sneak peek! Early-access partners have been using it for a month or two already, but we can’t guarantee it’ll always work for you until we start cutting releases. In particular, we’ve seen evidence of a recent regression that causes installation to fail on macOS. The repo’s main readme will be updated with special instructions as we figure this issue out.

New developers

If you’ve taken a DevCamp and haven’t started building yet, or if you’re totally new to Holochain, we encourage you to start on the right foot and learn RSM. You can read the Redux documentation, but know that, beyond the universal Holochain concepts, a lot of it will become obsolete. If you start with RSM right away, the first steps of your journey will be a bit rocky, but it’ll get better soon as we roll out tooling and documentation.

Developers with existing hApps

The core API, HDK, and admin API have changed. In general this is a good thing; devs with early access to RSM have told us that they can rewrite their DNAs’ functionality in about ⅓ of the space. But it does mean you’ll have to rewrite your DNA source code. Look for a migration guide on the Holochain Open Dev blog soon!

All developers

There’s a lot that’s changed, and the API is still unstable (though it’s stabilising rapidly). And as I mentioned the documentation is still being written. In the meantime, if you want to jump in before the official release, check out these resources from the team and the community:

I also encourage you to go to the Holochain Forum and connect with us and the community of hApp devs. Some of them have been given early access to the RSM repo and have been building with it for a couple months. We’d be happy to help you get up to speed!

Development status

Latest

Blessed (available via https://holochain.love)

  • Holonix: 0.0.81
  • Holochain Core: 0.0.51-alpha1
  • hc-happ-scaffold: 0.1.0
Install Holochain Learn about Holochain Join the Discussion