Summary

This week David Atkinson had an interview with Dr Adam Bumpus and Mike Gamble of smart energy startup RedGrid, about their team’s success switching their codebase over to Holochain RSM. They’ve had some very exciting results.

Speaking of RSM, I’ve seen some solid developments in the last couple weeks regarding key management and validation.

Lastly, our hardware partner has gotten the long-patiently-awaited Holopot Nano booting from our supplied image and is sending out sample Nano units to team members for testing on the Holo network soon.

Highlights

  • Mike Gamble of RedGrid: “so impressed with the turnaround of results”
  • Holochain RSM developments
  • HoloPort Nanos running HPOS; test units going out soon

Mike Gamble of RedGrid: “so impressed with the turnaround of results”

[Errata: second paragraph changed to correct some mistakes re: RedGrid's partnerships.]

RedGrid, a startup focusing on smart energy grid technology, believed that decentralisation would be fundamental to greening the grid. That meant their ‘smartness’ layer ought to be decentralised too. Initially drawn to blockchain, they quickly realised that it was too slow and not actually all that decentralised. When they discovered Holochain, they decided it was an ideal match.

They persevered with us through a protracted rewrite of Holochain, maintaining faith in our vision. But it hasn’t been easy, especially when they couldn’t coax out the necessary performance. Monash University, one of their major partners, put RedGrid’s Redux-based ‘Transactive Energy Market’ DNA to the test and found that data could only be committed at the rate of one entry per several minutes. (One outlier even took 17 minutes.)

RedGrid reached out to the Holo team, got invited early into the RSM repo, and (with the help of other generous developers in the ecosystem) rewrote their codebase within a month. This whirlwind of activity paid off: now, the same test supports double the number of nodes, committing ~87 entries per second without breaking a sweat.

In the words of Mike Gamble, RedGrid’s principal architect, Monash was “so impressed with the turnaround of results” that they’re now considering stripping out the message bus they’re using for other components and replacing it with Holochain as the “backbone”.

I’m looking forward to sitting down soon to talk with Mike to ‘talk shop’ about their experience and share more details in a future post. In the meantime, here’s a graph of how the Transactive Energy Market DNA now performs with Holochain RSM:

Yes, you read that right. 15,000 times speedup, not 15,000 percent.

Holochain RSM (Alpha) developments

Watching the various chat channels where the core devs work—both within the team and with our closest allies in the ecosystem—has been a thrilling experience for me. The work has felt so rapid yet so solid; features that previously took many months now seem to get built while I’m fixing my morning coffee. A few such developments have happened since we announced RSM two weeks ago. (Warning: Holochain educational deep dives ahead!)

Keystore finished

The Lair keystore, which manages all your secrets and signs data for you, is completed and integrated through the whole conductor. In RSM it’s used by the conductor for agent identity and by the network layer for TLS encryption. This means that the conductor can now generate agent keys and sign data with them, and the host functions `sign` and `verify_signature` work. `encrypt` and `decrypt` will come later. (Keystore functions from Holochain Redux, such as `keystore_derive_key`, will also come later. It’s likely you haven’t been using them anyway, though, because they’re mainly meant for DPKI implementations such as DeepKey.)

The new keystore is a separate binary, which makes it easier for us to manage the security of in-memory secrets in one place. It’s meant to be a long-running service that unlocks your secrets with a single password and makes them available to the appropriate services. (If you’re an SSH user, this is similar to Pageant or ssh-agent.)

Validation nearing completion

The next is not so much a feature as a collection of things needed for the most crucial part of Holochain: peer validation, the principle that allows a network of devices to have an ‘immune system’. In addition to a lot of low-level plumbing that ensures things are validated in a correct and efficient order, a few important dev-facing components have landed recently:

Validation callbacks are implemented, along with a follow-up to make them more ergonomic. In RSM, a validation callback is just a regular function that you can place anywhere in your zome code. As long as it’s publicly exposed to the host and follows a naming convention, the host will know how to call it: it starts by looking for the most specific callback, e.g., `validate_create_entry_blogpost`, then falls back to increasingly less specific ones until one of them returns an error. If you choose not to write any particular callback, that’s fine; it’ll just keep trying down the specificity cascade. (Note: you can also validate capability grants/claims and agent public keys, and create/update/delete actions. You can also validate link creates and deletes, which have no generic fallback. In the future you’ll also be able to validate agent membrane proofs (‘invite codes’) and DNA migration entries.)

Chain rollback checking is implemented. This feature might sound obscure, but to me it’s a huge part of what makes Holochain Holochain. Most apps will want the guarantee of a linear, unbranched, cryptographically signed history for every participant—imagine the chaos of a voting app that lets you ‘fork reality’ by voting for both Biden and Trump! So part of the work that the immune system does is ‘remember’ everything an agent has ‘said’, that is, all the headers she’s written to her chain. The ‘agent activity’ validation authorities, or the nodes that are called on to record her history, are her neighbours in the DHT. If they detect two headers with a common parent, that means the agent has tried to roll back and fork her chain. This was never present in Holochain Redux, and it takes us a good distance towards full handling of chain rollbacks.

The other three components are:

  • ‘Slow heal’ gossiping, in which agent activity authorities gossip with each other, thus being able to detect partition attacks. This is already implemented.
  • A host API function that will let any agent ask another agent’s activity authorities for her full history plus outstanding warrants. This will save honest counterparties from entering into a fraudulent transaction.
  • Warrants, which will act as antibodies to accelerate a response against a bad actor.

Lastly, the groundwork has been laid for validation packages. A validation package consists either of the entry and its header (an ‘element’), the author’s full chain prior to that element, or only the chain elements with the same entry type. This is a special type of dependency that the author of an entry is expected to collect, because they are the authority for their own source chain. In the future, the network will be able to compile this information by consulting the author’s neighbours.

Signals coming soon

Holochain Redux had `emit_signal()`, which pushed messages to the UI. This is critical for allowing a human to respond to a node-to-node message, such as when Alice wants Bob to respond to a transaction offer. Signals are being worked on for RSM and should land soon.

Networking and proxies in the works

As mentioned in the last Dev Pulse, networking is still a work in progress. (Correction: I got a bit too eager and said the neighbourhood sharding model is fully implemented; apparently it’s still being worked on.) The transport layer is the current focus, laying the foundation for nodes to choose another node to proxy their information for them. The proxy will blindly forward end-to-end-encrypted data between the other two peers. Because anyone can be a proxy, this reduces centralisation. This is a simple workaround for the problem of restrictive firewalls, which is one of the big obstacles that P2P networks face.

HoloPort Nanos running HPOS; test units going out soon

We’ve gotten news from our hardware partner that they’ve (finally) gotten around their networking issues and managed to use our bootstrapping script to build an OS image for the HoloPort Nano. This image has been shown to work, so they’re shipping out sample Nano units to a few team members for testing. Once we fix whatever the testing reveals, we plan to start shipping Nanos soon!

Development status

Latest

Blessed (available via https://holochain.love)

  • Holonix: 0.0.81
  • Holochain Core: 0.0.51-alpha1
  • hc-happ-scaffold: 0.1.2
Install Holochain Learn about Holochain Join the Discussion