The past few Holochain releases have focused on both increasing resilience and adding features for developers.

One task is particularly important, even though you’ll probably never see or experience it — and in fact, its invisibility will be the proof of its success. Right now, the core dev team is doing extensive sharding tests, tuning parameters, adjusting the algorithm, and testing again. Currently, they’re investigating how peers respond to dramatic changes in their neighbourhood, such as many neighbours leaving at once or adjusting their arcs of responsibility.

What they’re seeing is that nodes respond very quickly to changes in their neighbourhood — which is great in principle, but such rapid responses are causing ‘oscillations’ that take a while to calm down. It’s a bit like when you meet someone coming toward you in a hallway, then you both try to make way for each other but end up moving in the same direction. So you readjust, get in each other’s way again, and repeat a few more times before you figure it out. That’s what’s happening at the DHT level, and the dev team is working on strategies to dampen those oscillations and allow nodes to reach stability after disruptions.

As regular readers know, sharding is Holochain’s long-term plan for building scalable peer-to-peer networks. Sharding is already built into Holochain, but it’s currently disabled by default. This testing and improvement work is getting it ready for real-world use.

Now let’s dig into what’s happened in the last three Holochain releases.

Holochain 0.0.113: Small release

This version of Holochain, released three weeks ago, was a small one. It contained two changes:

• Your post_commit callbacks should now be ‘infallible’, which means that they don’t return a Result. In fact, the host doesn’t even expect or check for a return value. (#1049)
• Breaking: HeaderHashes and HeaderHashedVec no longer exist. They were previously only used in post_commit, which is fairly new. Note that I mistakenly reported this for 0.0.112. (#1049)

View the changelog.

Holochain 0.0.114: Introspection, network simulations, and more

There are quite a lot of changes in last week’s release. Developers will note that zome functions can get more info about their context, while under the hood, we worked on DHT resilience improvements and got prepared to write sharding tests.

• Breaking: zome_info now includes entry_defs, a vector of entry definitions from the same zome. No code changes needed, but zomes will need to be recompiled. (#1055)
• Breaking: dna_info now includes zome_names, a vector of strings containing the names of the zomes as defined by the DNA manifest. As with the above, only recompilation is necessary. (#1055)
• Breaking: call_info now includes the name of the function being called. (#1078)
• Breaking: Within a hApp manifest, both ‘nick’ (deprecated) and ‘slot’ for a DNA have been collapsed into one new property called ‘role’. It does the same thing as ‘slot’ — gives the DNA an identifier that other components of the hApp (UI and other DNAs) can use to easily address cells created with that DNA, regardless of the DNA hash. This role identifier is internal to the hApp, and the role can also be filled by DNAs and cells that are external to the hApp, such as a DNA that implements DPKI. The only change you need to make is to replace slots with roles in your hApp manifest. (#1045)
• Breaking: A source chain query now guarantees ordered results: elements will be returned in header sequence ascending (oldest to newest). (#1004)
• New: the admin API now has a DumpFullState endpoint, which lets developers inspect individual cell states during testing. (#1065)
• remote_signal now uses a more efficient method of sending notifications to peers, which can take advantage of a multicast approach. It will also now fail silently if it can’t contact the peer — which is a good reminder that signal delivery is not guaranteed and shouldn’t be relied on for important exchanges. (#1075)
• Breaking: Remote nodes will now return a Busy message if they’re overloaded with incoming gossip. This continues work in fixing consistency bottlenecks, allowing peers to process DHT data at a sustainable pace. This breaks the network protocol, which means that cells running on conductor 0.0.114 won’t be able to communicate with cells running on earlier conductors. (#1076)
• Sweetest tests can now create simulated networks, which is already being used in sharding tests. (#1036, #1037)

View the changelog.

Holochain 0.0.115: Network reliability, more introspection

This week’s release is a quiet one, mainly bringing improvements to the database and networking code. Once again, the *_info host functions, which lets a zome function call introspect its context, have gotten some additions which will require no code changes but will require recompilation of your zomes’ WASM.

• Breaking: zome_info now includes externs, a list of zome functions that comprise the zome’s public API. (#1081)
• Breaking: zome_info now includes properties, similar to the dna_info’s properties but specific to the zome. Note that this is just a placeholder and isn’t populated with any data, as it’s waiting for future changes to the DNA manifest spec. (#1080)
• Breaking: agent_info now includes the agent’s current chain state, as at the time agent_info was called. (This includes any elements written to the scratch space and waiting to be committed in the current zome call.) (#1079)
• Bugfix/breaking: Support for the the newest Lair keystore manager has been added, bringing more reliable connection management. If you decide to use it, you’ll need to rebuild your keystore, as the encryption passphrase is now hashed before being used as a key. (#1094)
• Bugfix: Data from private source chain elements are now no longer processed by the local DHT shard, preventing errors when trying to commit them. (#1104, #1110)
• Bugfix: ‘Delete’ DHT ops are now formed properly, preventing cells from shutting down. (#1105)
• Bugfix: Nodes will try to silently reconnect to peers if a connection is dropped. This improves network reliability. (#1083, #1111)
• Bugfix: Cells hosted on a HoloPort no longer error when trying to retrieve data from their DHT shard. (#1098)
• Bugfix: All database transactions are asynchronous; this fixes various database connection issues. (#1097)

View the changelog.

And two more releases

holochain-conductor-api 0.2.4

holochain-conductor-api, the JavaScript client library for talking to local Holochain conductors, has been updated and a new version has been released. Here are the changes:

• Update: Call zome functions using the new app API endpoint name. This fixes an issue where attempts to call a zone function used a previously deprecated and removed endpoint name and were returning an error. (#91)
• Update: Update app installation endpoint calls and data types to replace ‘slots’ and ‘nicks’ with Holochain 0.0.114’s ‘roles’, along with other hApp bundle format changes. (#90)

You can find it in all the usual places — you can get it via NPM as @holochain/conductor-api or download it from GitHub.

Tryorama 0.4.6

Tryorama, the JavaScript-based testing framework for hApps, has also seen a new release. As with holochain-conductor-api, this brings it in line with the changes in Holochain 0.0.114. Get it via NPM as @holochain/tryorama or download it from GitHub.

There’s a lot more going on in developer land, but as you may know, we’re preparing for the first public test of Elemental Chat. Right now, I’m busy writing educational materials for all the people who will be trying a hApp for the first time!

Cover photo by Motoki Tonn on Unsplash