Summary
Our developer retreat has come to a close. The core and hApps dev teams have appreciated the time we spent together, squashing bugs, looking to the future, and sharing a good bit of laughter. While continuing down the path towards Holochain Rust Alpha, we also reflected on our development practices. The unique approaches of our team, along with some of our apps already in the pipeline, could be game-changers for other dev teams around the world!
Note: Unless otherwise mentioned, all features and bug fixes are currently available in the `develop` branch for the trailblazers out there, and will be included in the upcoming `0.0.4` release.
Highlights
- Compilation Fixes: Building From Source
- Holochain Core Flow Fixes: Removing Barriers to DHT Access
- WASM Memory Management: Progress on Data Handling
- DHT Work: Maturing the Internal Networking Protocol
- Header Sharing Fixes: Full Source Chain Audits
- A New Name in the Department of Names: Container Becomes Conductor
- Conductor API Functions: `.holochain` Folder for DNA, Data, and Config
- Dev Retreat Recap
Details
1. Compilation Fixes: Building From Source
A number of developers have reported that building the `hc` dev tool from source was failing. We tracked it down to a change in the way that Rust crate dependencies are managed. We’ve updated the installation instructions, so you should be able to follow them to get the latest features and bug fixes from the `develop` branch.
2. Holochain Core Flow Fixes: Removing Barriers to DHT Access
We make heavy use of asynchronous programming to manage the complex flow of actions and data through Holochain-Core, relying on Rust’s futures library to make it easy. However, some things break that flow, particularly things that have to pass through the membrane of the WebAssembly runtime. In particular, it was impossible to talk to the DHT or add entries to your source chain from within the `genesis()` function (which runs when you join an app) or the `receive()` function (which is responsible for responding to direct messages from other nodes). We’ve now fixed those issues.
3. WASM Memory Management: Progress on Data Handling
As mentioned in a previous Dev Pulse, WebAssembly has a 64kb limit on how much data can be accessed at once. This affects the ability to manipulate images, videos, audio, and large pieces of text. Building on our groundwork from previous weeks, zome and validation functions are now able to receive large pieces of data and long source chains. We’re still working through some issues, but it’s ready for testing. Next, we’ll enable them for things that go out of your app’s DNA — that is, you’ll be able to return large pieces of data from your zome functions or commit them to your source chain.
4. DHT Work: Maturing the Internal Networking Protocol
In Holochain-Core, there’s currently a hack to solve a chicken-and-egg problem that prevented new DHT members from getting certain existing pieces of the DNA. Shocking, we know, but the truth is that every project has technical debt, and we keep a tight rein on ours. We’ve finished the preliminary work for removing this hack.
5. Header Sharing Fixes: Full Source Chain Audits
We’ve addressed two places where not all chain headers were showing up when needed. First, only headers for public entries were being sent to validation functions.
Now, all headers are sent, even for private entries, so the validator can confirm that the chain is in order. Private chain entries are not shared; they’re still kept safely on their author’s device.
Secondly, you can now request headers for local chain entries with the new function `hdk::query_result()`.
6. Consensus on a Replacement Name For “Container”
From the Department of Names Department: Our team has confirmed a new name for an important component of Holochain. We reached a consensus this week; “containers” have become “conductors.” We’ve changed the name to ‘conductor’ based on external and internal feedback.
We came up with the ‘container’ for holding all your running instances, but we never really came to like its given name.
If you’re a developer, the name container makes you think of something else and doesn’t adequately describe all that’s going on in this executable. The executable names will likely see some changes as well.
One of our devs explained the reasoning for the term conductor using similes:
- Like an orchestra conductor, it helps several parts work together as a whole.
- Like a train conductor, it runs the parts.
- Like an electricity conductor, it enables parts to communicate with one another.
7. Conductor API Functions: `.holochain` Folder for DNA, Data, and Config
We’ve made things tidier by allowing you to specify a place for everything to land and introducing a few conductor API functions for manipulating configuration and installing new DNA. When you install a DNA, create a new instance, or manipulate your configuration, they are all now stored in one folder. This is a folder called `.holochain` in the root of your user account folder, but you can change the name to whatever you like.
8. Dev Retreat Recap
The core and hApps dev team members descended on one of our core devs, Nicolas Luck’s, house for two weeks. We hacked together, discussed future plans, broke bread, and enjoyed each other’s company. We’re now making our way back to our respective corners of the world. We have tackled small and large issues, renewed clarity on naming, and fostered a strong desire to share our internal development methodology with the world.
Acorns and Oak Trees: Over the past couple years, our dev team has grown from a pair of visionaries, to a small band of core and app developers, to a globally distributed team of teams. We’ve tried various ways of working together, but nothing quite fit the needs of our teams. We’re evolving and dogfooding a new methodology we call Acorn/Squirrel. It’s thus far giving us huge gains in our collective ability to manage complexity in an efficient and asynchronous way.
All our tools are cobbled together from existing services, but we intend to grow this seed into something everyone can use (including a suite of Holochain-based tools for managing source code, projects, and funding).
Watch for more information in the coming months!
