Summary
We’re excited to see Holoscape, our end user environment, begin to take off. Holoscape will offer some significant benefits to developers, which you’ll find quite exciting. We’re also announcing updates for locking and unlocking keystores, as well as better documentation on how links work in Holochain. Lastly, this week we’re sharing an overview of how we do testing and how it’s evolved — we think it will be enlightening for anyone interested in where Holochain is going next.
Highlights
1. Three benefits of Holoscape
2. Locking and unlocking keystores via IPC unix domain socket
3. Documentation for our links ecosystem
4. Featured Topic: Testing framework
Details
1. Three benefits of Holoscape
P2P technologies typically enter the mainstream when an application using that technology and built with end users in mind comes on the scene. For example, think of how BitTorrent exploded in popularity when clients like Transmission or uTorrent made file sharing as easy as pasting a torrent file or link. More recently, the Dat protocol gained a lot of visibility when the Beaker Browser used it to allow users to seamlessly build websites and web apps without the need for an external server.
Along that vein, Holoscape will allow any end user to install Holochain and any hApp they want with one click. Specifically, it bundles a conductor with a UI for administration and a run-time for hApp UIs. This presents a huge opportunity to make Holochain accessible to a much broader user base.
Holoscape offers three key benefits for both the Holchain Core Dev Team and app developers: (1) easy deployment, (2) stronger product feedback, and (3) easier testing. Let’s look at each of these benefits in more detail.
Deployment is easy (and can get even easier)
In the future, deployment will be as simple for app developers as asking your end users to install Holoscape and try out any Holochain app. We then hope to make it even easier by using Electron App Packaging to allow Holoscape to be downloaded directly from the Holochain website. Developers could create their own app bundles that end users could use to install the apps inside Holoscape via an app store — it would all be a matter of a few clicks of the mouse.
A product focus enables stronger user feedback
By launching an end-user-focused client, we hope Holoscape will encourage a product-oriented approach to Holochain, making sure the needs of users are driving the evolution of the technology.
Having a direct relationship with end users, rather than waiting for feedback from developers, will help inform our work as Core developers and help us to see how we can help app developers in their projects. With Holoscape, we will have a closer feedback loop process.
Testing can reach more and more machines
Holoscape will also make testing easier. We want to conduct testing ‘in the wild,’ for everything, in addition to performing automated testing. We did this with the Basic Chat Electron package, installing it on several machines over the internet. Likewise, with Holoscape, it will be easier to test any app on multiple machines over the internet. As we said above, when you can do that in just a few clicks, feedback is easily passed from users to developers.
We’ll also get more insight through telemetry from the app, as well as information on the source chain, and hopefully, node connections. All of this will give us a more complete picture of what’s going on in the app.
2. Locking and unlocking keystores via IPC unix domain socket #1646
With the implementation of Holoscape, we’ve had to come up with a different way to unlock the keystores. Normally, passphrases are requested via the command line, but to make Holoscape function, we needed to make it possible to display a graphical dialog when passphrases are needed. Now, when Holoscape is initializing the Conductor and Conductor config file, it creates a UNIX domain socket for inter-process communication (Windows compatible implementation will follow), then writes the socket config into the config file where Holoscape can find it. Therefore, Holoscape users will be able to receive the passphrase prompt on their local machine.
3. Documentation for our links ecosystem #1628
We wanted to pull all our documentation on how links work in Holochain-Rust together into one comprehensive location. Besides get_links and friends, we also have descriptions of arguments LinksStatusRequest and Headers, and related processes like zome validation.
4. Featured Topic: Testing framework
As we’ve developed Holochain, our biggest question has perhaps been, “How do we test Holochain in action — in real world situations?” The evolution of our testing framework has happened simultaneously with that of Holochain itself, and we wanted to share how that framework has evolved and what it means for app developers building with Holochain.
Internal and external tools
One thing we’ve learned while developing testing tools is that it’s important to distinguish between internal testing (for people who are developing the core components of Holochain) and external testing (for people outside the Team making apps and other technologies that rely on the Holochain framework). Each type of testing has its own needs and points of focus, but that only became clear as we built Holochain out and saw what worked and what didn’t. It became especially clear when testing network multi-node communication — we realized our testing mostly concerned our own, internal needs, rather than those of app developers. For them, testing should be a tool to ensure that their apps function properly. Our tools were more like an integration test, exercising every bit of the system.
Consistency signals and Hachiko
In relation, we’ve worked to create a model of what “eventual consistency” means for Holochain. This led to the creation of the Hachiko library, which we use to help us reason about propagation of events across the network. It’s a difficult thing to test because we need a lot of detail about what has happened, not just on one machine, but on many machines. Fortunately, each node produces a flow of “consistency signals,” a report of what it’s produced or received. Hachiko allows us to stitch these signals from multiple nodes together into one story — a story of what has happened on each machine as well as of the relationships manifested in inter-node communications.
We use Hachiko in internal core tests, as well as external DNA tests to help us confirm correct behavior across multiple nodes; for instance, that action X on computer 1 triggers action Y on computers 2, 3, and 4.
The “puppet master”
With this testing framework in place, we now have something that can act as a “puppet master” or “orchestrator” for multiple computers. This puppet master is able to take all kinds of actions across multiple devices — it can understand what’s taking place within them (with respect to Holochain) and between them.
It’s also scalable; it will take as many Holochain nodes as you give it. Essentially, the puppet master can reproducibly and deterministically control multiple Holochain Conductors, understand what is going on within them, and determine if Holochain networking is functioning according to expectations.
The roadmap going forward
Our next steps for testing will include provisioning cloud hardware to run scaled-up, deterministic Holochain tests against a real-world network. So far, we have tested several physical devices networked over the internet, while simulating multi-node devices. As mentioned, these simulations have shown no real upper limit as to the number of nodes that we can test successfully, so we are hopeful that physical tests will show the same results.
We are also preparing to switch to Rust-based lib3h at the earliest opportunity in order to benefit from our testing, which will be instrumental in validating lib3h. We have grown comfortable using n3h in our tests, but given the broader shift toward lib3h, we know we need to quickly reach the same level of comfort after the switch.
We are very much on a journey. We have reached a certain level of maturity, both in Holochain itself and in our testing tools, and we know more about what works and what doesn’t. Thankfully, we can now avoid many of the roadblocks that have bedeviled us in the past.
Development Status:
- Released: 0.0.28-alpha1
- Next: 0.0.29-alpha1
For all changes related to this release, please see the changelog.
