Summary

I have some big news this week: THIS THING WORKS! We currently have about 50 HoloPort owners, our beloved Pre-Release Testers, chatting with each other using a fully peer-to-peer Holochain network. It’s working, it’s fast (usually), and most importantly, it’s all behaving correctly at the networking layer.

To me, this has been a long time coming — a lot of waiting, a lot of keeping the torch of hope lit through the discouragement of Holochain Redux’s performance issues and consistency bugs, followed by the nail-biting experience of watching the core dev team put all their chips into the Holochain RSM rewrite. It is so wonderful to see a Holochain app working like a traditional cloud app, and to know that it’s probably just going to get better.

My biggest regret is that this news is coming out at a time when much of the western world is buzzed on tryptophan (although apparently that’s not actually a thing) and eggnog. To those of you who take the time to read this update, I applaud you. To those who don’t read this until after the holidays, I applaud you too — good job on the whole ‘taking time to connect with loved ones’ thing!

Highlights

  • Elemental Chat on HoloPorts passing real world tests with flying colours
  • HoloPort Nano installation image updated to HPOS v20.09, ready for manufacturing test run #2
  • USB recovery image for HoloPorts
  • Useful new host functions: `remote_signal` and encryption/decryption
  • Build It! sessions 2 and 3: create a hApp with Vue in moments, building communities and managing personal information
  • Our three video series for creators: what are they about again?
  • Mock JavaScript conductor is cool, maybe even useful to you

Elemental Chat on HoloPorts passing real world tests with flying colours

I tell you, the joy among our team is palpable this holiday season — and not just because a lot of us are taking a much-needed break. We’ve finally started deploying Elemental Chat to HoloPorts! So far about 50 pre-release testers have signed up to use and abuse our toy chat application to see if Holochain is really ready to handle real-world interactive applications.

And so far, everything says it is. Holochain’s big promise — data integrity without centralised infrastructure or blockchain ledgers — appears to be a promise it can keep.

Data eventually gets to the right place, and nothing gets lost or corrupted along the way, which shows that all the plumbing is working correctly, from the DNA, through the core, to the networking layer.

You might wonder, “Is it all smoke and mirrors? Is there some centralised database keeping track of all the data?” It really is peer-to-peer, folks — the only centralisation point (besides the Holo routing infrastructure used to connect testers’ web browsers to their HoloPorts) is the proxy relay, whose only purpose is to connect individual peers to each other through restrictive firewalls. And that’s only centralised right now for convenience. (If you’re interested in the technical details, the proxy relay is sort of like a TURN server for P2P Holochain gossip.)

This is a stage we never reached with Holochain Redux. You may recall that we had to create sim2h, a sort of switchboard server, to centrally manage and analyze DHT traffic while we were working out all the consistency and performance issues, and sim2h introduced issues of its own. So it’s pretty exciting for us to see Holochain RSM working exactly as it should.

Now, it isn’t bug-free yet — testers have noticed that messages sometimes take a while to appear, sometimes their UI loses the connection to their HoloPorts, and a bug (recently fixed) has been causing their HoloPorts’ fans to get really loud. But none of these bugs are at the networking or data integrity levels. Some are bugs in the UI, some are trivial bugs in Holochain or the HPOS, and some are simply indicators that our next step is optimising the heck out of Holochain itself.

So what is Elemental Chat anyway? It’s definitely not a polished, production-ready application; it’s just a toy app. We decided to release it first instead of HoloFuel, for three reasons: first, we thought it would be more fun to test; second, we knew it would push against all of Holochain’s capabilities in ways that would show up any bugs; and third, we wanted something that would show us where the performance bottlenecks were.

There are weird bugs that only an engineer could love (and, because I’m an engineer, I’ve written a forum post about the surprising nature of eventual consistency and how you might design for this in your app). We don’t expect it to live long beyond our alpha testing phase (for a real Holochain-based chat app, follow our friends at Kizuna or contribute to their crowdfund). But it’s serving its purpose well and showing that Holochain is performing beyond our expectations.

HoloPort Nano installation image updated to HPOS v20.09, ready for manufacturing test run #2

The HoloPort Nano, with its modest hardware specs, has presented unique engineering challenges. In early stages of development, we were the only project actually using NixOS (the foundation for the HoloPort’s OS) on the Banana Pi M64 board which formed the guts of the Nano. That meant we had to blaze a lot of trails to get it working.

More recently, the OS’ auto-updater was maxing out the Nano’s small amount of flash storage. I recently learned that the dev team has figured out a fix for this, and a new install image has been delivered to the manufacturer. This new image is based on the newest version of HPOS, which brings a big raft of changes from NixOS as well as the new Holochain RSM conductor. (That means that you won’t have to wait for a year and a half worth of updates when you first turn on your Nano.) I and a few other team members have tested this image on our Nanos, and they do indeed auto-update!

Our next step is to debug the Holochain conductor, which is currently crashing on startup, and deliver a few more units to team members for further testing. If everything checks out, it’s on to production in the new year!

USB recovery image for HoloPorts

In the process of testing the Nano’s new OS image, the Holo team created a USB recovery image. We’ve also been using this image to repair a couple standard HoloPorts and HoloPort +’s that were sent back under warranty, and it appears to be successful in many cases. We plan to make this part of our support toolkit, and it should help in those rare cases where a bad unit accidentally gets sent out from the factory or a working unit accidentally bricks itself.

Useful new host functions: `remote_signal` and encryption/decryption

During the development of Elemental Chat, the devs came up with a design called ‘signals’ — not the signals you’re used to, but messages that one agent’s cell sends to all other currently online agents, saying “hey, here’s a new chat message!” This bypasses the DHT (and also sends the UI the kind of signal you’re used to) so chatting feels fast and responsive.

Unfortunately it resulted in just the opposite — sluggish performance. This is because, under the hood, it used `call_remote`, the feature that lets one agent calls another agent’s zome functions. `call_remote` expects (and waits around for) a return value. This resulted in unnecessary slowdowns, because you don’t need to wait around to learn whether your signal was received — you know everyone else will eventually get the message through the DHT’s slower but more reliable gossip. Not only that, but `call_remote` blocks the zome function and can’t make parallel calls to multiple peers. So the more people are online, the slower everything gets.

So the core devs added a new host function, `remote_signal`. Just like a UI signal, it’s a non-blocking, fire-and-forget function that doesn’t wait for a response from the receiver. This improved Elemental Chat’s performance immensely, and it’ll be a useful tool for anyone who wants to build real-time collaborative apps (think live updates on shared documents, new message pings/badges, typing indicators, that sort of thing).

Read the aforementioned forum post where I talk about Elemental Chat’s weird eventual consistency bugs; `remote_signal` plays a starring role there.

The core team has also added some encryption and decryption functions for you to start playing with. They’re useful for publishing secret data so it can benefit from the resilience and availability that the DHT offers. There are two flavours of encryption, both built on libsodium’s suite of ‘best practices’ algorithms, and both with thrillingly unreadable names:

  • `x_25519_x_salsa20_poly1305_encrypt` is good for a message meant for a single recipient. It takes one of your stored public keys (you should generate them often and throw them away, because they’re your source of entropy) and a public key that the recipient gives you, derives a shared encryption key from them, and encrypts a message with it. The recipient uses the sister decryption function on their end.
  • `x_salsa20_poly1305_encrypt` is good for a message meant for multiple recipients. It takes an encryption key and encrypts a message with it. Its sister decryption function needs the same key, which means it must have received it via a secure channel first. This function is less secure than the previous one, because the secret is visible in WASM memory and could be read by malware. We intend to make this more secure in the future.

These functions replace the as-yet-unimplemented `encrypt` and `decrypt` host functions.

Build It! sessions 2 and 3: create a hApp with Vue in moments, building communities and managing personal information

Session 2: create a hApp with Vue in moments

In the second video Philip shows off more of his work with Vue that he introduced in the first video (GitHub links: preset and plugin). With these tools, any Vue developer can scaffold a simple, working hApp, complete with conductor, in moments. This time he demos a more complex app called ‘Columns & Cards’, which will be familiar to anyone using kanban-style apps.

This is exciting to me because Vue has millions of developers, and Holochain does not (yet). This tool eliminates all the fiddling around with GitHub repos, Nix packages, compilation, configuration, etc that would be frustrating to a new hApp developer. I’m sure you’ve had that “bad first impression” experience that made you abandon a new piece of software; this tool is another step in making the first experience a good one.

Philip shared this tool with some Vue developers this weekend, and he says they remarked, “Oh wow! We can actually build Holochain apps now!”

Session 3: building communities and managing personal information

In the third video Philip demonstrates a hApp meant for building communities, which is what Holochain is meant for. Using Columns & Cards as a template, the Community Builder is made for a group of people who want to manage a shared contact list.

It’s a perfect opportunity for showing the redesigned Personas hApp, which is all about managing your personal identifying information across all cooperating hApps. Personas, originally called Personas & Profiles and later rebranded Identity Manager (see the first version here), will be a special class of hApp called a ‘conductor service’, which means it’ll be able to offer itself to any hApp that wants to use its functionality.

Personas is like an auto-form filler that also auto-updates your personal details as you change them, in any hApp that’s asked for those details. As you might have guessed from the name, you can create separate personas for your different spheres of life (family, work, volunteering, etc), each with different given names, phone numbers, email addresses, preferred usernames, and more. Eventually you’ll also be able to see which hApps have asked for that info and what they’re doing with it.

I don’t know about you, but I hate filling out the same information over and over and over again. Sure, my web browser does that for me these days, but if I change my address or phone number, Firefox isn’t going to go to every website for me and update that information. The only reason this is possible with hApps is that they all live on my device and it’s easy to make them talk to each other. I think this will be a big deal for anyone with tired fingers and accounts in a zillion different services.

Our three video series for creators: what are they about again?

We’re coming out with a lot of new video series lately in an effort to meet all sorts of creators and give them glimpses of how Holochain might be relevant to them. If you want to learn by example as you listen to other people talking about their projects, we think you’ll find at least one of those series interesting… but which one? Here’s a handy guide:

  • Ecosystem Sessions are long-form conversations on progress, beliefs, and philosophy with long-standing hApp builders. These people typically have already started creating a business model with Holochain in mind and may have even started coding, though the conversations usually revolve around the squishy ‘human’ layers: economics, society, ecology, politics, and the like.
  • Build It! is a short, live, guided tour on how to quickly develop working apps on Holochain using scaffolding and other rapid application development tools. It usually results in a working product by the end of the video. If you’re a developer, you’ll get a lot out of these.
  • Low-Code Zone (link TBA) explores the things that Holochain makes possible for actual projects. We look at tricky sociotechnical problems through agent-centric goggles, investigating concrete ways to apply the tools at hand. We hope entrepreneurs, CTOs, and other moderately technical decision-makers will get a good picture of how to apply Holochain to their own contexts as they absorb other people’s solutions.

Mock JavaScript conductor is cool, maybe even useful to you

The Holo Host dev team have created a mockable conductor library for use in tests. You can load up `Holo-Host/mock-conductor` with mock functionality, put it in place of a WebSocket connection to an actual conductor, and unit-test your UI. It was created for testing Envoy and Chaperone, but I figured some developers out there are going to say “thanks, I needed this!” so I’m sharing it with you all. Hope you enjoy this little stocking stuffer!

Finishing up

Whew, that felt like a long one. I’m glad there was so much news to report; it feels like a lot of good things are happening really quickly. We’re all going to do a bit of resting over the holidays, including me, but I encourage you to keep connecting with us and each other on the Holochain  forum, on Twitter, on Elemental Chat (if you’re a pre-release tester), and anywhere else you like to hang out with us. We may not be as responsive as usual, but we’ll be back in the fray soon.

I’ll see you again in the new year, where I’m likely to have more exciting updates in rapid succession! Until then, take care of yourselves, find some time for your souls to breathe, celebrate the fact that you faced a really hard year and got through it, and keep working and hoping for a brighter future.

Install Holochain Learn about Holochain Join the Discussion