This week, the focus was largely on cryptography. More and more of our long-term work providing cryptographic support to applications is stable and being incorporated into our official releases. We’ve also started the important work of perfecting performance profiling and tuning.


  1. Data Integrity/Identity: New KeyStore Ready — Available to Apps and Core
  2. Performance: Optimizations!
  3. Network Bootstrapping: ‘P2P Ready’ Message
  4. Debugging: Logging Flag for `hc run`
  5. Lessons Learned: Closing the Gap Between Core Devs and App Devs
Holochain Dev Pulse March 25–April 1st, 2019


1. Data Integrity/Identity: New KeyStore Ready — Available to Apps and Core

Identifying yourself on the Internet is a hard problem, currently solved by centralized PKIs like certificate authorities — doing it without centralizing power is an even harder problem.

However, we like hard problems, so we set out to create a distributed public key infrastructure (DPKI) that puts you at the center of your digital identity, while simultaneously solving many of the user-friendliness problems of decentralized trust.

Over the past couple of months, a good deal of our work has been related to this in some way or another. From low-level cryptography support, to the DeepKey App, to the Holo key generation sequence — this is a large project that spanned all of our dev teams. A lot of the work has already landed, but we still have more to do. This week, we rolled out changes to our KeyStore Manager that stores all your secrets, your private keys, and key seeds. They live in secure memory buffers in the Conductor, which performs cryptographic operations on your behalf.

DeepKey Key Management Interaction Flow

These changes allow us to realize all of the functionality of DeepKey, which is our default implementation of the DPKI pattern. We intend DeepKey to be replaceable, so you can write your own implementations to better suit your community’s needs.

Your device’s secrets are used by the Conductor to sign data and control access to the RPC interface, the N3H networking layer to encrypt communications, and the apps themselves. You’re already able to sign and verify signatures within your zome functions, but now you can generate new seeds and keys and specify which key to use for signing. Because you don’t have direct access to any of the secrets, you’ll refer to them by their string ID.

We’ve currently implemented HDK functions for signing, signature verification, and secret generation. We’ll be introducing encryption and decryption functions in the future.

New functions in `hdk::api`:

Read the full documentation.

Key Management Part 1 and Part 2.

Breaking changes: Any keys created with `hc keygen` will need to be regenerated.

2. Performance: Optimizations!

Our Holo dev team recently demoed full, end-to-end networking. It was very exciting to log in and use a Holochain chat app running on a public website; but as more testers joined, it got pretty slow. Each message would take a minute or more to send. This complaint was echoed by devs in our recent Sunshine Coast Holochain Intensive.

FlameGraph that shows relative CPU time spent in functions/callstacks. Performance Issues Before Resolution. Note: Left half is irrelevant start-up sequence consisting mainly of passphrase hashing.

After some performance tracing, we identified the cause as an inefficiency in the networking handler, which communicates messages between Holochain and the networking layer. When it receives a message, it needs to figure out for which DNA instance it intended. Unfortunately, it was re-hashing the DNA for every message — and there were a lot of them. The good news is that we managed to fix this issue pretty quickly. We also removed some redundant filesystem reads from our EAV implementation. The end-result looks way better:

Note: Performance Issues Optimized; Almost all CPU spent idling or in passphrase hash functions (=start-up).

We’re now eager to track down more performance issues, such as slow persistence implementations and high CPU usage when the Conductor is idle. This is a good place to be in, because not only do we now have the time and tools to do it, but we have a reason as well. Real apps are uncovering real bottlenecks and forcing us to remove them. Optimizations like this are no longer premature.

3. Network Bootstrapping: ‘P2P Ready’ Message

Bootstrapping a Holochain app network is currently a do-it-yourself process. We don’t yet support mDNS discovery or supply a bootstrap server like we did in Holochain-Proto. This will change, but for now, you have to start one node up as a bootstrap node and point all other nodes to it. Previously, the Conductor would output a friendly copy-and-paste URL, but it was hidden behind a log-level flag and tended to get lost in the debug stream. Now, you’ll see a message like this whenever you start the first node’s Conductor:

Network spawned with bindings:

- ipc: wss://
- p2p: [“wss://”]

4. Debugging: Logging Flag for `hc run`

In recent versions of Holochain, we were debugging concurrency issues in the networking components. We ended up turning on a firehose of debug messages for the development Conductor, which is inconvenient for app developers. Now those messages are suppressed by default, but you can turn them back on for `hc run` with the ` — logging` flag.

5. Lessons Learned: Closing the Gap Between Core Devs and App Devs

As a core developer, your role is to build the foundation of what makes Holochain work. This requires you to keep a fairly complex world model in your head as you reason about what’s next. It all also needs to connect in the context of the wider world — your target users and their needs — so it’s a bit of a juggling act.

A couple of core developers shared their experiences with recognizing areas where they weren’t receiving the right context. They said that as app development ramped up, they received more feedback on bugs and desired features, but the relative urgency of some of it wasn’t understood clearly.

They also shared their renewed appreciation for how important it is to close that communication loop — to get clarification about what’s being asked and why, get more details than needed, and investigate timelines and blockers.

The core devs also had an opportunity recently to interact more closely with app developers in the wild and within our own team. This has given us valuable insight into devs’ current pain points and how they’re using and experiencing Holochain. We also have a few team members who have formed an informal task force to champion a satisfying app developer experience. We’re expanding in all directions right now, so some changes will come slowly. Developer experience has always been one of our top priorities, so we encourage you to keep us accountable regarding this goal!

What are the specific sticking points that bug you most when developing on Holochain? Share your comments on the Holochain App Dev Channel on Mattermost.