Summary

We have released an important update to Try-o-rama — it’s been completely rewritten to be more reliable and easy to maintain. With new functionality, it’s also general enough to handle any testing situation, providing an all-in-one solution. We’re also sunsetting older versions of Try-o-rama and our testing tool, Diorama, so it’s important to get acquainted with this new release.

All-in-One Tool

Building a single, all-in-one tool for end-to-end DNA testing

For the past few months, we’ve been working toward a mature testing framework that allows developers to test their conductors in real world situations and under a variety of conditions. Our first breakthrough was Diorama, an application that helped us move away from Node.JS-based conductors in favor of Rust-based conductors. Diorama allowed for simple tests between Holochain instances running on a single conductor. It was intended primarily for third-party Holochain developers to test their zome logic in multi-node scenarios using in-memory networking.When the Holochain Core Dev Team needed to write its own end-to-end tests, they turned to Diorama. However, Diorama could only handle a single conductor, which meant they couldn’t get a good picture of how the networking code was actually running.

To solve this problem, they began to build an experimental version of Diorama, extending it to support tests with multiple conductors in a realistic networking setting.

This new tool was dubbed Try-o-rama — the logic being that Diorama incremented would be Triorama, but Try-o-rama sounded catchier! The initial version, 0.0.0, was used to implement Holochain Core’s app-spec tests and functioned as an end-to-end test suite enabling full Holochain functionality in multi-node scenarios.

A new problem emerged — we now had two testing tools, Diorama and Try-o-rama, both of which had their own limitations and peculiarities. Which tool should be used in a given situation? There has been some confusion in the dev community, even within internal Holochain teams, about how to proceed. A single tool for all conditions would be highly preferable and would help to consolidate the gains we have made in our testing framework.

Introducing Try-o-rama v0.1.0 and sunsetting older tools

Our solution is the release of a new and expanded version of Try-o-rama that has been completely rewritten to be more reliable and easier to maintain. Version 0.1.0 represents not only another extension to Holochain’s end-to-end testing capabilities, but also a single application that eliminates the need to figure out which testing tool you should use each time you’re developing.

The new Try-o-rama is general enough to handle any testing situation that we have encountered so far, as well as any testing situation we or other developers are likely to encounter in the future.

Most significantly, Try-o-rama can now not only specify multiple conductors for a test, but also arbitrarily start and stop conductor processes during a test. As our lib3h networking module is coming into fruition, we can write a whole new class of tests to ensure that networking is robust in realistic situations where nodes are constantly joining and leaving networks.

Testing Tools

The upgraded Try-o-rama means that older versions of our testing tools will no longer be supported, including Diorama and previous versions of Try-o-rama. We are working on helpful middleware for current test suites written with those applications in order to make the transition as smooth as possible.

How the new Try-o-rama works in practice

As in previous versions, the new Try-o-rama orchestrates scenario tests across multiple conductors. The difference is that these tests can be written to cover the more complex situations that you’re likely to encounter in the real world.

For example, say you have a DNA that implements chat functionality. You will want to write end-to-end tests across multiple nodes to ensure basic functionality. For instance, if nodes A and B are in a chatroom, and node A posts to the room, B can read it.

If you wanted to do a more comprehensive test of your DNA, you might try this:

  1. We start with offline nodes A, B, and C.
  2. Nodes A and B go online and join a room.
  3. Node A posts to the room, then goes offline.
  4. Node C comes online and joins the room.
  5. Node C should be able to fetch the message Node A sent, even though A is offline.

Thanks to the new Try-o-rama’s functionality, specifically the Conductor’s ability to dynamically start and stop, these sorts of tests are possible.

We’re interested to see how you incorporate the new Try-o-rama into your development processes, and encourage you to provide feedback. Our goal has always been for our testing tools to evolve as we evolve — your comments and suggestions help make that possible.

Development Status:

Install Holochain Learn about Holochain Join the Discussion