Rewriting Deno’s Testing Tools

For the past month and some change, actually might’ve been two months already (It takes a day to write a proof of concept, weeks to land it). I’ve been busy re-thinking Deno’s testing tools.

We’ve had a test runner since before I got involved and it worked just fine. But it was as barebones as you could make it so I rewrote all of it.

A new parallel runner

Our test runner was written in JavaScript, it worked but it was slow as it was a serial test runner that just merged all the tests into a single context and ran them one by one.

Meaning that if one test set a timeout for 60 seconds, or more realistically wait for a server response, nothing would until that has completed.

To deal with that, I rewrote the test runner from scratch in Rust so that I could have explicit control over the threading model.

Each test module now runs on its thread with its own scheduler and runtime.

Compilation and type-checking however is done ahead of time for the entire module graph, which makes it still fast.

deno test --jobs=4

Test times on our std test suite were cut in half from about 1 minute to 30 seconds for the entire test suite.

This has already landed and will ship in Deno version 1.10.

Per test case permissions

Internally we haven’t actually been using deno test for our own unit tests. Until now, I extended the test runner with per test case permissions letting us dog food and drop the ad-hoc test runner completely.

So your unit tests may now be defined with the permissions they require and they will be requested/revoked accordingly.

Deno.test({
  name: "write",
  permissions: {
    write: true,
  },
  async fn() {
    // ...
  },
});

Take note however, that if you run deno test with no permissions, any requests to get more permissions than what you started with will be denied.

This has already landed and will ship in Deno version 1.10.

Improved pretty output

This is still a work in progress, but the output format is getting some love. Primarily, while rewriting the reporter in Rust I’ve added the origin of the test.

Before:

running 2 tests 

After:

running 2 tests from file:///home/caspervonb/test.ts

It’s a small thing but a lot of things had to fall into place because we couldn’t do this in JavaScript without breaking some sandboxing rules so this tiny little change has has been pending for months.

This has already landed and will ship in Deno version 1.10.

Testable documentation code blocks

I think this is the cooler feature of the highlights. At-least I found it useful on our codebase. Turns out that a lot of our examples have that one sneaky typo in them, and are therefore broken altho the actual example itself is correct.

So what if your code examples could be type-checked and run? Well, with Deno can.

I’ve already implemented this but I don’t think I’ll ship it with 1.10 as I want to polish it more before letting it out there in the wild.

Also playing around a little bit with enabling the same for client side code, but I mainly work on Deno so we’ll see what happens with that.

Conclusion

I’ve gotta get back to writing code so this is the end of this little feature preview for Deno 1.10.

Wanna help me get to my next GitHub sponsors goal? Here’s a link for that.

More content at plainenglish.io