Skip to content

Demystifing Ember Async Testing

by cory on July 10th, 2014
I've been using Promises in my ember code for long enough now that I've started to internalize how they work. So, when I started writing acceptance tests using the ember async test helpers (`visit`, `click`, and so on), I assumed that since those helpers are referred to as "async helpers", they would be promises and I should chain them all together in my test to ensure that the asynchronous activity all happens in the correct order. I understood that ember's testing framework is now promise-aware, and I presumed that meant it would notice when the return value of my test was a promise, and wait for that promise to settle before finishing the test.

I started out writing test code like this:
test('visiting the index page', function(){
  return visit('/').then(function(){
    return click('button#success');
  }).then(function(){
    // all async activity from the IndexRoute and
    // clicking the success button will be finished
    // by now
    equal(currentURL(), '/success');
  });
});


[Note: This article assumes some familiarity with Javascript Promises. If you're not familiar with them, there are many resources online to learn, including an article I wrote about them that includes an interactive demonstration of promises in action.]

This should look familiar if you're comfortable with promises. Each promise-returning function (`visit`, `click`) is chained via a `.then` call to the next, ensuring that the `visit` promise is settled before we attempt the `click` promise, and so on. Test code like this works just fine, but it is a bit verbose.

Ember's Test Helpers are Fake Synchronous


What I didn't realize originally is that Ember's async test helpers are much smarter than I gave them credit for. In the Test Helpers section of the Ember Guides, this example test code is shown:
test('simple test', function(){
  expect(1);
  visit('/posts/new');
  fillIn("input.title", "My new post");
  click("button.submit");

  andThen(function(){
    equal(find("ul.posts li:last").text(), "My new post");
  });
});

What! If that code isn't at least lightly blowing your mind, then you either understand Ember's testing helpers better than I did, or you may not be paying close enough attention. I'll illustrate with an example. We want to ensure that the `fillIn` call is not made until the `visit` call has finished (which could take an arbitrary amount of time, due to the fact that it may involve fetching data from an API), and likewise that the `click` call isn't made until the `fillIn` operation has finished. And, finally, any action that Ember takes as a result of clicking "button.submit" (which could involve sending data to an API and/or a route transition) is also finished before the `andThen`'s anonymous function containing our test's assertion is called.

Naive Promise-Returning Test Helpers


Let's take a stab at writing our own pseudo versions of the promise-returning `visit`, `fillIn`, `click, and `andThen` helper functions:
function visit(url){
  return new Ember.RSVP.Promise(function(resolve, reject){
    var timeout = Math.random() * 50; // arbitrary delay
    Ember.run.later(function(){
      // route-visiting code such as `beforeModel` etc
      // would go here
      console.log('Finished visiting ' + url);
      resolve();
    }, timeout);
  });
}

// assume fillIn, and click are defined similarly

// Now, run these lines:
visit('/posts/new');
fillIn("input.title", "My new post");
click("button.submit");

// Console output:
// (All three statements run at essentially the same time,
//  finishing in a non-deterministic order)
// 'Filled in input.title with My new post' <-- 2nd promise finishes first
// 'Clicked button.submit' <-- 3rd promise finishes before 1st
// 'Finished visiting /posts/new' <-- 1st promise finishes last

Here is a live example showing what happens if we take the naive approach described above. As you can see, just because the `visit`, `fillIn`, and `click` method calls are on subsequent lines, this does not guarantee that their promises resolve in that order. All three helpers start nearly at the same time (you'll see the "1a", "2a", and "3a" log lines appear nearly instantly), but they finish in a random order.
Ember Async Test Helpers Part I

Now it should be somewhat clearer why Ember's test helpers are mysterious. How can Ember ensure that the `fillIn` method call only happens after the `visit` one is done, if we haven't explicitly chained them together? In a normal javascript context, you cannot delay execution of a line based on a previous line. Javascript is non-blocking, so each line executes as soon as the previous one has, even if the previous line has asynchronous behavior that isn't finished yet. This is different from some other languages you probably use. In Ruby, for example, if you had a line that requested data from the database (e.g., `user = User.find(1)`), the execution of the next line of ruby would block waiting until the database had returned its data, and on the next line you could reference the `user` variable, knowing that it had been fully hydrated from the database.

This is not the case for javascript, however, and it is why we used chained promises when we need to guarantee the order of asynchronous method calls. We cannot get away from the need to chain these calls together, but we can get away from explicitly chaining them together, if we are clever.

Context-Aware Test Helpers


Let's redefine our versions of the `visit` and other async helpers. Now, instead of simply returning a promise, they will detect the presence of a global, shared-context promise, and automatically chain themselves to that if it exists. If it doesn't exist, they'll create it and then chain themselves to it. Now our `visit` function, for example, will look like this:
function visit(url){
  // create the shared promise
  // if it doesn't exist
  if (!window.sharedPromise) {
    window.sharedPromise = Ember.RSVP.resolve();
  }

  // chain our code to the shared promise
  window.sharedPromise = window.sharedPromise.then(function(){
    // this is the original `visit` code:
    return new Ember.RSVP.Promise(function(resolve, reject) {
      // ...
      // simulate delay and then call `resolve`
      // ...
    });
  });
}

Visually, this is what is going to happen now:
async-test-helper-animation
Here is a live example showing these global promise-aware pseudo test helpers in action: Ember Async Test Helpers Part II

Putting it all together


As it turns out, this is almost exactly what Ember's testing helpers do! The Ember testing code maintains a shared promise (stored on `Ember.Test.lastPromise`) that it chains each additional async helper call onto. This is the relevant code snippet:


The Ember testing code does a few additional things (such as attempting to resolve nested async promises properly), but at a high level all it is doing is using a shared promise global variable to chain each async helper onto. This is how Ember testing helpers allow you to write code with asynchronous behavior but write it in a synchronous style, without having to explicitly think about promises or promise chains.

At this point you may be wondering how Ember actually knows that the `visit` (or `click`, etc.) async helper has finished all its async behavior. So far we've assumed that the `visit` helper returns a promise that resolves when its async activity is finished, but we've glossed over the actual mechanism by which Ember ensures the promise resolves at the proper moment. The answer to this is that Ember internally uses a heuristic for async activity that involves checking for run loop activity, pending ajax requests, router activity, and more, but that is a subject for a later blog post. Please follow me (@bantic) on Twitter to hear about future posts.

Summary

The one takeaway here is that when writing integration tests in Ember, every line of your test should be an async helper or the anonymous function passed to `andThen`. That is, every line of your test should be one of `visit`, `fillIn`, `click`, etc., or an `andThen` call, or inside of the `andThen`'s anonymous function. If you do this, the async-aware nature of Ember's testing code will allow you to write simple, clean tests and be confident that the code will run in the correct order, without having to clutter your test code with explicit asynchronous control flow.

Need Ember.js Testing Help? Hire us!

If your company needs help writing solid tests with Ember, my Ember.js consulting company, 201 Created is available for hire. Please get in touch!

From → general

One Comment
  1. joe permalink

    a year later, and still good!

Leave a Reply

Note: XHTML is allowed. Your email address will never be published.

Subscribe to this comment feed via RSS