Skip to content
Jul 13 14

Physics Simulations with Ember

by cory
I saw this cool javascript simulation of a double pendulum by @MicaelOl on hacker news the other day, and aside from taking me back to my Classical Mechanics class in undergraduate many years ago, it inspired me to try to do the same with Ember and SVG.

As a first stab, I made a single pendulum aka a Harmonic Oscillator:
{{ember-pendulum}}
And then after a little messing around with it (and copy-pasting the equations verbatim from the original javascript sim), I got an Ember double pendulum simulation!
{{double-pendulum}}
When I was an undergraduate I toiled on a final project related to the Lagrangian for my Classical Mechanics for a couple weeks, and a large part of that was programming a double-pendulum simulation in a java applet. It took me about 2 weeks to make it back then. When I tried to do this in Ember it took me about 1 hour. That's progress.
Jul 10 14

Demystifing Ember Async Testing

by cory
I've been using Promises in my ember code for long enough now that I've started to internalize how they work. So, when I started writing acceptance tests using the ember async test helpers (`visit`, `click`, and so on), I assumed that since those helpers are referred to as "async helpers", they would be promises and I should chain them all together in my test to ensure that the asynchronous activity all happens in the correct order. I understood that ember's testing framework is now promise-aware, and I presumed that meant it would notice when the return value of my test was a promise, and wait for that promise to settle before finishing the test.

I started out writing test code like this:
test('visiting the index page', function(){
  return visit('/').then(function(){
    return click('button#success');
  }).then(function(){
    // all async activity from the IndexRoute and
    // clicking the success button will be finished
    // by now
    equal(currentURL(), '/success');
  });
});


[Note: This article assumes some familiarity with Javascript Promises. If you're not familiar with them, there are many resources online to learn, including an article I wrote about them that includes an interactive demonstration of promises in action.]

This should look familiar if you're comfortable with promises. Each promise-returning function (`visit`, `click`) is chained via a `.then` call to the next, ensuring that the `visit` promise is settled before we attempt the `click` promise, and so on. Test code like this works just fine, but it is a bit verbose.

Ember's Test Helpers are Fake Synchronous


What I didn't realize originally is that Ember's async test helpers are much smarter than I gave them credit for. In the Test Helpers section of the Ember Guides, this example test code is shown:
test('simple test', function(){
  expect(1);
  visit('/posts/new');
  fillIn("input.title", "My new post");
  click("button.submit");

  andThen(function(){
    equal(find("ul.posts li:last").text(), "My new post");
  });
});

What! If that code isn't at least lightly blowing your mind, then you either understand Ember's testing helpers better than I did, or you may not be paying close enough attention. I'll illustrate with an example. We want to ensure that the `fillIn` call is not made until the `visit` call has finished (which could take an arbitrary amount of time, due to the fact that it may involve fetching data from an API), and likewise that the `click` call isn't made until the `fillIn` operation has finished. And, finally, any action that Ember takes as a result of clicking "button.submit" (which could involve sending data to an API and/or a route transition) is also finished before the `andThen`'s anonymous function containing our test's assertion is called.

Naive Promise-Returning Test Helpers


Let's take a stab at writing our own pseudo versions of the promise-returning `visit`, `fillIn`, `click, and `andThen` helper functions:
function visit(url){
  return new Ember.RSVP.Promise(function(resolve, reject){
    var timeout = Math.random() * 50; // arbitrary delay
    Ember.run.later(function(){
      // route-visiting code such as `beforeModel` etc
      // would go here
      console.log('Finished visiting ' + url);
      resolve();
    }, timeout);
  });
}

// assume fillIn, and click are defined similarly

// Now, run these lines:
visit('/posts/new');
fillIn("input.title", "My new post");
click("button.submit");

// Console output:
// (All three statements run at essentially the same time,
//  finishing in a non-deterministic order)
// 'Filled in input.title with My new post' <-- 2nd promise finishes first
// 'Clicked button.submit' <-- 3rd promise finishes before 1st
// 'Finished visiting /posts/new' <-- 1st promise finishes last

Here is a live example showing what happens if we take the naive approach described above. As you can see, just because the `visit`, `fillIn`, and `click` method calls are on subsequent lines, this does not guarantee that their promises resolve in that order. All three helpers start nearly at the same time (you'll see the "1a", "2a", and "3a" log lines appear nearly instantly), but they finish in a random order.
Ember Async Test Helpers Part I

Now it should be somewhat clearer why Ember's test helpers are mysterious. How can Ember ensure that the `fillIn` method call only happens after the `visit` one is done, if we haven't explicitly chained them together? In a normal javascript context, you cannot delay execution of a line based on a previous line. Javascript is non-blocking, so each line executes as soon as the previous one has, even if the previous line has asynchronous behavior that isn't finished yet. This is different from some other languages you probably use. In Ruby, for example, if you had a line that requested data from the database (e.g., `user = User.find(1)`), the execution of the next line of ruby would block waiting until the database had returned its data, and on the next line you could reference the `user` variable, knowing that it had been fully hydrated from the database.

This is not the case for javascript, however, and it is why we used chained promises when we need to guarantee the order of asynchronous method calls. We cannot get away from the need to chain these calls together, but we can get away from explicitly chaining them together, if we are clever.

Context-Aware Test Helpers


Let's redefine our versions of the `visit` and other async helpers. Now, instead of simply returning a promise, they will detect the presence of a global, shared-context promise, and automatically chain themselves to that if it exists. If it doesn't exist, they'll create it and then chain themselves to it. Now our `visit` function, for example, will look like this:
function visit(url){
  // create the shared promise
  // if it doesn't exist
  if (!window.sharedPromise) {
    window.sharedPromise = Ember.RSVP.resolve();
  }

  // chain our code to the shared promise
  window.sharedPromise = window.sharedPromise.then(function(){
    // this is the original `visit` code:
    return new Ember.RSVP.Promise(function(resolve, reject) {
      // ...
      // simulate delay and then call `resolve`
      // ...
    });
  });
}

Visually, this is what is going to happen now:
async-test-helper-animation
Here is a live example showing these global promise-aware pseudo test helpers in action: Ember Async Test Helpers Part II

Putting it all together


As it turns out, this is almost exactly what Ember's testing helpers do! The Ember testing code maintains a shared promise (stored on `Ember.Test.lastPromise`) that it chains each additional async helper call onto. This is the relevant code snippet:


The Ember testing code does a few additional things (such as attempting to resolve nested async promises properly), but at a high level all it is doing is using a shared promise global variable to chain each async helper onto. This is how Ember testing helpers allow you to write code with asynchronous behavior but write it in a synchronous style, without having to explicitly think about promises or promise chains.

At this point you may be wondering how Ember actually knows that the `visit` (or `click`, etc.) async helper has finished all its async behavior. So far we've assumed that the `visit` helper returns a promise that resolves when its async activity is finished, but we've glossed over the actual mechanism by which Ember ensures the promise resolves at the proper moment. The answer to this is that Ember internally uses a heuristic for async activity that involves checking for run loop activity, pending ajax requests, router activity, and more, but that is a subject for a later blog post. Please follow me (@bantic) on Twitter to hear about future posts.

Summary

The one takeaway here is that when writing integration tests in Ember, every line of your test should be an async helper or the anonymous function passed to `andThen`. That is, every line of your test should be one of `visit`, `fillIn`, `click`, etc., or an `andThen` call, or inside of the `andThen`'s anonymous function. If you do this, the async-aware nature of Ember's testing code will allow you to write simple, clean tests and be confident that the code will run in the correct order, without having to clutter your test code with explicit asynchronous control flow.
Apr 6 14

Hand-Crafted HTTP with Telnet: Exploring the protocol

by cory
I've been reading Ilya Grigorik's truly excellent High Performance Browser Networking lately and as I went through the HTTP section I realized that there were some gaps in my understanding of the protocol.

Fortunately, the HTTP 1.1 protocol is text-based, so we can create HTTP requests very easily by hand using telnet! The HTTP version 1.1 protocol is laid out in RFC 2616. It can look fairly intimidating at first but it's actually quite readable. At a high level, HTTP, which stands for Hypertext Transfer Protocol, outlines a mechanism for clients and servers to communicate to transfer resources (the R in URL) from one server to another. Your web browser is an HTTP client and it retrieves resources (HTML documents, CSS files, javascript files, images, etc) from HTTP servers.
If we want to make an HTTP request by hand, the most important place to start is Section 5 of RFC 2616, which explains how a client makes a request to a server. The format is, roughly:
Request-Line
Header line(s)
CRLF (empty line)
Message body (optional; for example if POSTing data)
What does Request-Line mean? The spec explains. Here it is, verbatim, where "SP" means whitespace.
Method SP Request-URI SP HTTP-Version CRLF
You're already familiar with the possible values for "Method" (GET/POST/PUT/etc). Request-URI is basically the url path that you are requesting (i.e., if you are asking for the root url, it is "/"). The http version is going to be 1.1, which is written as "HTTP/1.1".

So, to request the root URL at the server via GET, you would use this as the request line:
GET / HTTP/1.1
So far so good. The next part of the request is the headers. The full range of possible headers are described in Section 14. Suffice it to say for now that the headers are where you can include additional metadata about your request, such as your client's user-agent, the content-type, the language, caching options, and so on. The only one that is required by HTTP/1.1 is the "Host" header, which must match the address of the server that you are requesting the resource from. To request the homepage from google, then, this is the HTTP request text we would need to use.
GET / HTTP/1.1
Host: google.com
CRLF (empty line)
For example, say we have this very simple "Hello, World" node server, running on localhost at port 5000:

Here is what it looks like to connect via telnet and request the url "/":

telnetsimple
You may notice a couple things. First of all, as soon as an empty line (CRLF) is entered after the last header, we get back the server response. HTTP also defines a specific way for an HTTP server to respond, and it is fairly similar to the format of a client request. There's an initial line that includes the HTTP version and the status code ("HTTP/1.1 200 OK"), then some header lines, then a blank line (CRLF), then the message.

But why does it say "c" before the "Hello, World" text and "0" afterward? That wasn't part of our server's response, was it? That looks like random junk. Well, if you look at the headers again you'll notice that one of them specifies "Transfer-Encoding: chunked". Chunked Transfer Encoding wasn't in the HTTP/1.0 spec but has been added to the HTTP/1.1 spec. Chunked transfer encoding allows a server to send responses in, well, chunks, rather than doing it all at once. The HTTP spec says that before each chunk the server must send a line with the length of the following chunk (in hexadecimal), and a line with "0" on it after the chunk. If you count up the letters in "Hello, World" you'll get 12, which happens to equal "c" in hexadecimal.

Finally, if you're paying close attention you may further notice that the telnet session didn't end after the server sent us the "Hello, World" response. This is due to another change that was introduced in HTTP/1.1: persistent connections by default. In order to cut down on the latency overhead that is incurred by the TCP connection handshake, in HTTP/1.1 all connections are, by default, kept alive so that the client can request another resource without needing to re-connect. The server indicates that it intends to keep the connection alive by adding the "Connection: Keep-Alive" header to its response.

If we do not want this behavior, we can explicitly tell the server to close the connection by adding the "Connection: Close" header to our initial request. Here's an example of that:

telnetsimple-close

This time, the connection is immediately dropped by the server after it finishes responding.

Remember how I mentioned that the Host header is required by HTTP/1.1? Why would it be required? Shouldn't the server know what Host is being requested? The answer is, surprisingly, that it does not. The HTTP/1.0 standard assumed that one IP address was equivalent to one server, but we now have the concept of a multi-homed server, which just means that a single server at a single IP address can serve multiple domains. Without the Host header, a server running HTTP/1.0 can't tell which of the possible hosts should handle the incoming request.

In the follow-up to this post I'll look into how to hand-craft an HTTP request that actually sends data to the server. If you've ever wondered how the data in a form actually gets to the server, we'll go into that.
Feb 24 14

How does Release Date Affect Oscars Best Picture Wins?

by cory
I was looking for an excuse to play around with d3, and since the Oscars are coming up next weekend, I put together a little d3-powered chart showing the number of best picture wins by US release month.

If you're just here for the answer to the title question, it is: Quite a bit. About 25% of the winners were released in December, and over half were released in the 3 months at the end of the year.

I used the enjoyable and optimistic-sounding Cheerio npm module (DOM traversal/selection a la jQuery, but in node.js) to scrape , and followed along with Mike Bostock's gentle d3 bar chart tutorial to put the data together in jsbin.

Here are the results:
JS Bin

I was hoping to do a little more explanation of how this was generated, but I'll save that for a future post. The code is pretty quick-and-dirty, anyway. If you'd like to do something with the cleaned-up data, the movies.json file is on github.

What did I learn? Well, not a whole lot. My initial expectation was that the movies were released toward the end of the year. In retrospect, even though it seemed official, the Oscars.org best picture winners page was probably not the best one to use. I should have been clued in by the raw php code in the source HTML there. It seemed marginally easier to parse than Wikipedia, though, so I went with it, but the extra time spent cleaning up data by hand offset any gains.

I was a bit surprised to realize how complicated release dates can be. The same movie gets released multiple times in the same country, and also in other countries, and also sometimes gets re-released. I had to put together some simple heuristics to choose the most appropriate date from lists like this one for The Sting.

Not-too-surprising fact: Lots of movies sneak in a limited release somewhere late in December but aren't available widely in the US until the following year (like Million Dollar Baby, which had a limited release in mid-December and the wide release about 7 weeks later).

Most surprising thing: Casablanca appears to be the only film listed to win for a year other than the one in which it was released. According to IMDB it was released in 1942 yet somehow won the 16th Academy Awards in 1943. Perhaps the notion of 'release date' is even looser than I realized.
Feb 17 14

Ember Controller versus ObjectController

by cory
When doing Ember.js training, I commonly see confusion surrounding the different controller types that Ember offers. In this post I'll show a couple examples that illustrate the differences between the two.

Every time you you display a template in an ember application, you can use Handlebar's interpolation expressions to display property values in the rendered HTML (often called data binding). The source of these property values is the controller that Ember is using as the context for the handlebars template. Every time you display a template, its context is bound to some controller, either one that you have explicitly created or one that ember has auto-generated for you (you can see the generated controllers logged to the console if you create your app with "LOG_ACTIVE_GENERATION: true").

Ember.Controller



For example, if you have the template "Hello, {{name}}.", and a controller with the property `name: "Cory"`, then you'll see "Hello, Cory." You can see that below

Ember Starter Kit

Any property that is on the controller can be displayed by referencing it in the template. Properties can be simple key-value pairs as above, or the result of a function. When using a function as a property for display, you must add a `.property()` call to the end of the function definition. Pass any dependent keys as arguments to the `property()` call. For example, we can make a more sophisticated controller and template like this:

App.IndexController = Ember.Controller.extend({
  greeting: function(){
    if (this.get('isFancy')) {
      return "Greetings";
    } else {
      return "Hello";
    }
  }.property('isFancy'),
  name: "Cory"
});

Ember.TEMPLATES['index'] = Ember.Handlebars.compile("{{greeting}}, {{name}}");


This uses the controller's property `isFancy` to determine what type of greeting to display. Since we haven't specified `isFancy` explicitly yet, it will be falsy and the output of the template will still be "Hello, Cory".

If we explicitly define `isFancy` to be true, however, the output changes to "Salutations, Cory."

So far these examples all show usage of the standard controller, which we extended from `Ember.Controller`. There's another type of controller Ember offers, called the `ObjectController`, which we'll look at now.

Ember.ObjectController

A typical use case in Ember is to load in model data via an ajax call and display some of it to the user in the template. Typically the data is loaded in and returned from the `model` hook in the route. We'll make that change now, explicitly defining a route in our code instead of letting Ember auto-generate it as before. (If you have `LOG_ACTIVE_GENERATION` set to true, you'd see something like "generated -> route:index" in the console, pointing out that the index route had been automatically generated.)

App.IndexRoute = Ember.Route.extend({
  model: function(){
    var author = Ember.Object.create({name: 'Cory'});
    return author;
  }
});


This example isn't loading in data via ajax, but it will have the same result regardless. The model hook just has to return an object or a promise that resolves to an object.

Since we are planning on displaying model data in our template, it's time to change our controller to an ObjectController. Ember will automatically set the "model" property on our controller to the return value of the matching route's `model` hook. In this case we return the `author` object from `IndexRoute`'s `model` hook, so it will be set to the "model" property of the IndexController. We can now wire up our original starting template using the ObjectController instead. Knowing that its "model" property is the author object, along with what we learned earlier about property functions on a controller, we could get the same behavior we started with by doing this:

App.IndexRoute = Ember.Route.extend({
  model: function(){
    var author = Ember.Object.create({name: 'Cory'});
    return author;
  }
});

App.IndexController = Ember.ObjectController.extend({
  name: function(){
    return this.get('model.name');
  }.property('model.name')
});

Ember.TEMPLATES['index'] = Ember.Handlebars.compile("Hello, {{name}}");


This feels a little awkward and luckily Ember has anticipated this use case already and offers us a much terser option. The ObjectController is so named because it proxies property lookups to its `model` property (its "object", so to speak). When our template references the `name` property, the ObjectController's default behavior is to look up the `name` property on its model, exactly as we have just manually done above. This means we can remove all of this code and instead say:

App.IndexController = Ember.ObjectController.extend({});


The output is unchanged. Ember Starter Kit

Proxying Properties to the ObjectController's Model

Earlier I said that the ObjectController proxied property lookups to its model, and although this is correct there's an important caveat to mention: The ObjectController will first check whether the requested property has been defined on it. Only if the property has not been defined will it look up the property value on the model.

As an example, consider setting a `name` property on the ObjectController explicitly. What would happen in that case?

App.IndexController = Ember.ObjectController.extend({
  name: "impostor"
});
Ember Starter Kit

The index controller will intercept the lookup of the `name` property and return its own `name` property, effectively masking the `name` property on the model. This usually isn't desired behavior and can be the source of some confusion. Most of the time we will want to add properties to an ObjectController it will be to display different data that is not the model's direct concern. A very common example of this would be to display different items in the interface when the controller has been switched into "edit mode" (i.e., had its "isEditing" flag set to true) by the user. The `isEditing` property isn't the model's concern, it's a controller concern.

To continue building on our example, we'll once again make our code show a fancy greeting. This time, though, we'll assume that whether or not a fancy greeting should be displayed is the concern of the user viewing our app, not something that the author object itself ought to be concerned with. A perfect opportunity to augment the controller itself.

Adding the `isFancy` simple boolean property, and the `greeting` property function that we used before, onto the ObjectController allows us to show the fancier greeting once again. Here's the code, with a checkbox to toggle the "isFancy" property

Ember Starter Kit

I hope this illustrates the difference between normal Ember controllers and ObjectControllers, as well as how to use both. There are a few subtle ways to get tripped up when using one or the other that I'll list below.

Gotchas

Using a model hook in the route but not using an ObjectController

If you have set up a route to return a model but not set the corresponding controller to be an ObjectController, the properties you reference in your template will be looked up on the controller directly, resulting in nothing displaying in your template. This can be an easy one to overlook, especially as Ember will happily render your template without raising any errors about the missing properties.

Ember Starter Kit

Overriding setupController but not Setting the Controller's Model

We didn't cover it here, but in a Route you can define a `setupController` hook to do extra setup work on the controller. Ember's default controller setup behavior, as mentioned above, is to set the "model" property on the controller to the result of the "model" hook on the route. If you override the `setupController` hook, however, Ember won't do this default behavior for you anymore.

For example, suppose we decided to add a `setupController` hook to our fancy greeter's IndexRoute to set the "isFancy" property to true when entering this route? We could add this hook to our route like so:

setupController: function(controller, model) {
  controller.set('isFancy', true);
}


Unfortunately, although this does make it so the fancy checkbox is checked, we've lost the display of the name:

Ember Starter Kit

The solution is to either set the "model" property explicitly or just call `this._super(controller, model)` and allow Ember's default `setupController` behavior to happen. Here is is, fixed:

Ember Starter Kit

If you have any related questions, I'm @bantic on twitter. Get in touch. :)