| Authors: | Alex Russell
Pete Higgins Dustin Machi Jared Jurkiewicz |
|---|
Contents
Usually when people think of a scripting language, it's often in context of ease of use, simple syntax, and something those without traditional computer science educations can grasp and use. JavaScript unfortunately falls into this category. For a scripting language, though, is difficult to fully master. Making it worse are the environments that execute JavaScript; the infamous Web Browser. For example, Safari works similar to, but not quite the same as Mozilla FireFox, Opera, and even Google Chrome. When Internet Explorer enters the picture, it gets even more complicated as many JavaScript constructs (and DOM functions) that work in all the other browsers will not work in Internet Explorer. This becomes a Testing Nightmare. In fact, I'm sure the frustration of testing JavaScript has caused many an utterance of 'doh!' and other less ahem, polite phrases. So ... what can be done about this?
Just to get it out of the way ... D.O.H. is both an acronym of Dojo Objective Harness and a play on the now commonplace expression of frustration 'DOH!'. But aside from the very silly name it is a unit test framework developed by the Dojo Toolkit Community (primarily by Alex Russell), in an attempt to solve the complexities of testing JavaScript. For example, in JavaScript, the use of XHR to request remote data asynchronously is commonplace ... and yet few Unit Test frameworks provided any mechanism by which to actually be able test asynchronous functions. Furthermore, most Unit Test Frameworks end up with specific dependencies on certain browsers or runtime environments. Again, D.O.H. tries not to limit itself in this manner, it's both flexible, extendable, and runs in many environments from browsers to JS runtime environments such as Rhino.
It's main purpose is a framework for developers (both dojo committers and others), to unit test out JavaScript functions and custom widgets. Its goal isn't so much as a way to test a full-blown web application as it is to test the parts that make up the applications in a cross-browser (and even non-browser), compatible manner. By having a good set of unit tests, developers can feel confident that code changes they later make do not cause regressions in existing functions.
For a quick list of the features of D.O.H. see below:
- Firefox 3.6+
- Safari 4.x+
- Internet Explorer 6+
- Google Chrome All versions
- Opera 9.x and 10.x
- <and others!>
D.O.H. resides in the utils project of the dojo toolkit. Users can find it explicitly at: utils/doh. The following files structure shows the files important to the usage of the framework:
Okay, that's wonderful! It's a unit test environment that's flexible. So ... how the heck does someone use it? Well, let's start small. The first thing we can do is look at using the browser runner to run a small set of unit tests in dojo. This is done just by loading runner.html in a web browser. What runner.html does is load runner.js and _browserRunner.js. This defines a global object, 'doh', that has numerous test registration and assertion functions that can be used by unit tests. Runner.html also does something else, if no module URL was provided for it to load tests from, the runner.html will automatically load the file: dojo/tests/module.js. All that file does is use the dojo module loading system to load in all the test files for core dojo. It then just executes all the currently registered tests and displays the results in the browser.
Okay, so starting up D.O.H. is as simple as running runner.html ... but now you may be asking how do I load my tests? Simple! You pass it as a query parameter to the runner.html. For example, say you just want to run the dojo.date tests, you would load the following in your browser:
util/doh/runner.html?testModule=dojo/tests/date
What this does is instruct the runner to look in dojo/tests and load the date.js file. Please note here that you do not have to put your unit tests in dojo/tests! The browser runner just has a default search path to look in dojo/ for a module in addition to looking at the assumed path of tests/date.js.
What most developers do is define their own tests in their module's directory under tests/. you can see this as a common pattern in dojox. For example, dojox/data/tests have all the dojox.data unit tests. You can also note that dojox/data/tests has a runTests.html. All that runTests.html does is redirect back to util/doh/runner.html and set the testModule parameter. It's a simple helper file that module owners use to quick-launch their unit tests.
Okay, whee! The runner.html can be told what test module to load. So ... what the heck is a test module!? A test module is nothing more than a file that requires in more test files, calls doh.register() to register tests, or a combination of both. Basically a module is nothing more than the file that calls the doh functions to register tests before D.O.H. executes the test run. For example, the following code snippet is a test module:
define(["doh/runner"], function(doh){
doh.register("MyTests", [
function assertTrueTest(){
doh.assertTrue(true);
doh.assertTrue(1);
doh.assertTrue(!false);
},
{
name: "thingerTest",
setUp: function(){
this.thingerToTest = new Thinger();
this.thingerToTest.doStuffToInit();
},
runTest: function(){
doh.assertEqual("blah", this.thingerToTest.blahProp);
doh.assertFalse(this.thingerToTest.falseProp);
// ...
},
tearDown: function(){
}
},
// ...
]);
});
define([
"my/test/widget/Foo0",
"my/test/widget/Foo1",
"my/test/widget/Foo2"
]);
(Dojo 1.4 and higher)
Unit tests are excellent for testing code behavior and verifying the code works as expected across browsers. But it's not always what you want to do. There are times where code may work across browsers, but the code doesn't work fast on all browsers. It would be great if there was a way to do performance analysis of a function instead of just doing true/false unit test style testing.
The great news is that as of Dojo 1.4, D.O.H. has been updated to understand a different type of test fixture (or test registration), the performance test. Tests registered in this manner work a little differently. Instead of a single execution of 'runTest' or of the test function, it will execute that test function many times and then calculate the average cost for a single function run. This is powerful in determining the general function performance across browsers and identify implementations that underperform on various browsers.
What a performance fixture does is the following:
{
trial: number, // The trial number, 0 .. N trial run
testIterations: number, // The number of iterations the test function was run for the trial.
executionTime: number, // The total execution time of the trial, in milliseconds.
average: number // The average time a single iteration of the test function takes. executionTime/testIterations
}
You can also look at how to write performance tests by looking at: dojox/gfx/tests/performance/* in your extract of Dojo 1.4 or development trunk.
This section is intended for documentation on the various aspects of D.O.H., such as what registration functions there are and what they do, to information on test formats.
These functions are how you register tests within the D.O.H. framework to run. The two main functions you will use are doh.register and doh.registerUrl. The others are provided for completeness.
An almost 'magical' function. The doh.register() method accepts the function signatures of any of the other registration functions and determines the correct underlying function (listed below) to dispatch registration to. It's the function you'll most commonly use for registering Unit Tests.
This function registers a test as a member of the group 'group', and the test can either be a simple function definition or a 'Test Fixture', which is an object that defines the run requirements of the test.
This function registers an array of tests as a member of the group 'group'. The contents of the array of tests can be an array of simple test functions or an array of 'test fixtures', or a mix of them.
This function registers an object comprised of functions as a member of the group 'group'. Note that this function will only add in non-private (functions without an _ at the beginning of the name), as a test function. If you'd like to use fixtures (setUp(), tearDown(), and runTest()), please use doh.register(), doh.registerTest() or doh.registerTests().
This function registers a URL as a location to load tests from. The URL is used to populate the contents of an iframe, and usually refers to an HTML page that boot-loads D.O.H. internally for running tests in a segmented iframe. A good example showing this is the dojo/tests/fx.html. It loads dojo, doh, and then on dojo load completion calls doh.registerTests(). The D.O.H. instance in the iframe will proxy back the results of the test run to the primary D.O.H. instance.
These functions are the test functions you use inside your unit tests in order to check conditions are report errors if the conditions are not met. These should be familiar to anyone who has used other test frameworks before, such as JUnit.
This function asserts that particular condition is true. If the condition is not true, the function will throw an Error object.
- Note: This function is aliased to doh.t();
This function asserts that particular condition is false. If the condition is not false, the function will throw an Error object.
- Note: This function is aliased to doh.f();
This function asserts that the two particular inputs are 'equal to each other, such as 5 == 5, or "Bob" == "Bob". The equality is fairly loose, so this is not a good function to use if you want to assert something is equal and of the same data type. For example, this function will return that 5 == "5" (number 5 equals string 5). It will also do deep object equality, so beware of object cycles. If the equality condition is not met, the function will throw and Error object.
- Note: This function is aliased to doh.is();
This function asserts that the two particular inputs are not 'equal to each other, such as 5 != 6, or "Bob" == "Doe". The equality is fairly loose, so this is not a good function to use if you want to assert something is not equal and of the same data type. It will also do deep object equality/inequality when checking, so beware of object cycles. If the 'not equal' condition is not met, the function will throw and Error object.
- Note: This function is aliased to doh.isNot();
D.O.H. is flexible in how you define tests. In some respects, perhaps it is too flexible in that it can be confusing which way to define a test or how to define an asynchronous test. This following section is intended to help explain the various test function formats and when to use them.
A lot of testing involves checking a widget or JavaScript API or the like and doesn't require any initial setup or teardown to do. These are simple to define. You simply write it as a function object. This object can then be registered through the doh.register() functions.
function mySimpleTest(doh){
doh.assertTrue(true);
}
What the D.O.H. framework will do with this test is wrap it in a default 'test fixture', which has null setUp and tearDown functions and a default timeout of 500ms.
Simple asynchronous tests are no more complicated to write than the above synchronous test. You simply write it as a function object that returns a doh.Deferred object. The test framework detects the return type and knows that if a doh.Deferred is returned, then D.O.H. should pause here and wait for either the test timeout to fire or the asynchronous test to invoke either the Deferred to resolve, which is done via the getTestCallback() method. This object can then be registered through the doh.register() functions. The example below implements an async test via timeout to demonstrate it.
function mySimpleAsyncTest(doh){
var deferred = new doh.Deferred();
setTimeout(deferred.getTestCallback(function(){
doh.assertTrue(true);
}), 100);
return deferred;
}
What the D.O.H. framework will do with this test is wrap it in a default 'test fixture', which has null setUp and tearDown functions and a default timeout of 500ms.
Some testing will require setup and/or teardown, specific pre and post configurations that need to occur ... or the test requires a different timeout from the default (such as a long-running xhr). The test fixture format for a test allows for this configuration. See the following code snippit for an example of a test fixture.
{
name: "thingerTest",
setUp: function(){
// Setup to do before runTest.
this.thingerToTest = new Thinger();
this.thingerToTest.doStuffToInit();
},
runTest: function(){
// Our test function to run.
doh.assertEqual("blah", this.thingerToTest.blahProp);
doh.assertFalse(this.thingerToTest.falseProp);
// ...
},
tearDown: function(){
// cleanup to do after runTest.
},
timeout: 3000 // 3 second timeout.
}
Test fixtures can also do asynchronous tests. All that has to occur for the D.O.H. framework to recognize that a test fixture is asynchronous is that the runTest method returns a doh.Deferred or dojo.Deferred. See below for a fixture that operates asynchronously.
{
name: "thingerTest",
setUp: function(){
// Setup to do before runTest.
this.thingerToTest = new Thinger();
this.thingerToTest.doStuffToInit();
},
runTest: function(){
// Our test function to run.
var deferred = new doh.Deferred();
setTimeout(deferred.getTestCallback(function(){
doh.assertEqual("blah", this.thingerToTest.blahProp);
doh.assertFalse(this.thingerToTest.falseProp);
}), 100);
return deferred;
},
tearDown: function(){
// cleanup to do after runTest.
},
timeout: 3000 // 3 second timeout.
}
(Dojo 1.4 and higher)
Performance test fixtures are just like a regular test fixture, except that it has some extra options developers make use of. Specifically, it uses 'testType' to mark it as a "perf" test, which instructs the D.O.H. runner to treat the tests as performance and use the calibrate and execute test runner. It also allows users to define how long a performance test should run per trial (we recommend 50ms or more to avoid JS timing issues), how many trials to run, and how long to delay between each trial to give GC time to the browser.
{
testType: "perf", // Define this as a performance test. Used to select the runner in D.O.H.
trialDuration: 100, // Define that a trial (test run) of the performance test should run minimally for 100ms (however many runs of the function this means. It is calibrated).
trialIterations: 100, // Run 100 trials of the test function.
trialDelay: 100, // Wait 100MS between each trial to allow for GC, etc.
name: "thingerPerformanceTest",
setUp: function(){
// Setup to do before the trial runs of runTest.
},
runTest: function(){
// Our test function to do performance profiling.
myModule.doSomePerformanceRelatedThing();
},
tearDown: function(){
// cleanup to do after all the trials.
}
}
(Dojo 1.4 and higher)
Async performance test fixtures are just like synchronous performance test fixtures. The only difference is that these return doh.Deferreds to tell the runner framework to treat the test as async. Note: Performance profiling asynchronous operations will not always provide useful information. By the nature of being async means its performance profile will likely vary considerably. The below example demonstrates an async performance fixture through the use of setTimeout.
{
testType: "perf", // Define this as a performance test. Used to select the runner in D.O.H.
trialDuration: 100, // Define that a trial (test run) of the performance test should run minimally for 100ms (however many runs of the function this means. It is calibrated).
trialIterations: 100, // Run 100 trials of the test function.
trialDelay: 100, // Wait 100MS between each trial to allow for GC, etc.
name: "thingerAsyncPerfTest",
setUp: function(){
// Setup to do before runTest.
},
runTest: function(){
// Our test function to run.
var deferred = new doh.Deferred();
setTimeout(deferred.getTestCallback(function(){
myModule.doSomePerformanceRelatedThing();
}), 100);
return deferred;
},
tearDown: function(){
},
timeout: 3000 // 3 second timeout.
}