TestBox is a xUnit style and behavior-driven development (BDD) testing framework for CFML (ColdFusion). It does not depend on any other framework for operation and it has a clean and descriptive syntax that will make you giggle when writing your tests. It ships with not only the xUnit and BDD testing capabilities, but also an assertions and expectations library, several different runners, reporters and MockBox, for mocking and stubbing capabilities.

Info Note: This article is a primer to get you started with the xUnit capabilities of TestBox, if you would like to explore the BDD style capabilities of TestBox we recommend you visit the BDD primer chapter.

The xUnit functionalities of TestBox will require Railo 4.1+, Lucee 4.5+ or ColdFusion 9.01+.

TestBox relies on the fact of creating testing bundles which are basically CFCs. A bundle CFC will hold all the tests the TestBox runner will execute and produce reports on. Thus, sometimes this test bundle is referred to as a test suite in xUnit terms.

This bundle can contain 2 life-cycle methods, the beforeTests() and afterTests() methods will execute once before ALL your tests run and then after ALL your tests run. This is a great way to do any kind of global setup or teardown in your tests. It also contains a displayName argument in the component declaration which gives you a way to name your testing suite. We will see later the other annotations you can add to the component declaration further in the primer.

TestBox not only provides you with global life-cycle methods but also with localized test methods. This is a great way to keep your tests DRY (Do not repeat yourself)! TestBox provides the setup( currentMethod ) and teardown( currentMethod ) methods that each receives the name of the test method in question. which as their names indicate, they execute before a test and after a test in a test bundle.

component displayName="TestBox xUnit suite" labels="railo,cf"{

     function setup( currentMethod ){
          application.wirebox = new coldbox.system.ioc.Injector();
          structClear( request );
     }

     function teardown( currentMethod ){
          structDelete( application, "wirebox" );
          structClear( request );
     }

     function testThrows(){
          $assert.throws(function(){
               var hello = application.wirebox.getInstance( "myINvalidService" ).run();
          });
     }

     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = application.wirebox.getInstance( "MyValidService" ).run();;
          });
     }

}

Assertions are self-concatenated strings that evaluate an actual value to an expected value or condition. These are initiated by the global TestBox variable called $assert which contains tons of included assertion methods so you can evaluate your tests.

Evaluators

Each assertion evaluator will compare the actual value and an expected value or condition. It is responsible for either passing or failing this evaluation and reporting it to TestBox. Each evaluator also has a negative counterpart assertion by just prefixing the call to the method with a not expression.

      function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

Included Evaluators

TestBox has a plethora (That's Right! I said Plethora) of evaluators that are included in the release. The best way to see all the latest evaluator methods is to visit our API and digest the coldbox.system.Assertion class. There is also the ability to register and write custom assertion evaluators in TestBox via our addAssertions() function.

component displayName="TestBox xUnit suite for CF9" labels="railo,cf"{

/*********************************** LIFE CYCLE Methods ***********************************/

     function beforeTests(){
          application.salvador = 1;
     }

     function afterTests(){
          structClear( application );
     }

     function setup(){
          request.foo = 1;
     }

     function teardown(){
          structClear( request );
     }

/*********************************** Test Methods ***********************************/

     function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testIncludesWithCase(){
          $assert.includesWithCase( "hello", "he" );
          $assert.includesWithCase( [ "Monday", "Tuesday" ] , "Monday" );
     }

     function testnotIncludesWithCase(){
          $assert.notincludesWithCase( "hello", "aa" );
          $assert.notincludesWithCase( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

     function testIsEmpty(){
          $assert.isEmpty( [] );
          $assert.isEmpty( {} );
          $assert.isEmpty( "" );
          $assert.isEmpty( queryNew("") );
     }

     function testIsNotEmpty(){
          $assert.isNotEmpty( [1,2] );
          $assert.isNotEmpty( {name="luis"} );
          $assert.isNotEmpty( "HelloLuis" );
          $assert.isNotEmpty( querySim( "id, name
               1 | luis") );
     }

     function testSkipped() skip{
          $assert.fail( "This Test should fail" );
     }

     boolean function isRailo(){
          return structKeyExists( server, "railo" );
     }

     function testSkippedWithConstraint() skip="isRailo"{
          $assert.fail( "This Test should fail" );
     }

     function testFails(){
          //$assert.fail( "This Test should fail" );
     }

     function testFailsShortcut() labels="railo"{
          //fail( "This Test should fail" );
     }

     function testAssert() {
          $assert.assert( application.salvador == 1 );
     }

     function testAssertShortcut() {
          assert( application.salvador == 1 );
     }

     function testisTrue() {
          $assert.isTrue( 1 );
     }

     function testisFalse() {
          $assert.isFalse( 0 );
     }

     function testisEqual() {
          $assert.isEqual( 0, 0 );
          $assert.isEqual( "hello", "HEllO" );
          $assert.isEqual( [], [] );
          $assert.isEqual( [1,2,3, {name="hello", test="this"} ], [1,2,3, {test="this", name="hello"} ] );
     }

     function testisNotEqual() {
          $assert.isNotEqual( this, new coldbox.system.MockBox() );
          $assert.isNotEqual( "hello", "test" );
          $assert.isNotEqual( 1, 2 );
          $assert.isNotEqual( [], [1,3] );
     }

     function testisEqualWithCase() {
          $assert.isEqualWithCase( "hello", "hello" );
     }

     function testnullValue() {
          $assert.null( javaCast("null", "") );
     }

     function testNotNullValue() {
          $assert.notNull( 44 );
     }

     function testTypeOf() {
          $assert.typeOf( "array", [ 1,2 ] );
          $assert.typeOf( "boolean", false );
          $assert.typeOf( "component", this );
          $assert.typeOf( "date", now() );
          $assert.typeOf( "time", timeformat( now() ) );
          $assert.typeOf( "float", 1.1 );
          $assert.typeOf( "numeric", 1 );
          $assert.typeOf( "query", querySim( "id, name
               1 | luis") );
          $assert.typeOf( "string", "hello string" );
          $assert.typeOf( "struct", { name="luis", awesome=true } );
          $assert.typeOf( "uuid", createUUID() );
          $assert.typeOf( "url", "http://www.coldbox.org" );
     }

     function testNotTypeOf() {
          $assert.notTypeOf( "array", 1 );
          $assert.notTypeOf( "boolean", "hello" );
          $assert.notTypeOf( "component", {} );
          $assert.notTypeOf( "date", "monday" );
          $assert.notTypeOf( "time", "1");
          $assert.notTypeOf( "float", "Hello" );
          $assert.notTypeOf( "numeric", "eeww2" );
          $assert.notTypeOf( "query", [] );
          $assert.notTypeOf( "string", this );
          $assert.notTypeOf( "struct", [] );
          $assert.notTypeOf( "uuid", "123" );
          $assert.notTypeOf( "url", "coldbox" );
     }

     function testInstanceOf() {
          $assert.instanceOf( new coldbox.system.MockBox(), "coldbox.system.MockBox" );
     }

     function testNotInstanceOf() {
          $assert.notInstanceOf( this, "coldbox.system.MockBox" );
     }

     function testMatch(){
          $assert.match( "This testing is my test", "(TEST)$" );
     }

     function testMatchWithCase(){
          $assert.match( "This testing is my test", "(test)$" );
     }

     function testNotMatch(){
          $assert.notMatch( "This testing is my test", "(hello)$" );
     }

     function testKey(){
          $assert.key( {name="luis", awesome=true}, "awesome" );
     }

     function testNotKey(){
          $assert.notKey( {name="luis", awesome=true}, "test" );
     }

     function testDeepKey(){
          $assert.deepKey( {name="luis", awesome=true, parent = { age=70 } }, "age" );
     }

     function testNotDeepKey(){
          $assert.notDeepKey( {name="luis", awesome=true, parent = { age=70 } }, "luis" );
     }

     function testLengthOf(){
          $assert.lengthOf( "heelo", 5 );
          $assert.lengthOf( [1,2], 2 );
          $assert.lengthOf( {name="luis"}, 1 );
          $assert.lengthOf( querySim( "id, name
               1 | luis"), 1 );

     }

     function testNotLengthOf(){
          $assert.notLengthOf( "heelo", 3 );
          $assert.notLengthOf( [1,2], 5 );
          $assert.notLengthOf( {name="luis"}, 5 );
          $assert.notLengthOf( querySim( "id, name
               1 | luis"), 0 );

     }

     // railo 4.1+ or CF10+
      function testThrows(){
          $assert.throws(function(){
               var hello = invalidFunction();
          });
     }

      // railo 4.1+ or CF10+
     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = 1;
          });
     }

/*********************************** NON-RUNNABLE Methods ***********************************/

     function nonStandardNamesWillNotRun() {
          fail( "Non-test methods should not run" );
     }

     private function privateMethodsDontRun() {
          fail( "Private method don't run" );
     }

}

Custom Assertions

You can also register custom assertions within the $assert object. You will do this by reading our Custom Assertions section of our TestBox docs.

TestBox discovers test methods in your bundle CFC by applying the following discovery rules:

• Any method that has a test annotation on it

• Any public method that starts or ends with the word test

// Via inline annotation
function shouldBeAwesome() test{}

/**
* Via comment annotation
* @test
*/
function shouldBeAwesome(){}

// via conventions
function testShouldDoThis(){}
function shouldDoThisTest(){}

Each test method will test the state of the SUT (software under test) or sometimes referred to as code under test. It will do so by asserting that actual values from an execution match an expected value or condition. TestBox offers an assertion library that you have available in your bundle via the injected variable $assert. You can also use our expectations library if you so desire, but that is mostly used in our BDD approach.

function testIncludes(){
       $assert.includes( "hello", "HE" );
       $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
}

Each test function can also have some cool annotations attached to it.

We have prepared an awesome PDF ref card for working with TestBox xUnit:

Please refer to our section to take advantage of all the mocking and stubbing you can do. However, every BDD TestBundle has the following functions available to you for mocking and stubbing purposes:

• makePublic( target, method, newName ) - Exposes private methods from objects as public methods

• querySim( queryData ) - Simulate a query

• getMockBox( [generationPath] ) - Get a reference to MockBox

• createEmptyMock( [className], [object], [callLogging=true]) - Create an empty mock from a class or object

• createMock( [className], [object], [clearMethods=false], [callLogging=true]) - Create a spy from an instance or class with call logging

• prepareMock( object, [callLogging=true]) - Prepare an instance of an object for method spies with call logging

• createStub( [callLogging=true], [extends], [implements]) - Create stub objects with call logging and optional inheritance trees and implementation methods

• getProperty( target, name, [scope=variables], [defaultValue] ) - Get a property from an object in any scope

Tests and suites can be tagged with TestBox labels. Labels allows you to further categorize different tests or suites so that when a runner executes with labels attached, only those tests and suites will be executed, the rest will be skipped. Labels can be applied globally to the component declaration of the test bundle suite or granularly at the test method declaration.

TestBox comes also with a nice plethora of reporters:

• ANTJunit : A specific variant of JUnit XML that works with the ANT junitreport task

• Codexwiki : Produces MediaWiki syntax for usage in Codex Wiki

• Console : Sends report to console

• Doc : Builds semantic HTML to produce nice documentation

• Dot : Builds an awesome dot report

• JSON : Builds a report into JSON

• JUnit : Builds a JUnit compliant report

• Raw : Returns the raw structure representation of the testing results

• Simple : A basic HTML reporter

• Text : Back to the 80's with an awesome text report

• XML : Builds yet another XML testing report

• Tap : A test anything protocol reporter

• Min : A minimalistic view of your test reports

• MinText : A minimalistic view of your test reports for consoles

• NodeJS : User-contributed:

Tests and suites can be skipped from execution by using the skip annotation in the component or function declaration. The reporters will show that these suites or tests where skipped from execution. The value of the skip annotation can be a simple true or false or it can be the name of a UDF that exists in the same bundle CFC. This UDF must return a boolean value and it is evaluated at runtime.

You can tag a bundle component declaration with the boolean asyncAll annotation and TestBox will execute all specs in separate threads for you concurrently.

Caution Once you delve into the asynchronous world you will have to make sure your tests are also thread safe (var-scoped) and provide any necessary locking.

TestBox ships with several test runners internally but we have tried to simplify and abstract it with our TestBox object which can be found in the testbox.system package. The TestBox object allows you to execute tests from a CFC, CFM, HTTP, NodeJS, SOAP or REST. You can also make your CFC's extend from our BaseSpec class so you can execute it directly via the URL. The main execution methods are:

Info We encourage you to read the API docs included in the distribution for the complete parameters for each method.

run() Arguments

Here are the arguments you can use for initializing TestBox or executing the run() method

runRemote() arguments

Here are the arguments you can use for executing the runRemote() method:

The bundles argument which can be a single CFC path or an array of CFC paths or a directory argument so it can go and discover the test bundles from that directory. The reporter argument can be a core reporter name like: json,xml,junit,raw,simple,dots,tap,min,etc or it can be an instance of a reporter CFC. You can execute the runners from any cfm template or any CFC or any URL, that is up to you.

Global Runner

TestBox ships with a global runner that can be used to run pretty much anything. You can customize it or place it wherever you need it:

Test Browser

TestBox ships with a test browser that is highly configurable to whatever URL accessible path you want. It will then show you a test browser where you can navigate and execute not only individual tests, but also directory suites as well.

ANT Runner

In our test samples and templates we include an ANT runner that will be able to execute your tests via ANT. It can also leverage our ANTJunit reporter to use the junitreport task to produce JUnit compliant reports as well. You can find this runner in the test samples and runner template directory.

Bundle(s) Runner

Test Runner

Directory Runner

SOAP Runner

You can run tests via SOAP by leveraging the runRemote() method. The WSDL URL will be

HTTP/REST Runner You can run tests via HTTP/REST by leveraging the runRemote() endpoint. The URL will be

NodeJS Runner

There is a user-contributed NodeJS Runner that looks fantastic and can be downloaded here:

Just use node to install: