1 of 11

xUnit Tests

Unit testing is a software testing technique where individual components of a software application, known as units, are tested in isolation to ensure they work as intended. Each unit is a small application part, such as a function or method, and is tested independently from other parts. This helps identify and fix bugs early in the development process, ensures code quality, and facilitates easier maintenance and refactoring. Tools like TestBox allow developers to create and run automated unit tests, providing assertions to verify the correctness of the code.

TestBox supports xUnit style of testing, like in other languages, via the creation of classes and functions that denote the tests to execute. You can then evaluate the test either using or the library included with TestBox.

You will start by creating a test bundle (Usually with the word Test in the front or back), example: UserServiceTest or TestUserService.

Test Bundles

The testing bundle CFC is actually the suite in xUnit style as it contains all the test methods you would like to test with. Usually, this CFC represents a test case for a specific software under test (SUT), whether that's a model object, service, etc. This component can have some cool annotations as well that can alter its behavior.

component displayName="The name of my suite" asyncAll="boolean" labels="list" skip="boolean"{

}

TestBox relies on the fact of creating testing bundles which are basically CFCs. A bundle CFC will hold all the tests the TestBox runner will execute and produce reports on. Thus, sometimes this test bundle is referred to as a test suite in xUnit terms.

component displayName="My test suite" extends="testbox.system.BaseSpec"{

     // executes before all tests
     function beforeTests(){}

     // executes after all tests
     function afterTests(){}

}

Bundle Annotations

Caution If you activate the asyncAll flag for asynchronous testing, you HAVE to make sure your tests are also thread safe and appropriately locked.

Life-Cycle Methods

TestBox not only provides you with global life-cycle methods but also with localized test methods. This is a great way to keep your tests DRY (Do not repeat yourself)!

beforeTests() - Executes once before all tests for the entire test bundle CFC
afterTests() - Executes once after all tests complete in the test bundle CFC
setup( currentMethod ) - Executes before every single test case and receives the name of the actual testing method
teardown( currentMethod ) - Executes after every single test case and receives the name of the actual testing method

Examples

Test Methods

TestBox discovers test methods in your bundle CFC by applying the following discovery rules:

Any method that has a test annotation on it
Any public method that starts or ends with the word test

// Via inline annotation
function shouldBeAwesome() test{}

/**
* Via comment annotation
* @test
*/
function shouldBeAwesome(){}

// via conventions
function testShouldDoThis(){}
function shouldDoThisTest(){}

Each test method will test the state of the SUT (software under test) or sometimes referred to as code under test. It will do so by asserting that actual values from an execution match an expected value or condition. TestBox offers an assertion library that you have available in your bundle via the injected variable $assert. You can also use our expectations library if you so desire, but that is mostly used in our BDD approach.

function testIncludes(){
       $assert.includes( "hello", "HE" );
       $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
}

Each test function can also have some cool annotations attached to it.

Assertions

Assertions are self-concatenated strings that evaluate an actual value to an expected value or condition. These are initiated by the global TestBox variable called $assert which contains tons of included assertion methods so you can evaluate your tests.

Evaluators

Each assertion evaluator will compare the actual value and an expected value or condition. It is responsible for either passing or failing this evaluation and reporting it to TestBox. Each evaluator also has a negative counterpart assertion by just prefixing the call to the method with a not expression.

      function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

Included Evaluators

TestBox has a plethora (That's Right! I said Plethora) of evaluators that are included in the release. The best way to see all the latest evaluator methods is to visit our API and digest the coldbox.system.Assertion class. There is also the ability to register and write custom assertion evaluators in TestBox via our addAssertions() function.

component displayName="TestBox xUnit suite for CF9" labels="railo,cf"{

/*********************************** LIFE CYCLE Methods ***********************************/

     function beforeTests(){
          application.salvador = 1;
     }

     function afterTests(){
          structClear( application );
     }

     function setup(){
          request.foo = 1;
     }

     function teardown(){
          structClear( request );
     }

/*********************************** Test Methods ***********************************/

     function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testIncludesWithCase(){
          $assert.includesWithCase( "hello", "he" );
          $assert.includesWithCase( [ "Monday", "Tuesday" ] , "Monday" );
     }

     function testnotIncludesWithCase(){
          $assert.notincludesWithCase( "hello", "aa" );
          $assert.notincludesWithCase( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

     function testIsEmpty(){
          $assert.isEmpty( [] );
          $assert.isEmpty( {} );
          $assert.isEmpty( "" );
          $assert.isEmpty( queryNew("") );
     }

     function testIsNotEmpty(){
          $assert.isNotEmpty( [1,2] );
          $assert.isNotEmpty( {name="luis"} );
          $assert.isNotEmpty( "HelloLuis" );
          $assert.isNotEmpty( querySim( "id, name
               1 | luis") );
     }

     function testSkipped() skip{
          $assert.fail( "This Test should fail" );
     }

     boolean function isRailo(){
          return structKeyExists( server, "railo" );
     }

     function testSkippedWithConstraint() skip="isRailo"{
          $assert.fail( "This Test should fail" );
     }

     function testFails(){
          //$assert.fail( "This Test should fail" );
     }

     function testFailsShortcut() labels="railo"{
          //fail( "This Test should fail" );
     }

     function testAssert() {
          $assert.assert( application.salvador == 1 );
     }

     function testAssertShortcut() {
          assert( application.salvador == 1 );
     }

     function testisTrue() {
          $assert.isTrue( 1 );
     }

     function testisFalse() {
          $assert.isFalse( 0 );
     }

     function testisEqual() {
          $assert.isEqual( 0, 0 );
          $assert.isEqual( "hello", "HEllO" );
          $assert.isEqual( [], [] );
          $assert.isEqual( [1,2,3, {name="hello", test="this"} ], [1,2,3, {test="this", name="hello"} ] );
     }

     function testisNotEqual() {
          $assert.isNotEqual( this, new coldbox.system.MockBox() );
          $assert.isNotEqual( "hello", "test" );
          $assert.isNotEqual( 1, 2 );
          $assert.isNotEqual( [], [1,3] );
     }

     function testisEqualWithCase() {
          $assert.isEqualWithCase( "hello", "hello" );
     }

     function testnullValue() {
          $assert.null( javaCast("null", "") );
     }

     function testNotNullValue() {
          $assert.notNull( 44 );
     }

     function testTypeOf() {
          $assert.typeOf( "array", [ 1,2 ] );
          $assert.typeOf( "boolean", false );
          $assert.typeOf( "component", this );
          $assert.typeOf( "date", now() );
          $assert.typeOf( "time", timeformat( now() ) );
          $assert.typeOf( "float", 1.1 );
          $assert.typeOf( "numeric", 1 );
          $assert.typeOf( "query", querySim( "id, name
               1 | luis") );
          $assert.typeOf( "string", "hello string" );
          $assert.typeOf( "struct", { name="luis", awesome=true } );
          $assert.typeOf( "uuid", createUUID() );
          $assert.typeOf( "url", "http://www.coldbox.org" );
     }

     function testNotTypeOf() {
          $assert.notTypeOf( "array", 1 );
          $assert.notTypeOf( "boolean", "hello" );
          $assert.notTypeOf( "component", {} );
          $assert.notTypeOf( "date", "monday" );
          $assert.notTypeOf( "time", "1");
          $assert.notTypeOf( "float", "Hello" );
          $assert.notTypeOf( "numeric", "eeww2" );
          $assert.notTypeOf( "query", [] );
          $assert.notTypeOf( "string", this );
          $assert.notTypeOf( "struct", [] );
          $assert.notTypeOf( "uuid", "123" );
          $assert.notTypeOf( "url", "coldbox" );
     }

     function testInstanceOf() {
          $assert.instanceOf( new coldbox.system.MockBox(), "coldbox.system.MockBox" );
     }

     function testNotInstanceOf() {
          $assert.notInstanceOf( this, "coldbox.system.MockBox" );
     }

     function testMatch(){
          $assert.match( "This testing is my test", "(TEST)$" );
     }

     function testMatchWithCase(){
          $assert.match( "This testing is my test", "(test)$" );
     }

     function testNotMatch(){
          $assert.notMatch( "This testing is my test", "(hello)$" );
     }

     function testKey(){
          $assert.key( {name="luis", awesome=true}, "awesome" );
     }

     function testNotKey(){
          $assert.notKey( {name="luis", awesome=true}, "test" );
     }

     function testDeepKey(){
          $assert.deepKey( {name="luis", awesome=true, parent = { age=70 } }, "age" );
     }

     function testNotDeepKey(){
          $assert.notDeepKey( {name="luis", awesome=true, parent = { age=70 } }, "luis" );
     }

     function testLengthOf(){
          $assert.lengthOf( "heelo", 5 );
          $assert.lengthOf( [1,2], 2 );
          $assert.lengthOf( {name="luis"}, 1 );
          $assert.lengthOf( querySim( "id, name
               1 | luis"), 1 );

     }

     function testNotLengthOf(){
          $assert.notLengthOf( "heelo", 3 );
          $assert.notLengthOf( [1,2], 5 );
          $assert.notLengthOf( {name="luis"}, 5 );
          $assert.notLengthOf( querySim( "id, name
               1 | luis"), 0 );

     }

     // railo 4.1+ or CF10+
      function testThrows(){
          $assert.throws(function(){
               var hello = invalidFunction();
          });
     }

      // railo 4.1+ or CF10+
     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = 1;
          });
     }

/*********************************** NON-RUNNABLE Methods ***********************************/

     function nonStandardNamesWillNotRun() {
          fail( "Non-test methods should not run" );
     }

     private function privateMethodsDontRun() {
          fail( "Private method don't run" );
     }

}

Custom Assertions

You can also register custom assertions within the $assert object. You will do this by reading our Custom Assertions section of our TestBox docs.

Test and Suite Labels

Tests and suites can be tagged with TestBox labels. Labels allows you to further categorize different tests or suites so that when a runner executes with labels attached, only those tests and suites will be executed, the rest will be skipped. Labels can be applied globally to the component declaration of the test bundle suite or granularly at the test method declaration.

component displayName="TestBox xUnit suite" labels="railo,stg,dev"{

     function setup(){
          application.wirebox = new coldbox.system.ioc.Injector();
          structClear( request );
     }

     function teardown(){
          structDelete( application, "wirebox" );
          structClear( request );
     }

     function testThrows(){
          $assert.throws(function(){
               var hello = application.wirebox.getInstance( "myINvalidService" ).run();
          });
     }

     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = application.wirebox.getInstance( "MyValidService" ).run();;
          });
     }

     function testFailsShortcut() labels="dev"{
          fail( "This Test should fail when executed with labels" );
     }

}

Skipping Tests and Suites

Tests and suites can be skipped from execution by using the skip annotation in the component or function declaration or our skip() methods. The reporters will show that these suites or tests where skipped from execution.

Skip Annotation

The skip annotation can have the following values:

nothing - If you just add the annotation, we will detect it and skip the test
true - Skips the test
false - Does not skip the test
{udf_name} - It will look for a UDF with that name, execute it and the value must evalute to boolean.

// Skips ALL the tests if the testEnv() returns TRUE
component displayName="TestBox xUnit suite" skip="testEnv"{

     function setup(){
          application.wirebox = new coldbox.system.ioc.Injector();
          structClear( request );
     }

     function teardown(){
          structDelete( application, "wirebox" );
          structClear( request );
     }
     
     function betaTest() skip{
          ...
     }

     function testThrows() skip="true"{
          $assert.throws(function(){
               var hello = application.wirebox.getInstance( "myINvalidService" ).run();
          });
     }

     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = application.wirebox.getInstance( "MyValidService" ).run();;
          });
     }

     private boolean function testEnv(){
          return ( structKeyExists( request, "env") && request.env == "stg" ? true : false );
     }

}

Skip Methods

You can also skip manually by using the skip() method in the Assertion library and also in any bundle which is inherited by the BaseSpec class.

$assert.skip( message="", detail="" )

You can use the $assert.skip( message, detail ) method to skip any spec or suite a-la-carte instead of as an argument to the function definitions. This lets you programmatically skip certain specs and suites and pass a nice message.

function testThrows(){
     $assert.skip()
     $assert.throws(function(){
          var hello = application.wirebox.getInstance( "myINvalidService" ).run()
     })
}

skip( message="", detail="" )

The BaseSpec has this method available to you as well.

it( "can use a mocked stub", function(){
    
    // If conditions met, then skip
    if( !conditionsMet() ){
        skip( "conditions for execution not met" )
    }

    c = createStub().$( "getData", 4 )
    r = calc.add( 4, c.getData() )
    expect( r ).toBe( 8 )
    expect( c.$once( "getData" ) ).toBeTrue()
    
} );

Spies and Mocking

Please refer to our MockBox section to take advantage of all the mocking and stubbing you can do. However, every BDD TestBundle has the following functions available to you for mocking and stubbing purposes:

makePublic( target, method, newName ) - Exposes private methods from objects as public methods
querySim( queryData ) - Simulate a query
getMockBox( [generationPath] ) - Get a reference to MockBox
createEmptyMock( [className], [object], [callLogging=true]) - Create an empty mock from a class or object
createMock( [className], [object], [clearMethods=false], [callLogging=true]) - Create a spy from an instance or class with call logging
prepareMock( object, [callLogging=true]) - Prepare an instance of an object for method spies with call logging
createStub( [callLogging=true], [extends], [implements]) - Create stub objects with call logging and optional inheritance trees and implementation methods
getProperty( target, name, [scope=variables], [defaultValue] ) - Get a property from an object in any scope

Asynchronous-Testing

You can tag a bundle component declaration with the boolean asyncAll annotation and TestBox will execute all specs in separate threads for you concurrently.

Caution Once you delve into the asynchronous world you will have to make sure your tests are also thread safe (var-scoped) and provide any necessary locking.

Running Tests

Running tests is essential of course. There are many ways to run your tests, we will see the basics here, and you can check out our Running Tests section in our in-depth guide.

TestBox CLI

The easiest way to run your tests is to use the TestBox CLI via the testbox run command. Ensure you are in the web root of your project or have configured the box.json to include the TestBox runner in it as shown below. If not CommandBox will try to run by convention your site + test/runner.cfm for you.

You can also pass the runner URL via the testbox run command. Try out the testbox run help command.

Here is a simple box.json config that has a runner and some watcher config.

"testbox":{
    "runner":"http://localhost:49616/tests/runner.cfm",
    "watchers":[
        "system/**.cfc",
        "tests/**.cfc"
    ],
    "watchDelay":"250"
}

Check out the watcher command: testbox watch

URL Runner

Every test harness also has an HTML runner you can execute. By convention the URL is

http://localhost{port}/tests/runner.cfm

This will execute ALL tests in the tests/specs directory for you.

URL Spec Runner

You can also target a specific spec to execute via the URL

http://localhost{port}/tests/specs/MySpec.cfc

Global Runner

TestBox ships with a global runner that can run pretty much anything. You can customize it or place it wherever you need it:

Test Browser

TestBox ships with a test browser that is highly configurable to whatever URL-accessible path you want. It will then show you a test browser where you can navigate and execute not only individual tests but also directory suites.

Reporters

TestBox comes also with a nice plethora of reporters:

ANTJunit : A specific variant of JUnit XML that works with the ANT junitreport task
Codexwiki : Produces MediaWiki syntax for usage in Codex Wiki
Console : Sends report to console
Doc : Builds semantic HTML to produce nice documentation
Dot : Builds an awesome dot report
JSON : Builds a report into JSON
JUnit : Builds a JUnit compliant report
Raw : Returns the raw structure representation of the testing results
Simple : A basic HTML reporter
Text : Back to the 80's with an awesome text report
XML : Builds yet another XML testing report
Tap : A test anything protocol reporter
Min : A minimalistic view of your test reports
MinText : A minimalistic view of your test reports for consoles
NodeJS : User-contributed: https://www.npmjs.com/package/testbox-runner

Assertions

Evaluators

      function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

Included Evaluators

component displayName="TestBox xUnit suite for CF9" labels="railo,cf"{

/*********************************** LIFE CYCLE Methods ***********************************/

     function beforeTests(){
          application.salvador = 1;
     }

     function afterTests(){
          structClear( application );
     }

     function setup(){
          request.foo = 1;
     }

     function teardown(){
          structClear( request );
     }

/*********************************** Test Methods ***********************************/

     function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testIncludesWithCase(){
          $assert.includesWithCase( "hello", "he" );
          $assert.includesWithCase( [ "Monday", "Tuesday" ] , "Monday" );
     }

     function testnotIncludesWithCase(){
          $assert.notincludesWithCase( "hello", "aa" );
          $assert.notincludesWithCase( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

     function testIsEmpty(){
          $assert.isEmpty( [] );
          $assert.isEmpty( {} );
          $assert.isEmpty( "" );
          $assert.isEmpty( queryNew("") );
     }

     function testIsNotEmpty(){
          $assert.isNotEmpty( [1,2] );
          $assert.isNotEmpty( {name="luis"} );
          $assert.isNotEmpty( "HelloLuis" );
          $assert.isNotEmpty( querySim( "id, name
               1 | luis") );
     }

     function testSkipped() skip{
          $assert.fail( "This Test should fail" );
     }

     boolean function isRailo(){
          return structKeyExists( server, "railo" );
     }

     function testSkippedWithConstraint() skip="isRailo"{
          $assert.fail( "This Test should fail" );
     }

     function testFails(){
          //$assert.fail( "This Test should fail" );
     }

     function testFailsShortcut() labels="railo"{
          //fail( "This Test should fail" );
     }

     function testAssert() {
          $assert.assert( application.salvador == 1 );
     }

     function testAssertShortcut() {
          assert( application.salvador == 1 );
     }

     function testisTrue() {
          $assert.isTrue( 1 );
     }

     function testisFalse() {
          $assert.isFalse( 0 );
     }

     function testisEqual() {
          $assert.isEqual( 0, 0 );
          $assert.isEqual( "hello", "HEllO" );
          $assert.isEqual( [], [] );
          $assert.isEqual( [1,2,3, {name="hello", test="this"} ], [1,2,3, {test="this", name="hello"} ] );
     }

     function testisNotEqual() {
          $assert.isNotEqual( this, new coldbox.system.MockBox() );
          $assert.isNotEqual( "hello", "test" );
          $assert.isNotEqual( 1, 2 );
          $assert.isNotEqual( [], [1,3] );
     }

     function testisEqualWithCase() {
          $assert.isEqualWithCase( "hello", "hello" );
     }

     function testnullValue() {
          $assert.null( javaCast("null", "") );
     }

     function testNotNullValue() {
          $assert.notNull( 44 );
     }

     function testTypeOf() {
          $assert.typeOf( "array", [ 1,2 ] );
          $assert.typeOf( "boolean", false );
          $assert.typeOf( "component", this );
          $assert.typeOf( "date", now() );
          $assert.typeOf( "time", timeformat( now() ) );
          $assert.typeOf( "float", 1.1 );
          $assert.typeOf( "numeric", 1 );
          $assert.typeOf( "query", querySim( "id, name
               1 | luis") );
          $assert.typeOf( "string", "hello string" );
          $assert.typeOf( "struct", { name="luis", awesome=true } );
          $assert.typeOf( "uuid", createUUID() );
          $assert.typeOf( "url", "http://www.coldbox.org" );
     }

     function testNotTypeOf() {
          $assert.notTypeOf( "array", 1 );
          $assert.notTypeOf( "boolean", "hello" );
          $assert.notTypeOf( "component", {} );
          $assert.notTypeOf( "date", "monday" );
          $assert.notTypeOf( "time", "1");
          $assert.notTypeOf( "float", "Hello" );
          $assert.notTypeOf( "numeric", "eeww2" );
          $assert.notTypeOf( "query", [] );
          $assert.notTypeOf( "string", this );
          $assert.notTypeOf( "struct", [] );
          $assert.notTypeOf( "uuid", "123" );
          $assert.notTypeOf( "url", "coldbox" );
     }

     function testInstanceOf() {
          $assert.instanceOf( new coldbox.system.MockBox(), "coldbox.system.MockBox" );
     }

     function testNotInstanceOf() {
          $assert.notInstanceOf( this, "coldbox.system.MockBox" );
     }

     function testMatch(){
          $assert.match( "This testing is my test", "(TEST)$" );
     }

     function testMatchWithCase(){
          $assert.match( "This testing is my test", "(test)$" );
     }

     function testNotMatch(){
          $assert.notMatch( "This testing is my test", "(hello)$" );
     }

     function testKey(){
          $assert.key( {name="luis", awesome=true}, "awesome" );
     }

     function testNotKey(){
          $assert.notKey( {name="luis", awesome=true}, "test" );
     }

     function testDeepKey(){
          $assert.deepKey( {name="luis", awesome=true, parent = { age=70 } }, "age" );
     }

     function testNotDeepKey(){
          $assert.notDeepKey( {name="luis", awesome=true, parent = { age=70 } }, "luis" );
     }

     function testLengthOf(){
          $assert.lengthOf( "heelo", 5 );
          $assert.lengthOf( [1,2], 2 );
          $assert.lengthOf( {name="luis"}, 1 );
          $assert.lengthOf( querySim( "id, name
               1 | luis"), 1 );

     }

     function testNotLengthOf(){
          $assert.notLengthOf( "heelo", 3 );
          $assert.notLengthOf( [1,2], 5 );
          $assert.notLengthOf( {name="luis"}, 5 );
          $assert.notLengthOf( querySim( "id, name
               1 | luis"), 0 );

     }

     // railo 4.1+ or CF10+
      function testThrows(){
          $assert.throws(function(){
               var hello = invalidFunction();
          });
     }

      // railo 4.1+ or CF10+
     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = 1;
          });
     }

/*********************************** NON-RUNNABLE Methods ***********************************/

     function nonStandardNamesWillNotRun() {
          fail( "Non-test methods should not run" );
     }

     private function privateMethodsDontRun() {
          fail( "Private method don't run" );
     }

}

Custom Assertions

You can also register custom assertions within the $assert object. You will do this by reading our Custom Assertions section of our TestBox docs.