TestBox 2.6.0 is a minor release with some great new functionality and tons of fixes. You can find the release notes here and the major updates for this release.

Chained DSL Improvements

The methods aroundEach(), beforeEach() and afterEach() can now be chained as they return the Base Spec as well. So get funky with the DSL.

Null Support

You can now pass null into an expectation and TestBox will gracefully represent it instead of blowing up!

Method Mocking Performance

Eager Failures

We have added a new argument to all TestBox run(), runRemote(), runRaw() commands: eagerFailure, which defaults to false. If this is turned on, then TestBox will gracefully short-circuit out of testing further Test Bundles if it finds any failures or errors on previous ones. This is useful for large suites that you want to stop testing if a failure is discovered.

Help for the Color Blind

Before

After

Expanding of x exclusion Methods

You can now prefix the following methods with the letter x to exclude them from execution:

• story()

• given()

• when()

• then()

• feature()

Release Notes

Bugs

New Features

Improvements

TestBox & MockBox Manual v2.x

Features At A Glance

Here are a simple listing of features TestBox brings to the table:

• BDD style testing

• xUnit style testing

• Testing life-cycle methods

• Ability to extend and create custom test runners

• Ability to extend and create custom test reporters

• Extensible reporters, bundled with tons of them:

  • JSON

  • XML

  • JUnit 4 XML

  • Text (80's style)

  • Console

  • Simple HTML

  • Min - Minimalistic Heaven

  • Raw

  • CommandBox

• Asynchronous testing

• Multi-suite capabilities

• Test skipping

• Suite skipping

• Dynamic skipping support via runtime executions

• Test one or more suites exclusively

• Test one or more tests/specs exclusively

• Test labels and tagging

• Testing debug output stream

• Clickable suite titles to filter test execution

• Much more!

Versioning

<major>.<minor>.<patch>

And constructed with the following guidelines:

• Breaking backward compatibility bumps the major (and resets the minor and patch)

• New additions without breaking backward compatibility bumps the minor (and resets the patch)

• Bug fixes and misc changes bumps the patch

License

• Copyright by Ortus Solutions, Corp

• TestBox is a registered trademark by Ortus Solutions, Corp

Info: The ColdBox Websites, Documentation, logo and content have a separate license and they are a separate entity.

Discussion & Help

Reporting a Bug

Professional Open Source

• Custom Development

• Professional Support & Mentoring

• Training

• Server Tuning

• Security Hardening

• Code Reviews

Resources

HONOR GOES TO GOD ABOVE ALL

Because of His grace, this project exists. If you don't like this, then don't read it, its not for you.

Therefore being justified by faith, we have peace with God through our Lord Jesus Christ: By whom also we have access by faith into this grace wherein we stand, and rejoice in hope of the glory of God. - Romans 5:5

TestBox 2.7.0 is a minor release with some great new functionality and tons of fixes. You can find the release notes here and the major updates for this release.

Release Notes

Bugs

New Features

Improvements

TestBox & MockBox Manual v2.x

Features At A Glance

Here are a simple listing of features TestBox brings to the table:

• BDD style testing

• xUnit style testing

• Testing life-cycle methods

• Ability to extend and create custom test runners

• Ability to extend and create custom test reporters

• Extensible reporters, bundled with tons of them:

  • JSON

  • XML

  • JUnit 4 XML

  • Text (80's style)

  • Console

  • Simple HTML

  • Min - Minimalistic Heaven

  • Raw

  • CommandBox

• Asynchronous testing

• Multi-suite capabilities

• Test skipping

• Suite skipping

• Dynamic skipping support via runtime executions

• Test one or more suites exclusively

• Test one or more tests/specs exclusively

• Test labels and tagging

• Testing debug output stream

• Clickable suite titles to filter test execution

• Much more!

Versioning

<major>.<minor>.<patch>

And constructed with the following guidelines:

• Breaking backward compatibility bumps the major (and resets the minor and patch)

• New additions without breaking backward compatibility bumps the minor (and resets the patch)

• Bug fixes and misc changes bumps the patch

License

• Copyright by Ortus Solutions, Corp

• TestBox is a registered trademark by Ortus Solutions, Corp

Info: The ColdBox Websites, Documentation, logo and content have a separate license and they are a separate entity.

Discussion & Help

Reporting a Bug

Professional Open Source

• Custom Development

• Professional Support & Mentoring

• Training

• Server Tuning

• Security Hardening

• Code Reviews

Resources

HONOR GOES TO GOD ABOVE ALL

Because of His grace, this project exists. If you don't like this, then don't read it, its not for you.

Therefore being justified by faith, we have peace with God through our Lord Jesus Christ: By whom also we have access by faith into this grace wherein we stand, and rejoice in hope of the glory of God. - Romans 5:5

TestBox 2.8.0 is a minor release with some great new functionality and tons of fixes. You can find the release notes here and the major updates for this release.

Release Notes

Bugs

New Features

Improvements

TestBox 2.5.0 is a minor release with some great new functionality and tons of fixes. You can find the release notes here and the major updates for this release. One of the biggest features for TestBox that was not part of TestBox, was the addition of TestBox Watchers to CommandBox.

TestBox Watchers

CommandBox CLI has added watching capabilities to the testbox namespace. Which means you can now run the following command in the root of your project: testbox watch and it will monitor all your CFCs for changes. If a change is detected, then it will fire the new testbox run command with our CLI reporter.

Life-Cycle Data Binding

We had introduced spec data binding in TestBox for creating dynamic specs and passing dynamic data into them. We have extended this feature into life-cycle methods:

• beforeEach()

• afterEach()

• aroundEach()

You can now pass in an argument called data which is a struct of dynamic data to pass into the life-cycle method. You can then pickup this data in the closure for the life-cycle. Here is a typical example:

describe( "Ability to bind data to life-cycle methods", function(){

    var data = [
        "spec1",
        "spec2"
    ];

    for( var thisData in data ){
        describe( "Trying #thisData#", function(){

            beforeEach( data={ myData = thisData }, body=function( currentSpec, data ){
                targetData = arguments.data.myData;
            });

            it( title="should account for life-cycle data binding", 
                data={ myData = thisData},
                body=function( data ){
                    expect(    targetData ).toBe( data.mydata );
                }
            );

            afterEach( data={ myData = thisData }, body=function( currentSpec, data ){
                targetData = arguments.data.myData;
            });
        });
    }

    for( var thisData in data ){

        describe( "Trying around life-cycles with #thisData#", function(){

            aroundEach( data={ myData = thisData }, body = function( spec, suite, data ){
                targetData = arguments.data.myData;
                arguments.spec.body( data=arguments.spec.data );
            });

            it( title="should account for life-cycle data binding", 
                data={ myData = thisData },
                body=function( data ){
                    expect(    targetData ).toBe( data.mydata );
                }
            );

        });

    }
});

Release Notes

Bugs

Improvements

• The majority of code examples in this book are done in cfscript.

• All ColdFusion examples designed to run on the open soure Railo Platform or Adobe ColdFusion 9.0.2+

External Trademarks & Copyrights

Flash, Flex, ColdFusion, and Adobe are registered trademarks and copyrights of Adobe Systems, Inc.

Notice of Liability

The information in this book is distributed “as is”, without warranty. The author and Ortus Solutions, Corp shall not have any liability to any person or entity with respect to loss or damage caused or alleged to be caused directly or indirectly by the content of this training book, software and resources described in it.

Contributing

Charitable Proceeds

Shalom Children's Home

Shalom now cares for over 80 children in El Salvador, from newborns to 18 years old. They receive shelter, clothing, food, medical care, education and life skills training in a Christian environment. The home is supported by a child sponsorship program.

We have personally supported Shalom for over 6 years now; it is a place of blessing for many children in El Salvador that either have no families or have been abandoned. This is good earth to seed and plant.

TestBox 2.2.0 is a minor release with some great new functionality and tons of fixes.

Release Notes

Bugs

New Features

Improvements

Luis Fernando Majano Lainez

Luis has a passion for Jesus, tennis, golf, volleyball and anything electronic. Random Author Facts:

• He played volleyball in the Salvadorean National Team at the tender age of 17

• The Lord of the Rings and The Hobbit is something he reads every 5 years. (Geek!)

• His first ever computer was a Texas Instrument TI-86 that his parents gave him in 1986. After some time digesting his very first BASIC book, he had written his own tic-tac-toe game at the age of 9. (Extra geek!)

• He has a geek love for circuits, microcontrollers and overall embedded systems.

• He has of late (during old age) become a fan of running and bike riding with his family.

Keep Jesus number one in your life and in your heart. I did and it changed my life from desolation, defeat and failure to an abundant life full of love, thankfulness, joy and overwhelming peace. As this world breathes failure and fear upon any life, Jesus brings power, love and a sound mind to everybody!

“Trust in the LORD with all your heart, and do not lean on your own understanding.” Proverbs 3:5

Contributors

Jorge Emilio Reyes Bendeck

Jorge started working as project manager and business developer at Ortus Solutions, Corp. in 2013, . At Ortus he fell in love with software development and now enjoys taking part on software development projects and software documentation! He is a fellow Cristian who loves to play the guitar, worship and rejoice in the Lord!

Therefore, if anyone is in Christ, the new creation has come: The old has gone, the new is here! 2 Corinthians 5:17

TestBox 2.1.0 is a minor release with some great new functionality and tons of fixes.

Release Notes

Bugs Fixed

Improvement

New Feature

Global Enhancements

• Better whitespace management on text enabled reporting

• ColdFusion 11 compatibilities

• ColdFusion 9 support on xUnit and Assertions

• HTML Runners now have a high request timeout for long-lasting tests

Debugging Enhancements

TestBox's debug() method has been enhanced to provide greater messages and telemetry. You can even control the depth of the dumps of each of the debugged data as it is sent. Here is the new signature:

The output will now include information as to what spec produced the output, timestamps, data, and thread data.

Expectation Chaining

You can now chain your matchers on a specific expectation to produce an even nicer DSL when trying to assert data on a single expectation:

AroundEach() Life-Cycle Method

We have added a new life-cycle method to the suites called aroundEach() which will completely wrap your spec in another closure. This is an elegant way for you to provide a complete around AOP advice to a specificiation. You can use it to surround the execution in transaction blocks, ORM rollbacks, logging, and so much more. This life-cycle method will decorate ALL specificiations within a single suite. The method signature is below:

The method receives a structure of data representing the spec. This contains the following elements:

• body : The actual closure for the spec that you will use to execute within it.

• labels : The labels used in the spec

• name : The name of the spec

• order : The order of execution of the spec

• skip : The skip flag or closure that determines if the spec runs

The method also receives a structure of metadata about the suite this spec is contained in. It has information about life-cycle closures, async information, names, parents, etc. Here is a very practical example of creating and around each closure to provide rollbacks for specs:

This simple around each life-cycle closure will rollback ALL my spec's executions even if they throw exceptions.

Simple Reporter Bundle Filter

The simple reporter has been enhanced to add a bundle test filter that can help reduce noise when looking for a specific bundle result:

run() Enhancements

The traditional BDD run() has been enhanced so it now receives the TestBox TestResults object as a reference and TestBox as a reference. This way you can have more metadata and access to what will be reported to users in a reporter. You can also use it to decorate it or store much more information that can be picked up later by reports. You also have access to the TestBox class so you can see how the test is supposed to execute, what labels was it passed, directories, options, etc.

Our RefCards will get you up and running in now time.

BDD RefCard

xUnit RefCard

TestBox 2.4.0 is a minor release with some great new functionality and tons of fixes. This release has been a great community effort as many people in the community contributed to its release. Special thanks to Eric Peterson, Joe Gooch and Sean Corfield for their additions, testing and contributions.

New toSatisfy matcher

This new matcher is thanks to Sean Corfield. It allows you to create your own closure that will evaluate the expectation and then decide if it passes the given truth test.

New expectAll() collection expectation

Sean was busy in this release and provided us with this awesome feature in which you can call on a new expectAll() and pass either an array or struct. TestBox will then iterate for you and call all the chained matchers upon the collection items.

New mintext Reporter

This new reporter is to enhance console based runners in order for the report to be more legible.

New JSON Matchers & Assertions

You can now use a toBeJSON() matcher or a $assert.isJSON assertion.

No more runRemote

You no longer need to pass ?method=runRemote in the URL when executing a test bundle via the URL. This will automatically be added for you.

New Fluent API for Testing Declarations

Thanks to Joe Gooch you can now use the new methods in the TestBox cfc

• addDirectory()

• addDirectories()

• addBundles()

You can chain them as you see fit and they will aggregate the specs collected.

Release Notes

Here is the full release notes for this release

Bugs

New Features

Improvements

TestBox is a xUnit style and behavior-driven development (BDD) testing framework for CFML (ColdFusion). It does not depend on any other framework for operation and it has a clean and descriptive syntax that will make you giggle when writing your tests. It ships with not only the xUnit and BDD testing capabilities, but also an assertions and expectations library, several different runners, reporters and MockBox, for mocking and stubbing capabilities.

TestBox is a next generation testing framework for ColdFusion (CFML) that is based on BDD (Behavior Driven Development) for providing a clean obvious syntax for writing tests. It contains not only a testing framework, runner, assertions, mocking/stubbing and expectations library. It also supports xUnit style of testing with MXUnit compatibilities.

Important TestBox is a standalone testing package and is meant to be used with any ColdFusion (CFML) application, framework or library. IT IS NOT ONLY FOR COLDBOX APPLICATIONS.

Useful Resources

TestBox has the following supported IDE Tools:

TestBox 2.3.0 is a minor release with some great new functionality and tons of fixes. This release has been a great community effort as many people in the community contributed to its release.

Release Notes

Bug

New Feature

Improvement

Note Please note the --saveDev flag which tells CommandBox that TestBox is a development dependency and not a production dependency.

This will install TestBox in a /testbox folder from where you called the command. You can also extract the download zip and place it anywhere you like and create a mapping called /testbox that points to testbox in the distribution folder, this is the most secure approach. However, you can just place it in the webroot if you like.

The download structure includes:

• apidocs : The API docs for TestBox

• system : The main system framework folder

• test-browser : This is a little utility to facilitate navigating big testing suites. This helps navigate to the suites you want and execute them instead of typing all the time.

• test-harness : A simplified version of the TestBox runner that can be placed anywhere in your application. It includes an ANT build that allows you to execute your tests and produce results via ANT and also JUnit compliant reports via the junitreport task.

• test-runner : The pre-built TestBox Global Runner. You can pass in a directory mapping or bundles and select labels and boom run the tests.

• tests : Several sample tests and runners which actually are used to build TestBox

The only required folder is system and it does not need to be web-accessible. The other folders are all optional tools to help you that expect to be web-accessible. You can safely remove everything except the system folder if you want to build your own test browsers. If you are placing TestBox outside of your web root, the /testbox mapping needs to point to the parent folder containing the system directory as all TestBox models start with testbox.system in their package path.

If you are going to be using the xUnit style of testing which includes MXUnit compatibility, then you don't need the latest CFML engines. BDD uses closures to declare your specs, so it it is only available on newer versions of your CFML engine.

• ColdFusion 10+

• Lucee 4.5+

A test suite begins with a call to our TestBox describe() function with at least two arguments: a title and a body function/closure. The title is the name of the suite to register and the body function/closure is the block of code that implements the suite. When applying BDD to your tests, this function is used to describe your story scenarios that you will implement. There are more arguments which you can see below:

function run( testResults, testBox ){

     describe("A suite", function(){
          it("contains spec with an awesome expectation", function(){
               expect( true ).toBeTrue();
          });
     });

}

TestBox relies on the fact of creating testing bundles which are basically CFCs. A bundle CFC will hold all the suites and specs a TestBox runner will execute and produce reports on.

component extends="testbox.system.BaseSpec"{

     // executes before all suites
     function beforeAll(){}

     // executes after all suites
     function afterAll(){}

     // All suites go in here
     function run( testResults, testBox ){

     }

}

This bundle CFC can contain 2 life-cycle functions and a single run() function where you will be writing your test suites and specs. The beforeAll() and afterAll() methods are called life-cycle methods. They will execute once before the run() function and once after the run() function. This is a great way to do any kind of global setup or teardown in your tests.

The run() function receives the TestBox testResults object as a reference and testbox as a reference as well. This way you can have metadata and access to what will be reported to users in a reporter. You can also use it to decorate the results or store much more information that can be picked up later by reports. You also have access to the testbox class so you can see how the test is supposed to execute, what labels was it passed, directories, options, etc.

We have prepared an awesome PDF ref card for working with TestBox BDD:

TestBox does not depend on any other framework for operation and it has a clean and descriptive syntax that will make you giggle when writing your tests. It ships with not only the xUnit and BDD testing capabilities, but also an assertions and expectations library, several different runners, reporters and MockBox, for mocking and stubbing capabilities.

Before and After Specs

If you are familiar with xUnit style frameworks, the majority of them provide a way to execute functions before and after every single test case or spec in BDD. This is a great way to keep your tests DRY (Do not repeat yourself)! TestBox provides the beforeEach() and afterEach() methods that each take in a closure as their argument that receive the name of the spec that's about to be executed or just executed. As their names indicate, they execute before a spec and after a spec in a related describe block.

describe("A spec (with setup and tear-down)", function() {

     beforeEach(function( currentSpec ) {
          coldbox = 22;
          application.wirebox = new coldbox.system.ioc.Injector();
     });

     afterEach(function( currentSpec ) {
          coldbox = 0;
          structDelete( application, "wirebox" );
     });

     it("is just a function, so it can contain any code", function() {
          expect( coldbox ).toBe( 22 );
     });

     it("can have more than one expectation and talk to scopes", function() {
          expect( coldbox ).toBe( 22 );
          expect( application.wirebox.getInstance( 'MyService' ) ).toBeComponent();
     });
});

Around Specs

The aroundEach() life-cycle method will completely wrap your spec in another closure. This is an elegant way for you to provide a complete around AOP advice to a specification. You can use it to surround the execution in transaction blocks, ORM rollbacks, logging, and so much more. This life-cycle method will decorate ALL specs within a single suite and any children suites. The method signature is below:

aroundEach( function( spec, suite ){
     // execute the spec
     arguments.spec.body();
} );

The method receives a structure of data representing the spec. This contains the following elements:

• body : The actual closure for the spec that you will use to execute within it.

• labels : The labels used in the spec

• name : The name of the spec

• order : The order of execution of the spec

• skip : The skip flag or closure that determines if the spec runs

The method also receives a structure of metadata about the suite this spec is contained in. It has information about life-cycle closures, async information, names, parents, etc. Here is a very practical example of creating and around each closure to provide rollbacks for specs:

aroundEach( function( spec, suite ){

     transaction action="begin"{
          try{/
               // execute the spec
               arguments.spec.body();
          } catch( Any e ){
               rethrow;
          } finally{
               transaction action="rollback";
          }
     }
} );

This simple around each life-cycle closure will rollback ALL my spec's executions even if they throw exceptions.

Annotations

Lifecycle Nesting Order

When you use beforeEach(), afterEach(), and aroundEach() at the same time, there is a specific order they fire in. For a given describe block, they will fire in this order. Remember, aroundEach() is split into two parts-- the half of the method before you call spec.body() and the second half of the method.

1. beforeEach

2. aroundEach (first half)

3. it() (the spec.body() call)

4. aroundEach (second half)

5. afterEach()

Here's an example:

describe( 'my describe', function(){
	
    beforeEach( function( currentSpec ){
        // I run first
    } );
    	 
    aroundEach( function( spec, suite ){
        // I run second
        arguments.spec.body();
        // I run fourth
    });
    
    afterEach( function( currentSpec ){
        // I run fifth
    } );
    
    it( 'my it', function(){
        // I run third
    } );
    
} );

If there are more than one it() blocks, the process repeats for each one. Steps 1, 2, 4, 5 will wrap every single it().

When you nest more than one describe block inside the other, the before/around/after order is the same but drills down to the innermost describe and then bubbles back up. That means the outermost beforeEach() starts and we end on the outermost afterEach().

Here's what an example flow would look like that had before/after/around specified in two levels of describes with a single it() in the inner most describe.

1. Outermost beforeEach() call

2. Innermost beforeEach() call

3. Outermost aroundEach() call (first half)

4. Innermost aroundEach() call (first half)

5. The it() block

6. Innermost aroundEach() calls (second half)

7. Outermost aroundEach() call (second half)

8. Innermost afterEach() call

9. Outermost afterEach() call

This works regardless of the number of levels and can obviously have many permutations, but the basic order is still the same. Before/around/after and starting at the outside working in, and back out again. This process happens for every single spec or it() block. This is as opposed to the beforeAll() and afterAll() method which only run once for the entire CFC regardless of how many specs there are.

Feature: Box Size
    In order to know what size box I need
    As a distribution manager
    I want to know the volume of the box

    Scenario: Get box volume
        Given I have entered a width of 20
        And a height of 30
        And a depth of 40
        When I run the calculation
        Then the result should be 24000

TestBox provides you with feature(), scenario() and story() wrappers for describe() blocks. As such we can write our requirements in test form like so:

feature( "Box Size", function(){

    describe( "In order to know what size box I need
              As a distribution manager
              I want to know the volume of the box", function(){

        scenario( "Get box volume", function(){
            given( "I have entered a width of 20
                And a height of 30
                And a depth of 40", function(){
                when( "I run the calculation", function(){
                      then( "the result should be 24000", function(){
                          // call the method with the arguments and test the outcome
                          expect( myObject.myFunction(20,30,40) ).toBe( 24000 );
                      });
                 });
            });
        });
    });
});

The output from running the test will read as the original requirements, providing you with not only automated tests but also a living document of the requirements in a business-readable format.

story("As a distribution manager, I want to know the volume of the box I need", function() {
    given("I have a width of 20
        And a height of 30
        And a depth of 40", function() {
        when("I run the calculation", function() {
              then("the result should be 24000", function() {
                  // call the method with the arguments and test the outcome
                  expect(myObject.myFunction(20,30,40)).toBe(24000);
              });
         });
    });
});

As feature(), scenario() and story() are wrappers for describe() you can intermix them so that your can create tests which read as the business requirements. As with describe(), they can be nested to build up blocks.

describe("A spec", function() {
     it("is just a closure, so it can contain any code", function() {
          coldbox = 0;
          coldbox++;

          expect( coldbox ).toBe( 1 );
     });

     it("can have more than one expectation", function() {
          coldbox = 0;
          coldbox++;

          expect( coldbox ).toBe( 1 );
          expect( coldbox ).toBeTrue();
     });
});

Nesting describe Blocks

Calls to our describe() function can be nested with specs at any level or point of execution. This allows you to create your tests as a related tree of nested functions. Please note that before a spec is executed, TestBox walks down the tree executing each beforeEach() and afterEach() function in the declared order. This is a great way to logically group specs in any level as you see fit.

describe("A spec", function() {

     beforeEach(function( currentSpec ) {
          coldbox = 22;
          application.wirebox = new coldbox.system.ioc.Injector();
     });

     afterEach(function( currentSpec ) {
          coldbox = 0;
          structDelete( application, "wirebox" );
     });

     it("is just a function, so it can contain any code", function() {
          expect( coldbox ).toBe( 22 );
     });

     it("can have more than one expectation and talk to scopes", function() {
          expect( coldbox ).toBe( 22 );
          expect( application.wirebox.getInstance( 'MyService' ) ).toBeComponent();
     });

     describe("nested inside a second describe", function() {

          beforeEach(function( currentSpec ) {
               awesome = 22;
          });

          afterEach(function( currentSpec ) {
               awesome = 22 + 8;
          });

          it("can reference both scopes as needed ", function() {
            expect( coldbox ).toBe( awesome );
          });
     });

     it("can be declared after nested suites and have access to nested variables", function() {
          expect( awesome ).toBe( 30 );
     });

});

An expectation is a nice assertion DSL that TestBox exposes so you can pretty much read what should happen in the testing scenario. A spec will pass if all expectations pass. A spec with one or more expectations that fail will fail the entire spec.

function run(){

     describe("A suite", function(){
          it("contains spec with an awesome expectation", function(){
               expect( true ).toBeTrue();
          });

          it("contains a spec with more than 1 expectation", function(){
               expect( [1,2,3] ).toBeArray();
               expect( [1,2,3] ).toHaveLength( 3 );
          });
     });

}

They are closures Ma!

function run(){

     describe("A suite is a closure", function(){
          c = new Calculator();

          it("and so is a spec", function(){
               expect( c ).toBeTypeOf( 'component' );
          });
     });

}

Spec Data Binding

The data argument can be used to pass in a structure of data into the spec so it can be used later within the body closure. This is great when doing looping and creating dynamic closure calls:

// Simple Example
it( title="can handle binding", body=function( data ){
    expect(    data.keep ).toBeTrue();
}, data={ keep = true } );

// Complex Example. Let's iterate over a bunch of files and create dynamic specs
for( var filePath in files ){

  it( 
    title="#getFileFromPath( filePath )# should be valid JSON", 
    // pass in a struct of data to the spec for later evaluation
    data={ filePath = filePath },
    // the spec closure accepts the data for later evaluation
    body=function( data ) {
      var json = fileRead( data.filePath );
      var isItJson = isJSON( json );

      expect( json ).notToBeEmpty();
      expect( isItJson ).toBeTrue();

      if( isItJson ){
          var jsonData = deserializeJSON(json);
          if( getFileFromPath( filePath ) != "index.json"){
              expect( jsonData ).toHaveKey( "name" );
              expect( jsonData ).toHaveKey( "type" );
          }
      }

  });
}

With TestBox's BDD syntax, it is possible to create suites dynamically; however, there are a few things to be aware of.

Setup for dynamic suites must be done in the pseudo-constructor (versus in beforeAll()). This is because variables-scoped variables set in beforeAll() are not available in the describe closure (even though they are available in it closures). This behavior can be explained by the execution sequence of a BDD bundle: When the bundle's run() method is called, it collects preliminary test data via describes. After preliminary test data are collected, the beforeAll() runs, followed by the describe closures.

Additionally, care must be taken to pass data into the it closures, otherwise strange behavior will result (the values from the last loop iteration will be repeated in the body of each looped it).

Example

The following bundle creates suites dynamically, by looping over test metadata.

Expectations are self-concatenated strings that evaluate an actual value to an expected value or condition. These are initiated by the global TestBox method called expect() which takes in a value called the actual value or expectAll() which takes in an array or struct which will be the actual value. It is concatenated in our expectations DSL with a matcher function that will most likely take in an expected value or condition to test. You can also concatenate the matchers and do multiple evaluations on a single actual value.

Matchers

Each matcher implements a comparison or evaluation of the actual value and an expected value or condition. It is responsible for either passing or failing this evaluation and reporting it to TestBox. Each matcher also has a negative counterpart assertion by just prefixing the call to the matcher with a not expression.

Included Matchers

Custom Matchers

Specs and suites can be skipped from execution by prefixing certain functions with the letter x or by using the skip argument in each of them. The reporters will show that these suites or specs where skipped from execution. The functions you can prefix are:

• it()

• describe()

• story()

• given()

• when()

• then()

• feature()

Skip Argument

The skip argument can be a boolean value or a closure. If the value is true then the suite or spec is skipped. If the return value of the closure is true then the suite or spec is skipped. Using the closure approach allows you to dynamically at runtime figure out if the desired spec or suite is skipped. This is such a great way to prepare tests for different CFML engines.

Specs and suites can be tagged with TestBox labels. Labels allows you to further categorize different specs or suites so that when a runner executes with labels attached, only those specs and suites will be executed, the rest will be skipped.

As you can see from our arguments for a test suite, you can pass an asyncAll argument to the describe() blocks that will allow TestBox to execute all specs in separate threads for you concurrently.

Caution Once you delve into the asynchronous world you will have to make sure your tests are also thread safe (var-scoped) and provide any necessary locking.

TestBox is a xUnit style and behavior-driven development (BDD) testing framework for CFML (ColdFusion). It does not depend on any other framework for operation and it has a clean and descriptive syntax that will make you giggle when writing your tests. It ships with not only the xUnit and BDD testing capabilities, but also an assertions and expectations library, several different runners, reporters and MockBox, for mocking and stubbing capabilities.

Info Note: This article is a primer to get you started with the xUnit capabilities of TestBox, if you would like to explore the BDD style capabilities of TestBox we recommend you visit the BDD primer chapter.

TestBox ships with several test runners internally but we have tried to simplify and abstract it with our testbox object which can be found in the testbox.system package. The TestBox object allows you to execute tests from a variety of entry points and languages such as CFC, CFM, HTTP, NodeJS, SOAP or REST. You can also make your CFC's extend from our BaseSpec class so you can execute it directly via the URL. The main execution methods are:

// Create TestBox object
testbox = new testbox.system.TestBox();

// You can add fluent specs via addDirectory(), addDirectories(), addBundles()
testbox.addDirectory( "/tests/specs" );

// Run tests and produce reporter results
testbox.run()

// Run tests and get raw testbox.system.TestResults object
testbox.runRaw()

// Run tests and produce reporter results from SOAP, REST, HTTP
testbox.runRemote()

// Run via Spec URL
http://localhost/test/spec.cfc?method=runRemote

Info We encourage you to read the API docs included in the distribution for the complete parameters for each method.

run() Arguments

Here are the arguments you can use for initializing TestBox or executing the run() method

runRemote() arguments

Here are the arguments you can use for executing the runRemote() method:

The bundles argument which can be a single CFC path or an array of CFC paths or a directory argument so it can go and discover the test bundles from that directory. The reporter argument can be a core reporter name like: json,xml,junit,raw,simple,dots,tap,min,etc or it can be an instance of a reporter CFC. You can execute the runners from any cfm template or any CFC or any URL, that is up to you.

Global Runner

TestBox ships with a global runner that can be used to run pretty much anything. You can customize it or place it wherever you need it:

Test Browser

TestBox ships with a test browser that is highly configurable to whatever URL accessible path you want. It will then show you a test browser where you can navigate and execute not only individual tests, but also directory suites as well.

ANT Runner

In our test samples and templates we include an ANT runner that will be able to execute your tests via ANT. It can also leverage our ANTJunit reporter to use the junitreport task to produce JUnit compliant reports as well. You can find this runner in the test samples and runner template directory.

Bundle(s) Runner

<cfset r = new coldbox.system.TestBox( "coldbox.testing.cases.testing.specs.BDDTest" ) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox( bundles="coldbox.testing.cases.testing.specs.BDDTest", testSpecs="OnlyThis,AndThis,AndThis" ) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox( bundles="coldbox.testing.cases.testing.specs.BDDTest", testSuites="Custom Matchers,A Spec" ) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox( [ "coldbox.testing.cases.testing.specs.BDDTest", "coldbox.testing.cases.testing.specs.BDD2Test" ] ) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox( bundles="coldbox.testing.cases.testing.specs.BDDTest", labels="railo" ) >
<cfoutput>#r.run(reporter="json")#</cfoutput>

Test Runner

Directory Runner

<cfset r = new coldbox.system.TestBox( directory="coldbox.testing.cases.testing.specs" ) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox( directory={ mapping="coldbox.testing.cases.testing.specs", recurse=false } ) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox(
      directory={ mapping="coldbox.testing.cases.testing.specs",
      recurse=true,
      filter=function(path){
            return ( findNoCase( "test", arguments.path ) ? true : false );
      }}) >
<cfoutput>#r.run()#</cfoutput>

<cfset r = new coldbox.system.TestBox(
      directory={ mapping="coldbox.testing.cases.testing.specs",
      recurse=true,
      filter=function(path){
            return ( findNoCase( "test", arguments.path ) ? true : false );
      }}) >
<cfset fileWrite( 'testreports.json', r.run() )>

SOAP Runner

You can run tests via SOAP by leveraging the runRemote() method. The WSDL URL will be

http://localhost/testbox/system/TestBox.cfc?wsdl

HTTP/REST Runner You can run tests via HTTP/REST by leveraging the runRemote() endpoint. The URL will be

http://localhost/testbox/system/TestBox.cfc

NodeJS Runner

There is a user-contributed NodeJS Runner that looks fantastic and can be downloaded here:

Just use node to install:

npm install -g testbox-runner

We have prepared an awesome PDF ref card for working with TestBox xUnit:

TestBox comes also with a nice plethora of reporters:

• ANTJunit : A specific variant of JUnit XML that works with the ANT junitreport task

• Codexwiki : Produces MediaWiki syntax for usage in Codex Wiki

• Console : Sends report to console

• Doc : Builds semantic HTML to produce nice documentation

• Dot : Builds an awesome dot report

• JSON : Builds a report into JSON

• JUnit : Builds a JUnit compliant report

• Raw : Returns the raw structure representation of the testing results

• Simple : A basic HTML reporter

• Text : Back to the 80's with an awesome text report

• XML : Builds yet another XML testing report

• Tap : A test anything protocol reporter

• Min : A minimalistic view of your test reports

• MinText : A minimalistic view of your test reports in consoles

The xUnit functionalities of TestBox will require Railo 4.1+, Lucee 4.5+ or ColdFusion 9.01+.

TestBox relies on the fact of creating testing bundles which are basically CFCs. A bundle CFC will hold all the tests the TestBox runner will execute and produce reports on. Thus, sometimes this test bundle is referred to as a test suite in xUnit terms.

component displayName="My test suite" extends="testbox.system.BaseSpec"{

     // executes before all tests
     function beforeTests(){}

     // executes after all tests
     function afterTests(){}

}

This bundle can contain 2 life-cycle methods, the beforeTests() and afterTests() methods will execute once before ALL your tests run and then after ALL your tests run. This is a great way to do any kind of global setup or teardown in your tests. It also contains a displayName argument in the component declaration which gives you a way to name your testing suite. We will see later the other annotations you can add to the component declaration further in the primer.

Assertions are self-concatenated strings that evaluate an actual value to an expected value or condition. These are initiated by the global TestBox variable called $assert which contains tons of included assertion methods so you can evaluate your tests.

Evaluators

Each assertion evaluator will compare the actual value and an expected value or condition. It is responsible for either passing or failing this evaluation and reporting it to TestBox. Each evaluator also has a negative counterpart assertion by just prefixing the call to the method with a not expression.

      function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

Included Evaluators

component displayName="TestBox xUnit suite for CF9" labels="railo,cf"{

/*********************************** LIFE CYCLE Methods ***********************************/

     function beforeTests(){
          application.salvador = 1;
     }

     function afterTests(){
          structClear( application );
     }

     function setup(){
          request.foo = 1;
     }

     function teardown(){
          structClear( request );
     }

/*********************************** Test Methods ***********************************/

     function testIncludes(){
          $assert.includes( "hello", "HE" );
          $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testIncludesWithCase(){
          $assert.includesWithCase( "hello", "he" );
          $assert.includesWithCase( [ "Monday", "Tuesday" ] , "Monday" );
     }

     function testnotIncludesWithCase(){
          $assert.notincludesWithCase( "hello", "aa" );
          $assert.notincludesWithCase( [ "Monday", "Tuesday" ] , "monday" );
     }

     function testNotIncludes(){
          $assert.notIncludes( "hello", "what" );
          $assert.notIncludes( [ "Monday", "Tuesday" ] , "Friday" );
     }

     function testIsEmpty(){
          $assert.isEmpty( [] );
          $assert.isEmpty( {} );
          $assert.isEmpty( "" );
          $assert.isEmpty( queryNew("") );
     }

     function testIsNotEmpty(){
          $assert.isNotEmpty( [1,2] );
          $assert.isNotEmpty( {name="luis"} );
          $assert.isNotEmpty( "HelloLuis" );
          $assert.isNotEmpty( querySim( "id, name
               1 | luis") );
     }

     function testSkipped() skip{
          $assert.fail( "This Test should fail" );
     }

     boolean function isRailo(){
          return structKeyExists( server, "railo" );
     }

     function testSkippedWithConstraint() skip="isRailo"{
          $assert.fail( "This Test should fail" );
     }

     function testFails(){
          //$assert.fail( "This Test should fail" );
     }

     function testFailsShortcut() labels="railo"{
          //fail( "This Test should fail" );
     }

     function testAssert() {
          $assert.assert( application.salvador == 1 );
     }

     function testAssertShortcut() {
          assert( application.salvador == 1 );
     }

     function testisTrue() {
          $assert.isTrue( 1 );
     }

     function testisFalse() {
          $assert.isFalse( 0 );
     }

     function testisEqual() {
          $assert.isEqual( 0, 0 );
          $assert.isEqual( "hello", "HEllO" );
          $assert.isEqual( [], [] );
          $assert.isEqual( [1,2,3, {name="hello", test="this"} ], [1,2,3, {test="this", name="hello"} ] );
     }

     function testisNotEqual() {
          $assert.isNotEqual( this, new coldbox.system.MockBox() );
          $assert.isNotEqual( "hello", "test" );
          $assert.isNotEqual( 1, 2 );
          $assert.isNotEqual( [], [1,3] );
     }

     function testisEqualWithCase() {
          $assert.isEqualWithCase( "hello", "hello" );
     }

     function testnullValue() {
          $assert.null( javaCast("null", "") );
     }

     function testNotNullValue() {
          $assert.notNull( 44 );
     }

     function testTypeOf() {
          $assert.typeOf( "array", [ 1,2 ] );
          $assert.typeOf( "boolean", false );
          $assert.typeOf( "component", this );
          $assert.typeOf( "date", now() );
          $assert.typeOf( "time", timeformat( now() ) );
          $assert.typeOf( "float", 1.1 );
          $assert.typeOf( "numeric", 1 );
          $assert.typeOf( "query", querySim( "id, name
               1 | luis") );
          $assert.typeOf( "string", "hello string" );
          $assert.typeOf( "struct", { name="luis", awesome=true } );
          $assert.typeOf( "uuid", createUUID() );
          $assert.typeOf( "url", "http://www.coldbox.org" );
     }

     function testNotTypeOf() {
          $assert.notTypeOf( "array", 1 );
          $assert.notTypeOf( "boolean", "hello" );
          $assert.notTypeOf( "component", {} );
          $assert.notTypeOf( "date", "monday" );
          $assert.notTypeOf( "time", "1");
          $assert.notTypeOf( "float", "Hello" );
          $assert.notTypeOf( "numeric", "eeww2" );
          $assert.notTypeOf( "query", [] );
          $assert.notTypeOf( "string", this );
          $assert.notTypeOf( "struct", [] );
          $assert.notTypeOf( "uuid", "123" );
          $assert.notTypeOf( "url", "coldbox" );
     }

     function testInstanceOf() {
          $assert.instanceOf( new coldbox.system.MockBox(), "coldbox.system.MockBox" );
     }

     function testNotInstanceOf() {
          $assert.notInstanceOf( this, "coldbox.system.MockBox" );
     }

     function testMatch(){
          $assert.match( "This testing is my test", "(TEST)$" );
     }

     function testMatchWithCase(){
          $assert.match( "This testing is my test", "(test)$" );
     }

     function testNotMatch(){
          $assert.notMatch( "This testing is my test", "(hello)$" );
     }

     function testKey(){
          $assert.key( {name="luis", awesome=true}, "awesome" );
     }

     function testNotKey(){
          $assert.notKey( {name="luis", awesome=true}, "test" );
     }

     function testDeepKey(){
          $assert.deepKey( {name="luis", awesome=true, parent = { age=70 } }, "age" );
     }

     function testNotDeepKey(){
          $assert.notDeepKey( {name="luis", awesome=true, parent = { age=70 } }, "luis" );
     }

     function testLengthOf(){
          $assert.lengthOf( "heelo", 5 );
          $assert.lengthOf( [1,2], 2 );
          $assert.lengthOf( {name="luis"}, 1 );
          $assert.lengthOf( querySim( "id, name
               1 | luis"), 1 );

     }

     function testNotLengthOf(){
          $assert.notLengthOf( "heelo", 3 );
          $assert.notLengthOf( [1,2], 5 );
          $assert.notLengthOf( {name="luis"}, 5 );
          $assert.notLengthOf( querySim( "id, name
               1 | luis"), 0 );

     }

     // railo 4.1+ or CF10+
      function testThrows(){
          $assert.throws(function(){
               var hello = invalidFunction();
          });
     }

      // railo 4.1+ or CF10+
     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = 1;
          });
     }

/*********************************** NON-RUNNABLE Methods ***********************************/

     function nonStandardNamesWillNotRun() {
          fail( "Non-test methods should not run" );
     }

     private function privateMethodsDontRun() {
          fail( "Private method don't run" );
     }

}

Custom Assertions

You can also register custom assertions within the $assert object. You will do this by reading our Custom Assertions section of our TestBox docs.

TestBox discovers test methods in your bundle CFC by applying the following discovery rules:

• Any method that has a test annotation on it

• Any public method that starts or ends with the word test

// Via inline annotation
function shouldBeAwesome() test{}

/**
* Via comment annotation
* @test
*/
function shouldBeAwesome(){}

// via conventions
function testShouldDoThis(){}
function shouldDoThisTest(){}

Each test method will test the state of the SUT (software under test) or sometimes referred to as code under test. It will do so by asserting that actual values from an execution match an expected value or condition. TestBox offers an assertion library that you have available in your bundle via the injected variable $assert. You can also use our expectations library if you so desire, but that is mostly used in our BDD approach.

function testIncludes(){
       $assert.includes( "hello", "HE" );
       $assert.includes( [ "Monday", "Tuesday" ] , "monday" );
}

Each test function can also have some cool annotations attached to it.

• makePublic( target, method, newName ) - Exposes private methods from objects as public methods

• querySim( queryData ) - Simulate a query

• getMockBox( [generationPath] ) - Get a reference to MockBox

• createEmptyMock( [className], [object], [callLogging=true]) - Create an empty mock from a class or object

• createMock( [className], [object], [clearMethods=false], [callLogging=true]) - Create a spy from an instance or class with call logging

• prepareMock( object, [callLogging=true]) - Prepare an instance of an object for method spies with call logging

• createStub( [callLogging=true], [extends], [implements]) - Create stub objects with call logging and optional inheritance trees and implementation methods

• getProperty( target, name, [scope=variables], [defaultValue] ) - Get a property from an object in any scope

You can tag a bundle component declaration with the boolean asyncAll annotation and TestBox will execute all specs in separate threads for you concurrently.

Caution Once you delve into the asynchronous world you will have to make sure your tests are also thread safe (var-scoped) and provide any necessary locking.

Tests and suites can be tagged with TestBox labels. Labels allows you to further categorize different tests or suites so that when a runner executes with labels attached, only those tests and suites will be executed, the rest will be skipped. Labels can be applied globally to the component declaration of the test bundle suite or granularly at the test method declaration.

Tests and suites can be skipped from execution by using the skip annotation in the component or function declaration. The reporters will show that these suites or tests where skipped from execution. The value of the skip annotation can be a simple true or false or it can be the name of a UDF that exists in the same bundle CFC. This UDF must return a boolean value and it is evaluated at runtime.

component displayName="TestBox xUnit suite" skip="testEnv"{

     function setup(){
          application.wirebox = new coldbox.system.ioc.Injector();
          structClear( request );
     }

     function teardown(){
          structDelete( application, "wirebox" );
          structClear( request );
     }

     function testThrows() skip="true"{
          $assert.throws(function(){
               var hello = application.wirebox.getInstance( "myINvalidService" ).run();
          });
     }

     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = application.wirebox.getInstance( "MyValidService" ).run();;
          });
     }

     private boolean function testEnv(){
          return ( structKeyExists( request, "env") && request.env == "stg" ? true : false );
     }

}

• makePublic( target, method, newName ) - Exposes private methods from objects as public methods

• querySim( queryData ) - Simulate a query

• getMockBox( [generationPath] ) - Get a reference to MockBox

• createEmptyMock( [className], [object], [callLogging=true]) - Create an empty mock from a class or object

• createMock( [className], [object], [clearMethods=false], [callLogging=true]) - Create a spy from an instance or class with call logging

• prepareMock( object, [callLogging=true]) - Prepare an instance of an object for method spies with call logging

• createStub( [callLogging=true], [extends], [implements]) - Create stub objects with call logging and optional inheritance trees and implementation methods

• getProperty( target, name, [scope=variables], [defaultValue] ) - Get a property from an object in any scope

TestBox not only provides you with global life-cycle methods but also with localized test methods. This is a great way to keep your tests DRY (Do not repeat yourself)! TestBox provides the setup( currentMethod ) and teardown( currentMethod ) methods that each receives the name of the test method in question. which as their names indicate, they execute before a test and after a test in a test bundle.

component displayName="TestBox xUnit suite" labels="railo,cf"{

     function setup( currentMethod ){
          application.wirebox = new coldbox.system.ioc.Injector();
          structClear( request );
     }

     function teardown( currentMethod ){
          structDelete( application, "wirebox" );
          structClear( request );
     }

     function testThrows(){
          $assert.throws(function(){
               var hello = application.wirebox.getInstance( "myINvalidService" ).run();
          });
     }

     function testNotThrows(){
          $assert.notThrows(function(){
               var hello = application.wirebox.getInstance( "MyValidService" ).run();;
          });
     }

}

BDD

BDD stands for behavior driven development and is highly based on creating specifications and expectations of results in a readable DSL (Domain Specifc Language).

xUnit

xUnit style of testing is the more traditional TDD or test driven development approach where you are creating a test case CFC that matches the software under test and for each method in the SUT you create a test method in the test case CFC. We cover these two topics in our primers chapter (Chapter 3).

No matter what testing style you pick you, TestBox will execute the tests for you and then send your results to a reporter that can produce awesome testing reports for you.

At runtime we provide the inheritance via mixins so you don't have to worry about it. However, if you want to declare the inheritance you can do so and this will give you the following benefits:

• Some IDEs will be able to give you introspection for methods and properties

• You will be able to use the HTML runner by executing directly the runRemote method on the CFC Bundle

• Your tests will run faster

component extends="testbox.system.BaseSpec"{
}

TestBox comes also with a nice plethora of reporters:

• ANTJunit : A specific variant of JUnit XML that works with the ANT junitreport task

• Codexwiki : Produces MediaWiki syntax for usage in Codex Wiki

• Console : Sends report to console

• Doc : Builds semantic HTML to produce nice documentation

• Dot : Builds an awesome dot report

• JSON : Builds a report into JSON

• JUnit : Builds a JUnit compliant report

• Raw : Returns the raw structure representation of the testing results

• Simple : A basic HTML reporter

• Text : Back to the 80's with an awesome text report

• XML : Builds yet another XML testing report

• Tap : A test anything protocol reporter

• Min : A minimalistic view of your test reports

• MinText : A minimalistic view of your test reports for consoles

No matter what style you decide to use, you will still end up building a Testing Bundle CFC. This CFC will either contain BDD style suites and specs or xUnit style method tests. These components are simple components with no inheritance that can contain several different life-cycle callback methods. Internally we do the magic of making this CFC inherit all capabilities and methods from our BaseSpec class (testbox.system.BaseSpec).

Caution We highly encourage you to use the inheritance approach so you can get introspection, faster execution and ability to execute CFC directly.

TestBox ships with several test runners internally but we have tried to simplify and abstract it with our TestBox object which can be found in the testbox.system package. The TestBox object allows you to execute tests from a CFC, CFM, HTTP, NodeJS, SOAP or REST. You can also make your CFC's extend from our BaseSpec class so you can execute it directly via the URL. The main execution methods are:

// Run tests and produce reporter results
testbox.run()

// Run tests and get raw testbox.system.TestResults object
testbox.runRaw()

// Run tests and produce reporter results from SOAP, REST, HTTP
testbox.runRemote()

// Run via Spec URL
http://localhost/test/spec.cfc?method=runRemote

Info We encourage you to read the API docs included in the distribution for the complete parameters for each method.

run() Arguments

Here are the arguments you can use for initializing TestBox or executing the run() method