Running Tests

Test All Things!

TestBox tests can be run from different sources from what we call Runners. These can be from different sources:

• CLI

  • TestBox CLI (Powered by CommandBox)

  • BoxLang Scripts

  • NodeJS

• Web Server

  • Runner

  • TestBundle Directly

• Custom

Your test harness already includes the web runner: runner.bx or runner.cfm. You can execute that directly in your browser to get the results or run it via the CLI: testbox run. We invite you to explore the different runners available to you.

CommandBox RunnerWeb RunnerBundle(s) RunnerDirectory RunnerANT RunnerNodeJS RunnerGlobal RunnerTest Browser

Custom Runners

However, you can create your own custom runners as long as you instantiate the TestBox class and execute one of it's runnable methods. The main execution methods are:

// Run tests and produce reporter results
testbox.run()

// Run tests and get raw testbox.system.TestResults object
testbox.runRaw()

// Run tests and produce reporter results from SOAP, REST, HTTP
testbox.runRemote()

// Run via Spec URL
http://localhost/tests/spec.cfc?method=runRemote

// Via CommandBox
testbox run

We encourage you to read the API docs included in the distribution for the complete parameters for each method.

run()

Here are the arguments you can use for initializing TestBox or executing the run() method

/**
 * Run me some testing goodness, this can use the constructed object variables or the ones
 * you can send right here.
 *
 * @bundles      The path, list of paths or array of paths of the spec bundle classes to run and test
 * @directory    The directory to test which can be a simple mapping path or a struct with the following options: [ mapping = the path to the directory using dot notation (myapp.testing.specs), recurse = boolean, filter = closure that receives the path of the class found, it must return true to process or false to continue process ]
 * @reporter     The type of reporter to use for the results, by default is uses our 'simple' report. You can pass in a core reporter string type or an instance of a testbox.system.reports.IReporter. You can also pass a struct if the reporter requires options: {type="", options={}}
 * @labels       The list or array of labels that a suite or spec must have in order to execute.
 * @excludes     The list or array of labels that a suite or spec must not have in order to execute.
 * @options      A structure of configuration options that are optionally used to configure a runner.
 * @testBundles  A list or array of bundle names that are the ones that will be executed ONLY!
 * @testSuites   A list or array of suite names that are the ones that will be executed ONLY!
 * @testSpecs    A list or array of test names that are the ones that will be executed ONLY!
 * @callbacks    A struct of listener callbacks or a class with callbacks for listening to progress of the testing: onBundleStart,onBundleEnd,onSuiteStart,onSuiteEnd,onSpecStart,onSpecEnd
 * @eagerFailure If this boolean is set to true, then execution of more bundle tests will stop once the first failure/error is detected. By default this is false.
 */
any function run(
	any bundles,
	any directory,
	any reporter,
	any labels,
	any excludes,
	struct options,
	any testBundles      = [],
	any testSuites       = [],
	any testSpecs        = [],
	any callbacks        = {},
	boolean eagerFailure = false
)

• The bundles argument which can be a single CFC path or an array of CFC paths or a directory argument so it can go and discover the test bundles from that directory.

• The reporter argument can be a core reporter name like: json,xml,junit,raw,simple,dots,tap,min,etc or it can be an instance of a reporter CFC.

• You can execute the runners from any cfm template or any CFC or any URL, that is up to you.