TestBox : Behavior Driven Development (BDD)
API DocsSourceSupportBoxLang
v6.x
v6.x
  • Introduction
    • Release History
      • What's New With 6.3.0
      • What's New With 6.2.1
      • What's New With 6.2.0
      • What's New With 6.1.0
      • What's New With 6.0.1
      • What's New With 6.0.0
    • About This Book
      • Author
  • Getting Started
    • Overview
    • Installation
      • IDE Tools
      • MXUnit Compatibility
    • Writing Tests
    • Running Tests
      • BoxLang CLI Runner
      • CommandBox Runner
      • Web Runner
      • Bundle(s) Runner
      • Directory Runner
      • ANT Runner
      • NodeJS Runner
      • Global Runner
      • Test Browser
    • BDD Tests
      • Bundles: Group Your Tests
      • Suites: Describe Your Tests
        • Dynamic Suites
      • Specs
      • Expectations
      • Suite Groups
        • Given-When-Then Blocks
      • Life-Cycle Methods
      • Life-Cycle Data Binding
      • Specs and Suite Labels
      • Skipping Specs and Suites
      • Focused Specs and Suites
      • Spies & Mocking
      • Asynchronous Testing
      • Running Tests
      • Reporters
    • xUnit Tests
      • Test Bundles
      • Life-Cycle Methods
      • Test Methods
      • Assertions
      • Test and Suite Labels
      • Skipping Tests and Suites
      • Spies and Mocking
      • Asynchronous-Testing
      • Running Tests
      • Reporters
  • Digging Deeper
    • Life-Cycle Annotations
    • Assertions
      • Custom Assertions
    • Expectations
      • Matchers
      • Not Operator
      • Expecting Exceptions
      • Custom Matchers
    • Output Utilities
    • Runner Listeners
    • Reporters
      • Custom Reporters
    • Modules
    • Code Coverage
      • Running Code Coverage
      • Configuring Code Coverage
      • Known Behaviors
    • Continous Integration
      • Github Actions
      • Gitlab
      • Travis
  • Mocking
    • MockBox
      • What is Mocking?
      • Our Approach and Benefits
      • Creating MockBox
      • Creating a Mock Object
      • Creating a Stub Object
      • Mocking Methods
        • $() Method
        • $args() Method
        • $getProperty() Method
        • $property() Method
        • $querySim() Method
        • $results() Method
        • $spy()
        • $throws() Method
      • Verification Methods
        • $count()
        • $times() or $verifyCallCount()
        • $never()
        • $atLeast()
        • $once()
        • $atMost()
        • $callLog()
        • $reset()
        • $debug()
      • Some Examples
      • Conclusion
    • Mocking Data
Powered by GitBook

Social Media

  • YouTube
  • x
  • FaceBook
  • LinkedIn

Downloads

  • CommandBox
  • BoxLang
  • Try BoxLang

Support

  • Professional
  • Community
  • Slack
  • CFCasts

Copyright & Register Trademark by Ortus Solutions, Corp & Ortus Software, LLC

On this page
  • Web Server Testing
  • Script Locations
  • Examples:
  • Execution Options
  • Reporting Options
  • Runner Options
  • Filtering Options

Was this helpful?

Edit on GitHub
Export as PDF
  1. Getting Started
  2. Running Tests

BoxLang CLI Runner

Run your tests directly with BoxLang in the CLI.

PreviousRunning TestsNextCommandBox Runner

Last updated 1 month ago

Was this helpful?

TestBox ships with a BoxLang CLI runner for Linux/Mac and Windows. This will allow you to execute your tests from the CLI and, in the future, via VSCode easily and extremely fast. It can also execute and stream the executions to see the testing progress when running in verbose mode. The runner can also execute specs/tests written in CFML or BoxLang in the BoxLang runtime.

Please note that this is a BoxLang-only feature.

BoxLang allows you to build web applications, CLI, serverless, Android, and more. You can use this runner to test each environment. However, please note that if you will be testing web servers from the CLI only, you will need to install the web support module into the Operating System runtime.

Web Server Testing

If you want to test from the CLI your web application with no web server, then you will need to install the bx-web-support module into the CLI. Remember that BoxLang is multi-runtime, and you cannot only build web applications but CLI or OS-based applications.

// BoxLang OS Binary
install-bx-module bx-web-support

This will add web support to the CLI (BIFS, components, etc.) and a mock HTTP server so you can do full life-cycle testing from the CLI, like running your app on a web server. This runner does not require a web server to function; thus, if you are building a web app, you will need this module if you still want to execute your tests in the CLI Runtime.

Script Locations

The scripts are located in the following directory: /testbox/ from the TestBox installation package.

  • run - For Linux/Mac

  • run.bat - For Windows

This is the entry point for executing tests at the CLI level. Please note that the test execution does NOT involve a web server. This is for pure CLI testing.

Examples:

The runner must be run from the root of your BoxLang project:

Mac/Linux

./testbox/run
./testbox/run my.bundle
./testbox/run --directory=tests.specs
./testbox/run --bundles=my.bundle

Windows Examples:

./testbox/run.bat
./testbox/run.bat my.bundle
./testbox/run.bat --directory=tests.specs
./testbox/run.bat --bundles=my.bundle

Execution Options

  • --bundles A list of test bundles to run, defaults to *, ex: path.to.bundle1,path.to.bundle2, . Mutually exclusive with --directory

  • --bundles-pattern A pattern to match test bundles defaults to "*Spec*.cfc|*Test*.cfc|*Spec*.bx|*Test*.bx"

  • --directory A list of directories to look for tests to execute. Please use dot-notation, not absolute notation. Mutually exclusive with --bundles. Ex: tests.specs Defaults to tests.specs

  • --recurse : Recurse into subdirectories, defaults to true

  • --eager-failure : Fail fast, defaults to false

  • --verbose : Verbose output defaults to false. This will stream the output of the status of the tests as they run.

Reporting Options

  • --reporter The reporter to use.

  • --reportpath : The path to write the report file defaults to the /tests/results folder by convention

  • --properties-summary : Generate a properties file with the summary of the test results, defaults to true.

  • --properties-filename : The name of the properties file to generate defaults to TEST.properties

  • --write-report : Write the report to a file in the report path folder, defaults to true

  • --write-json-report : Write the report as JSON alongside the requested report, defaults to false

  • --write-visualizer : Write the visualizer to a file in the report path folder, defaults to false

Runner Options

You can also pass options to the selected runner by using the following convention:

--runner-{option}=value

This will pass a struct of options to the runners once it runs.

Filtering Options

  • --labels : A list of labels to use in the execution.

  • --excludes : A list of labels to exclude in the execution.

  • --filter-bundles : A list of bundles to filter by, defaults to *

  • --filter-suites : A list of suites to filter by, defaults to *

  • --filter-specs : A list of test names or spec names to filter by, defaults to *

If you are building exclusively a web application, we suggest you use the which will call your runner via HTTP from the CLI. You can also just use the .

CommandBox runner
Web Runner