arrow-left

All pages
gitbookPowered by GitBook
1 of 10

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Release History

A brief history of TestBox

In this section, you will find the release notes for each version we release under this major version. If you are looking for the release notes of previous major versions, use the version switcher at the top left of this documentation book. Here is a breakdown of our major version releases.

hashtag
Version 6.x - September 2024

In this release, we have dropped more legacy engines and added full support for not only running TestBox in BoxLangarrow-up-right, but writing every spec in BoxLangarrow-up-right. We have added tons of bug fixes and major improvements but also a great CLI runner for BoxLang.

hashtag
Version 5.x - May 2023

In this release, we have dropped legacy engines and added support for the JVM language, Adobe 2023 and Lucee 6. We have also added major updates to spying and expectations. We continue in this series to focus on productivity and fluency in the Testing language in preparation for more ways to test.

hashtag
Version 4.x - April 2020

In this release, we have dropped support for legacy CFML engines and introduced the ability to mock data and relationships and build JSON documents.

hashtag
Version 3.x

In this release, we focused on dropping engine supports for legacy CFML engines. We had a major breakthrough in introducing Code Coverage thanks to the folks as well. This major release also came with a new UI for all reporters and streamlined the result viewports.

hashtag
Version 2.x

This version spawned off with over 8 minor releases. We focused on taking TestBox 1 to yet a high level. Much more attention to detail and introducing modern paradigms like given-when-then. Multiple interception points, async executions, and ability to chain methods.

hashtag
Version 1.x

This was our first major version of TestBox. We had completely migrated from MXUnit, and it introduced BDD to the ColdFusion (CFML) world.

BoxLangarrow-up-right
FusionReactorarrow-up-right

What's New With 6.2.0

January 31, 2025

This release certifies the entire test suite to run on BoxLang alone, no modules needed, not even CFML compat.

hashtag
Improvement

refactor of evaluate to use a direct approach instead of requiring the module for $getproperty mocking

BoxLang only updates and testing

TESTBOX-413arrow-up-right
TESTBOX-414arrow-up-right

Introduction

TestBox is a next-generation testing framework based on BDD (Behavior Driven Development) and TDD (Test Driven Development), providing a clean, obvious syntax for writing tests.

hashtag
TestBox BDD v6.x

TestBox is a next-generation testing framework for the BoxLangarrow-up-right JVM language and ColdFusion (CFML) based on BDDarrow-up-right (Behavior Driven Development) for providing a clean, obvious syntax for writing tests. It contains not only a testing framework, console/web runner, assertions, and expectations library but also ships with MockBox, A mocking and stubbing companion.

hashtag
Features At A Glance

Here is a simple listing of features TestBox brings to the table:

  • BDD style or xUnit style testing

  • Testing life-cycle methods

  • integration for mocking and stubbing

hashtag
Versioning

TestBox is maintained under the guidelines as much as possible. Releases will be numbered in the following format:

And constructed with the following guidelines:

  • bumpBreaking backward compatibility bumps the major (and resets the minor and patch)

  • New additions without breaking backward compatibility bump the minor (and resets the patch)

  • Bug fixes and misc changes bump the patch

hashtag
License

TestBox is open source and licensed under the License. If you use it, please try to mention it in your code or website.

  • Copyright by Ortus Solutions, Corp

  • TestBox is a registered trademark by Ortus Solutions, Corp

circle-info

The ColdBox Websites, Documentation, logo, and content have a separate license, and they are separate entities.

hashtag
Discussion & Help

  • Help Group:

  • BoxTeam Slack :

hashtag
Reporting a Bug

We all make mistakes from time to time :) So why not let us know about it and help us out? We also love pull requests, so please star us and fork us:

  • By Jira:

hashtag
Professional Open Source

TestBox is a professional open source software backed by offering services like:

  • Custom Development

  • Professional Support & Mentoring

  • Training

hashtag
Resources

  • Official Site:

  • Current API Docs:

  • Help Group:

hashtag
HONOR GOES TO GOD ABOVE ALL

Because of His grace, this project exists. If you don't like this, don't read it, it's not for you.

Therefore being justified by faith, we have peace with God through our Lord Jesus Christ: By whom also we have access by faith into this grace wherein we stand, and rejoice in hope of the glory of God. - Romans 5:5

class{

  function run(){
  	describe( "My calculator features", () => {
	
		beforeEach( () => {
			variables.calc = new Calculator()
		} )
			
		// Using expectations library
		it( "can add", () => {
			expect( calc.add(1,1) ).toBe( 2 )
		} )
		
		// Using assert library
		test( "it can multiply", () => {
			assertIsEqual( calc.multiply(2,2), 4 )
		} )
	} )
  }

}
Mocking data library for mocking JSON/complex data and relationships
  • Ability to extend and create custom test runners and reporters

  • Extensible reporters, bundled with tons of them:

    • JSON

    • XML

    • JUnit XML

    • Text

    • Console

    • TAP ()

    • Simple HTML

    • Min - Minimalistic Heaven

    • Raw

    • CommandBox

  • Asynchronous testing

  • Multi-suite capabilities

  • Test skipping

  • Test labels and tagging

  • Testing debug output stream

  • Code Coverage via FusionReactorarrow-up-right

  • Much more!

  • Server Tuning
  • Security Hardening

  • Code Reviews

  • Much Morearrow-up-right

  • Source Code: https://github.com/Ortus-Solutions/TestBoxarrow-up-right

  • Bug Tracker: https://ortussolutions.atlassian.net/browse/TESTBOXarrow-up-right

  • Twitter: @ortussolutionsarrow-up-right

  • Facebook: https://www.facebook.com/ortussolutionsarrow-up-right

  • MockBoxarrow-up-right
    Semantic Versioningarrow-up-right
    Apache 2arrow-up-right
    https://community.ortussolutions.com/c/communities/testbox/11arrow-up-right
    https://boxteam.ortussolutions.comarrow-up-right
    https://github.com/Ortus-Solutions/TestBoxarrow-up-right
    https://ortussolutions.atlassian.net/browse/TESTBOXarrow-up-right
    Ortus Solutions, Corparrow-up-right
    https://www.ortussolutions.com/products/testboxarrow-up-right
    https://apidocs.ortussolutions.com/testbox/currentarrow-up-right
    https://community.ortussolutions.com/c/communities/testbox/11arrow-up-right
    Runner
    Ortus Solutions, Corp

    What's New With 6.2.1

    February 6, 2025

    hashtag
    Bugs

    More BoxLang only mode support

    version number fixes for snapshot

    What's New With 6.3.0

    March 31, 2025

    We are excited to announce the release of TestBox 6.3.0. This version emphasizes the all-new BoxLang Reporter, now with vibrant color outputs and seamless execution through BoxLang, improving the testing experience. To benefit from these improvements, update now and ensure your testing remains reliable and efficient.

    You can find the new BoxLang runners in the root of the TestBox package:

    • /testbox/run - Unix/Mac runner

    /**
     * My calculator features
     */
    class{
    
    	property calc;
    	
    	function setup(){
    	    calc = new Calculator()
    	}
    	
    	// Function name includes the word 'test'
    	// Using expectations library
    	function testAdd(){
    	    expect( calc.add(1,1) ).toBe( 2 )
    	}
    		
    	// Any name, but with a test annotation
    	// Using assertions library
    	@test
    	function itCanMultiply(){
    	    $assert.isEqual( calc.multiply(2,2), 4 )
    	}
    }
    component{
    
      function run(){
      	describe( "My calculator features", () => {
    	
    		beforeEach( () => {
    			variables.calc = new Calculator()
    		} );
    			
    		// Using expectations library
    		it( "can add", () => {
    			expect( calc.add(1,1) ).toBe( 2 )
    		} );
    		
    		// Using assert library
    		test( "it can multiply", () => {
    			$assert.isEqual( calc.multiply(2,2), 4 )
    		} );
    	} );
      }
    
    }
    /**
     * My calculator features
     */
    component{
    	
    	property calc;
    	
    	function setup(){
    	    calc = new Calculator()
    	}
    	
    	// Function name includes the word 'test'
    	// Using expectations library
    	function testAdd(){
    	    expect( calc.add(1,1) ).toBe( 2 )
    	}
    		
    	// Any name, but with a test annotation
    	// Using assertions library
    	function itCanMultiply() test{
    	    $assert.isEqual( calc.multiply(2,2), 4 )
    	}
    }
    <major>.<minor>.<patch>
    TESTBOX-415arrow-up-right
    TESTBOX-416arrow-up-right
    /testbox/run.bat - Windows runner

    These runners will allow you to run any of your specs and bundles right from the CLI using BoxLang. Please read more in the

    hashtag
    Improvements

    • BoxLang only usage improvements

    • Console reporter now includes colors and the ability to execute via boxlang

    hashtag
    Bugs

    • initArgs.bundles can be an array or simple value, consolidate it in boxlang runner

    • Reporter options not being passed correctly

    Test Anything Protocolarrow-up-right
    BoxLang Runner section.
    TESTBOX-417arrow-up-right
    TESTBOX-419arrow-up-right
    TESTBOX-418arrow-up-right
    TESTBOX-420arrow-up-right
    BoxLang Console Reporter

    What's New With 6.0.1

    December 5, 2024

    hashtag
    Improvements

    TESTBOX-407arrow-up-right switch from java method array.addAll() to cfml array.Append()

    hashtag
    Bugs

    The identification of the web pageContext had a typo: getpageContext instead of just page context for BoxLang detection

    TESTBOX-404arrow-up-right

    About This Book

    Learn about the authors of TestBox and how to support the project.

    The source code for this book is hosted on GitHub: https://github.com/Ortus-Solutions/testbox-docsarrow-up-right. You can freely contribute to it and submit pull requests. The contents of this book is copyrighted by Ortus Solutions, Corparrow-up-right and cannot be altered or reproduced without the author's consent. All content is provided "As-Is" and can be freely distributed.

    hashtag
    External Trademarks & Copyrights

    Flash, Flex, ColdFusion, and Adobe are registered trademarks and copyrights of Adobe Systems, Inc.

    BoxLang, ColdBox, CommandBox, FORGEBOX, TestBox, ContentBox, and Ortus Solutions are all trademarks and copyrights of Ortus Solutions, Corp.

    hashtag
    Notice of Liability

    The information in this book is distributed “as is” without warranty. The author and Ortus Solutions, Corp shall not have any liability to any person or entity concerning loss or damage caused or alleged to be caused directly or indirectly by the content of this training book, software, and resources described in it.

    hashtag
    Contributing

    We highly encourage contributions to this book and our open-source software. The source code for this book can be found in our where you can submit pull requests.

    hashtag
    Charitable Proceeds

    15% of the proceeds of this book will go to charity to support orphaned kids in El Salvador - . So please donate and purchase the printed version of this book; every book sold can help a child for almost 2 months.

    hashtag
    Shalom Children's Home

    Shalom Children’s Home () is one of the ministries that are dear to our hearts located in El Salvador. During the 12-year civil war that ended in 1990, many children were left orphaned or abandoned by parents who fled El Salvador. The Benners saw the need to help these children and received 13 children in 1982. Little by little, more children came on their own, churches and the government brought children to them for care, and the Shalom Children’s Home was founded.

    Shalom now cares for over 80 children in El Salvador, from newborns to 18 years old. They receive shelter, clothing, food, medical care, education, and life skills training in a Christian environment. The home is supported by a child sponsorship program.

    We have personally supported Shalom for over 6 years now; it is a place of blessing for many children in El Salvador who either has no families or have been abandoned. This is a good earth to seed and plant.

    What's New With 6.1.0

    January 28, 2025

    hashtag
    New Features

    • TESTBOX-412arrow-up-right Updated to use cbMockData now instead of MockDataCFC

    hashtag
    Improvements

    • Support BoxLang without needing compat

    hashtag
    Bugs

    • Allow toHaveKey to support struct-like objects

    • Error when using the url.excludes with the HTML runner

    • fix missing `cfloop` on test browser

    TESTBOX-409arrow-up-right
    TESTBOX-408arrow-up-right
    TESTBOX-410arrow-up-right
    TESTBOX-411arrow-up-right
    GitHub repositoryarrow-up-right
    http://www.harvesting.org/arrow-up-right
    https://www.harvesting.org/arrow-up-right
    Shalom Children's Home

    What's New With 6.0.0

    September 27, 2024

    TestBox 6.x series is a major bump in our library. Here are the major areas of improvement and the full release notes.

    hashtag
    BoxLang Has Arrived!

    www.boxlang.io

    BoxLangarrow-up-right is the newest JVM language that can support running not only BoxLang language files but CFML files with our bx-compat-cfml module. In TestBox 6 we now not only support and certify that it runs in CFML compatibility mode, but you can now create all your tests, specs and harnesses in BoxLang. You can find the base test harness for BoxLang here:

    Here is a sample Spec written in BoxLang

    hashtag
    TestBox CLI

    We have also updated the testbox-cli to now support BoxLang native generation. It will detect if you are in a BoxLang server or if you have the new language entry in your box.json

    Also all generation commands have a new boxlang argument, which is a boolean argument you can use to explicitly generate Boxlang code.

    circle-check

    The CLI will detect if it's a BoxLang project if:

    • If there is a CommandBox BoxLang server detected in the root of the project

    hashtag
    BoxLang CLI Runner

    We have also created a new runner for BoxLang exclusively. This runner allows you to run your specs, and bundles from the CLI with no web server required. You can find the entire docs for the runner in our page. The runner must be run from the root of your BoxLang project:

    hashtag
    Mac/Linux

    hashtag
    Windows Examples:

    circle-check

    Remember that BoxLang not only allows you to build web server applications, but also CLI applications, serverless (AWS Lambdas, Azure Functions), Android and more.

    hashtag
    Headless Web Server Testing

    We have also created a new module called bx-web-support which will allow you to do headless web server testing right from the CLI.

    This will add web support to the CLI (BIFS, components, etc.) and a mock HTTP server so you can do full life-cycle testing from the CLI like if running your app in a web server. This runner does not require a web server to function, thus if you are building a web app, you will need this module if you still want to continue to execute your tests in the CLI Runtime.

    circle-check

    If you are building exclusively a web application, we suggest you use the which will call your runner via HTTP from the CLI. You can also just use the .

    hashtag
    Engine Support

    Adobe 2018 has been dropped and BoxLang is now fully supported and certified with special features JUST for BoxLang.

    hashtag
    New Environment Helpers

    All the test bundles will now inherit several new methods to assist in environment and OS detection:

    Method
    Purpose

    hashtag
    New Spec Alias: test()

    We have added a new method for your BDD tests called test() so you can create even further human-readable tests apart from it(), then()

    hashtag
    Display Names

    If you are building your tests in xUnit mode, then you are getting more features especially for your reports. Instead of just seeing the name of the function, you can now annotate it with a displayName annotation and give it a human readable title.

    hashtag
    BoxLang Dynamic Assertions

    If you are writing your specs in BoxLang you will start to get further advantages than in CFML. Here is the first one, dynamic assertion methods. Before, in order to use the assertions library you had to use the $assert variable and call the assertion methods on it:

    Now, you can use our dynamic delegator and just simply your assertions:

    All these dynamic methods will proxy to the

    hashtag
    Release Notes

    hashtag
    New Features

    MockBox converted to script

    BoxLang classes support

    New environment helpers to do skip detections or anything you see fit: isAdobe, isLucee, isBoxLang, isWindows, isMac, isLinux

    new `test(), xtest(), ftest()` alias for more natuarl testing

    debug() get's two new arguments: label and showUDFs

    DisplayName on a bundle now shows up in the reports

    xUnit new annotation for @DisplayName so it can show instead of the function name

    BoxLang CLI mode and Runner

    New matcher: toHaveKeyWithCase()

    Assertions: key() and notKey() now have a CaseSensitive boolean argument

    hashtag
    Improvements

    showUDFs = false option with debug()

    TextReporter doesn't correctly support testBundles URL param

    adding missing focused argument to spec methods

    Generating a repeatable id for specs to track them better in future UIs

    hashtag
    Bugs

    If test spec descriptor contains a comma, it can not be drilled down to run that one spec directly

    describe handler in non-called test classes being executed

    hashtag
    Tasks

    Drop Adobe 2018 support

    If the
    runner
    defined in your
    box.json
    is called
    boxlang
  • If the language=boxlang in your box.json is detected

  • isAdobe()

    Are you running the test in an Adobe CFML Engine

    isLucee()

    Are you running the test in a Lucee CFML Engine

    isBoxLang()

    Are you running the test in a BoxLang Engine

    isWindows()

    Are you in a windows OS

    isLinux()

    Are you in a Linux OS

    isMac()

    Are you in a Mac OS

    https://github.com/Ortus-Solutions/TestBox/tree/development/bx/testsarrow-up-right
    BoxLang CLI Runner
    CommandBox runner
    Web Runner
    assertions library.
    TESTBOX-391arrow-up-right
    TESTBOX-392arrow-up-right
    TESTBOX-393arrow-up-right
    TESTBOX-394arrow-up-right
    TESTBOX-397arrow-up-right
    TESTBOX-398arrow-up-right
    TESTBOX-399arrow-up-right
    TESTBOX-401arrow-up-right
    TESTBOX-402arrow-up-right
    TESTBOX-403arrow-up-right
    TESTBOX-289arrow-up-right
    TESTBOX-331arrow-up-right
    TESTBOX-395arrow-up-right
    TESTBOX-396arrow-up-right
    TESTBOX-123arrow-up-right
    TESTBOX-338arrow-up-right
    TESTBOX-400arrow-up-right
    class extends="testbox.system.BaseSpec"{
    
        function run(){
    	describe( "My First Test", ()=>{
    		test( "it can add", ()=>{
    			expect( sum( 1, 2 ) ).toBe( 3 )
    		} )
    	} )
        }
    
        private function sum( a, b ){
            return a + b
        }
    
    }
    {
        "name":"MyBoxLang Project",
        "version":"1.0.0",
        "language" : "boxlang" // or CFML or JAVA
    }
    testbox create bdd MyTest --boxlang
    testbox generate harness --boxlang
    ./testbox/bin/run
    ./testbox/bin/run my.bundle
    ./testbox/bin/run --directory=tests.specs
    ./testbox/bin/run --bundles=my.bundle
    ./testbox/bin/run.bat
    ./testbox/bin/run.bat my.bundle
    ./testbox/bin/run.bat --directory=tests.specs
    ./testbox/bin/run.bat --bundles=my.bundle
    // CommandBox
    install bx-web-support
    
    // BoxLang OS Binary
    install-bx-module bx-web-support
    describe("User Authentication", () => {
    
        test("should successfully login with valid credentials", () => {
            var user = authenticate("validUser", "validPassword");
            expect( user.isAuthenticated() ).toBe(true);
        });
    
        test("should fail login with invalid credentials", () => {
            var user = authenticate("invalidUser", "invalidPassword");
            expect( user.isAuthenticated() ).toBe(false);
        });
    
    });
    // Before
    function testAddition(){
        assert( calc.add(2,3) == 5 )
    }
    
    function testMultiply(){
        assert( calc.multiply(2,3) == 6 )
    }
    
    // After
    
    @DisplayName "My calculator can add"
    function testAddition(){
        assert( calc.add(2,3) == 5 )
    }
    
    @DisplayName "My calculator can multiply"
    function testMultiply(){
        assert( calc.multiply(2,3) == 6 )
    }
    // Before
    function testAddition(){
        assert( calc.add(2,3) == 5 )
    }
    
    function testMultiply(){
        assert( calc.multiply(2,3) == 6 )
    }
    
    // After
    
    function testAddition() DisplayName="My calculator can add"{
        assert( calc.add(2,3) == 5 )
    }
    
    function testMultiply() DisplayName="My calculator can multiply"{
        assert( calc.multiply(2,3) == 6 )
    }
    $assert.isTrue()
    $assert.isFalse()
    $assert.isEqual()
    $assert.isNotEqual()
    $assert.null()
    assertIsTrue()
    assertIsFalse()
    assertIsEqual()
    assertIsNotEqual()
    assertNull()

    Author

    hashtag
    Luis Fernando Majano Lainez

    Luis Majano is a Computer Engineer with over 16 years of software development and systems architecture experience. He was born in San Salvador, El Salvadorarrow-up-right in the late 70’s, during a period of economical instability and civil war. He lived in El Salvador until 1995 and then moved to Miami, Florida where he completed his Bachelors of Science in Computer Engineering at Florida International Universityarrow-up-right. Luis resides in The Woodlands, Texas with his beautiful wife Veronica, baby girl Alexia and baby boy Lucas!

    He is the CEO of Ortus Solutionsarrow-up-right, a consulting firm specializing in web development, ColdFusion (CFML), Java development and all open source professional services under the ColdBox and ContentBox stack. He is the creator of ColdBox, ContentBox, WireBox, MockBox, LogBox and anything “BOX”, and contributes to many open source ColdFusion projects. He is also the Adobe ColdFusion user group manager for the . You can read his blog at

    Luis has a passion for Jesus, tennis, golf, volleyball and anything electronic. Random Author Facts:

    • He played volleyball in the Salvadorean National Team at the tender age of 17

    • The Lord of the Rings and The Hobbit is something he reads every 5 years. (Geek!)

    • His first ever computer was a Texas Instrument TI-86 that his parents gave him in 1986. After some time digesting his very first BASIC book, he had written his own tic-tac-toe game at the age of 9. (Extra geek!)

    Keep Jesus number one in your life and in your heart. I did and it changed my life from desolation, defeat and failure to an abundant life full of love, thankfulness, joy and overwhelming peace. As this world breathes failure and fear upon any life, Jesus brings power, love and a sound mind to everybody!

    “Trust in the LORD with all your heart, and do not lean on your own understanding.” Proverbs 3:5

    hashtag
    Contributors

    hashtag
    Jorge Emilio Reyes Bendeck

    Jorge is an Industrial and Systems Engineer born in El Salvador. After finishing his Bachelor studies at the Monterrey Institute of Technology and Higher Education , Mexico, he went back to his home country where he worked as the COO of. In 2012 he left El Salvador and moved to Switzerland in persuit of the love of his life. He married her and today he resides in Basel with his lovely wife Marta and their daughter SofĂ­a.

    Jorge started working as project manager and business developer at Ortus Solutions, Corp. in 2013, . At Ortus he fell in love with software development and now enjoys taking part on software development projects and software documentation! He is a fellow Cristian who loves to play the guitar, worship and rejoice in the Lord!

    Therefore, if anyone is in Christ, the new creation has come: The old has gone, the new is here! 2 Corinthians 5:17

    He has a geek love for circuits, microcontrollers and overall embedded systems.

  • He has of late (during old age) become a fan of running and bike riding with his family.

  • Inland Empirearrow-up-right
    www.luismajano.comarrow-up-right
    ITESMarrow-up-right
    Industrias Bendek S.A.arrow-up-right