May 10, 2022
TestBox 5.x series is a major bump in our library. Here are the major areas of improvement and the full release notes.
We have dropped Adobe 2016 support and added support for Adobe 2023 and Lucee 6+
Due to memory limitations in CI environments, larger codebases cannot run all tests as a single testbox run command. Instead, specs are run in a methodical folder-by-folder sequence, separating the testbox run out over many requests and thus working around the Out-Of-Memory exceptions.
While this works, it prevents accurate code coverage reporting since only a small portion of the tests are executed during any request. The generated code coverage report only shows a tiny fraction of the coverage - say, 2% - and not the whole picture
TestBox 5 introduces a CoverageReporter component which
Runs on every TestBox code coverage execution
Loads any previous coverage data from a JSON file
Combines the previous coverage data with the current execution's coverage data (file by file and line by line)
Persists the COMBINED coverage data to a JSON file.
Returns the COMBINED coverage data for the CoverageBrowser.cfc to build as an HTML report
When setting url.isBatched=true and executing the batched test runner, the code coverage report will grow with each sequential testbox run command.
MockBox now supports a $spy( method ) method that allows you to spy on methods with all the call log goodness but without removing all the methods. Every other method remains intact, and the actual spied method remains active. We decorate it to track its calls and return data via the $callLog() method.
Example of CUT:
Example Test:
We have focused on this release to lazy load everything as much as possible to allow for much better testing performance. Check it out!
You can now use the skip( message ) method to skip any spec or suite a-la-carte instead of as an argument to the function definitions. This lets you programmatically skip certain specs and suites and pass a nice message.
TESTBOX-341 toHaveLength param should be numeric
TESTBOX-354 Element $DEBUGBUFFER is undefined in THIS
TESTBOX-356 Don't assume TagContext has length on simple reporter
TESTBOX-357 notToThrow() incorrectly passes when no regex is specified
TESTBOX-360 full null support not working on Application env test
TESTBOX-361 MockBox Suite: Key [aNull] doesn't exist
TESTBOX-362 Cannot create subfolders within testing spec directories.
TESTBOX-333 Add contributing.md to the repo
TESTBOX-339 full null support automated testing
TESTBOX-353 allows globbing path patterns in test bundles argument
TESTBOX-355 Add debugBuffer to JSONReporter
TESTBOX-366 ANTJunit Reporter better visualization of the failed origin and details
TESTBOX-368 Support list of Directories for HTMLRunner to allow a more modular tests structure
TESTBOX-370 `toHaveKey` works on queries in Lucee but not ColdFusion
TESTBOX-371 Add CoverageReporter for batching code coverage reports
TESTBOX-137 Ability to spy on existing methods: $spy()
TESTBOX-342 Add development dependencies to box.json
TESTBOX-344 Performance optimizations for BaseSpec creations by lazy loading external objects
TESTBOX-345 add a skip([message]) like fail() for skipping from inside a spec
TESTBOX-365 New build process using CommandBox
TESTBOX-372 Adobe 2023 and Lucee 6 Support
A brief history of TestBox
In this section, you will find the release notes for each version we release under this major version. If you are looking for the release notes of previous major versions, use the version switcher at the top left of this documentation book. Here is a breakdown of our major version releases.
In this release, we have dropped legacy engines and added support for the JVM language, Adobe 2023 and Lucee 6. We have also added major updates to spying and expectations. We continue in this series to focus on productivity and fluency in the Testing language in preparation for more ways to test.
In this release, we have dropped support for legacy CFML engines and introduced the ability to mock data and relationships and build JSON documents.
In this release, we focused on dropping engine supports for legacy CFML engines. We had a major breakthrough in introducing Code Coverage thanks to the folks as well. This major release also came with a new UI for all reporters and streamlined the result viewports.
This version spawned off with over 8 minor releases. We focused on taking TestBox 1 to yet a high level. Much more attention to detail and introducing modern paradigms like given-when-then. Multiple interception points, async executions, and ability to chain methods.
This was our first major version of TestBox. We had completely migrated from MXUnit, and it introduced BDD to the ColdFusion (CFML) world.
July 28, 2023
Updated mixerUtil for faster performance and new approaches to dynamic mixins
Add `bundlesPattern` to testbox.system.TestBox `init` method
TestBox Modules
expect(sut).toBeInstanceOf("something")) breaks if sut is a query
cbstreams doesn't entirely work outside of ColdBox
toBeInstanceOf() Expectation handle Java classes
August 1, 2023
The variable thisSuite isn't defined if the for loop in the try/catch is never reached before the error. (#150)
TESTBOX-379 New expectations: toBeIn(), toBeInWithCase() so you can verify a needle in string or array targets
TESTBOX-380 New matchers and assertions: toStartWith(), toStartWithCase(), startsWith(), startsWthCase() and their appropriate negations
TESTBOX-381 New matchers and assertions: toEndWith(), toEndWithCase(), endsWith(), endsWithCase() and their appropriate negations
TESTBOX-378 onSpecError suiteSpecs is invalid, it's suiteStats
