Roadmap

This library is free, and will stay free but needs your support to sustain its development. There are lots of new features and maintenance to do. If you work for a company using doctest or have the means to do so, please consider financial support.

Pledgie Patreon PayPal

Planned features for future releases - order changes constantly...

For 1.2:

  • integrate static analysis on the CI: clang
  • decorators for test cases - like in boost test
    • depends_on (decorator)
    • description (decorator)
    • enabled / disabled (decorator)
    • enable_if (decorator)
    • fixture (decorator)
    • label (decorator)
    • precondition (decorator)
    • expected_failures (decorator)
    • timeout (decorator)
    • shouldFail => reported as "x failed as expected" (both asserts and test cases)
    • mayFail
    • alternative mechanism to tags
    • run X times (should also multiply with the global test run times)
    • !!! and think about how these will be accessed and filtered from the command line
  • time stuff
    • reporting running time of tests
    • count a test case as failed if it exceeds X ms (but no force-killing!) Entering test module "decorator_08" test.cpp(6): Entering test case "test1" test.cpp(6): Leaving test case "test1"; testing time: 1ms test.cpp(11): Entering test case "test2" test.cpp(13): error: in "test2": check false has failed test.cpp(11): Leaving test case "test2"; testing time: 2ms test.cpp(39): Entering test case "test3" test.cpp(41): error: in "test3": check false has failed test.cpp(39): Leaving test case "test3"; testing time: 2ms test.cpp(45): Test case "test4" is skipped because test2 and test3 failed Leaving test module "decorator_08"; testing time: 16ms
  • runtime performance
    • move string implementation to the fwd part - use new/delete
    • lazily stringify expressions - only when needed
    • optimize createStream/freeStream to reuse a pool of ostringstream objects
    • get rid of local statics on the hot path - like in getContextState()
    • make a pool allocator for the String class - currently very unoptimized
    • add move semantics to the String class
  • mocking - interop with doctest (+ maybe send PRs with doctest integration info)
  • benchmarking
    • make the bench.py script more usable - with command line arguments
    • redo the compile time ones - also look into CATCH_CONFIG_FAST_COMPILE
    • remove old benchmarks for doctest 1.0
    • add runtime benchmarks
  • change docs a bit
  • address the coverage issue... look at how these projects do it:
  • builds with GCC 7 when it is released (should be in late April)

For 1.3:

For 1.4:

  • running tests a few times
  • test execution in separate processes - fork() for UNIX and this for Windows
  • killing a test that exceeds a time limit (will perhaps require threading or processes)
  • symbolizer - for a stack trace - when an assertion fails - and it's in a user function with some deep callstack away from the current test case - how to know the exact code path that lead to the failing assert
  • ability to make the framework not capture unexpected exceptions - as requested here
  • add Approx ability to compare with absolute epsilon - Catch PR
  • ability to customize the colors in the console output (may also use styles - based on this)
  • implement breaking into the debugger under linux - see here and here
  • better testing of the library
    • should unit test internals
    • should test stuff that should not compile

For 2.0:

  • remove C++98 support
    • remove the config identifiers for C++11 features
    • use variadic templates where appropriate
    • update type lists to C++11
    • update traits - use declval, etc.
    • move initialization of fields from initializer lists to class bodies

Things that are being considered but not part of the roadmap yet:

  • rpm package? like this: https://github.com/vietjtnguyen/argagg/blob/master/packaging/rpm/argagg.spec
  • get the current test case/section path - https://github.com/philsquared/Catch/issues/522
  • when no assertion is encountered in a test case it should fail
  • failure reporting should print out previous SECTIONs for data-driven testing - as requested here
  • Bitwise() class that has overloaded operators for comparison - to be used to check objects bitwise against each other
  • detect floating point exceptions
  • checkpoint/passpoint - like in boost test (also make all assert/subcase/logging macros to act as passpoints and print the last one on crashes or exceptions)
  • log levels - like in boost test
  • queries for the current test case - name (and probably decorators)
  • thread safety - asserts/subcases/captures should be safe to be used by multiple threads simultaneously
  • support for running tests in parallel in multiple threads
  • death tests - as in google test
  • command line
    • ability to specify ASC/DESC for the order option
    • command line error handling/reporting
    • ability for the user to extend the command line - as requested here
    • option to list files in which there are test cases who match the current filters
    • option for filters to switch from "match any" to "match all" mode
    • option to list test suites and test cases in a tree view
    • add a "wait key" option - as requested here
  • setup / teardown support
    • global setup / teardown - can be currently achieved by providing a custom main function
    • per test suite
    • see how it's done in boost test - with the fixture decorator
    • perhaps for fixtures in addition to the constructor / destructor - since throwing in the destructor might terminate the program
    • or just ignore all of this this - it would require globals or classes and inheritance - and we already have subcases
  • doctest in a GUI environment? with no console? APIs for attaching a console? querying if there is one? investigate...
  • the set holding all registered tests should use a specialized allocator to minimize program startup time
  • ability to provide a temp folder that is cleared between each test case
  • make the _MESSAGE assert macros work with variadic arguments - and maybe write the ones for binary/unary/fast asserts as well
  • move from operator "<<" to "<=" for capturing the left operand when decomposing binary expressions with templates
  • think about silencing warnings about unused variables when DOCTEST_CONFIG_DISABLE is used - see commit 6b61e8aa3818c5ea100cedc1bb48a60ea10df6e8 or issue #61
  • think about optionally using <typeinfo> and libcxxabi for demangling so users don't have to use TYPE_TO_STRING()

Things that are very unlikely to enter the roadmap:

  • think about counting failures added with FAIL() or ADD_FAIL_AT() as failed asserts
  • think about removing the binary asserts (leaving only the fast binary asserts) because normal asserts + no try/catch in asserts are almost the same
  • move the "react()" part (the one that throws for REQUIRE asserts - or for when "abort-after=" is reached) to a function call in the while() part of the asserts
  • stop using underscores for the begining of identifiers - the anonymous variables - against the standard...
  • templated fixture test cases
  • test with missed warning flags for GCC
  • utf8???
  • handle wchar strings???
  • hierarchical test suites - using a stack for the pushed ones
  • ability to specify the width of the terminal in terms of characters (for example 60 - less than 80 - the default)
  • ability to re-run only newly compiled tests based on time stamps using __DATE__ and __TIME__ - stored in some file
  • add underscores to all preprocessor identifiers not intended for use by the user
  • put everything from the detail namespace also in a nested anonymous namespace to make them with internal linkage
  • ability to put everything from doctest into an anonymous namespace - to allow the use of multiple different versions of doctest within the same binary (executable/dll) - like the stb libraries can

Home