Gitiles
Code Review Sign In
gerrit.cesnet.cz / github / onqtam / doctest / f1688b5b3ec6b7e0b80663873fee7d750ab9595e / . / doc / html_generated / roadmap.html
blob: acb67fb7edd7225512d8507b16f48bc1252024ec [file] [log] [blame]
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]1<!DOCTYPE html>
2<html>
3<title>roadmap</title>
4<xmp theme="united" style="display:none;">
5
6## Roadmap
7
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]8This library is free, and will stay free but needs your support to sustain its development. There are lots of [**new features**](roadmap.html) and maintenance to do. If you work for a company using **doctest** or have the means to do so, please consider financial support.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]9
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]10[![Patreon](https://cloud.githubusercontent.com/assets/8225057/5990484/70413560-a9ab-11e4-8942-1a63607c0b00.png)](http://www.patreon.com/onqtam)
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]11[![PayPal](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.me/onqtam/10)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]12
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]13Planned features for future releases - order changes constantly...
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]14
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]15### For 1.3:
16
17- move from printf to using streams in the current console reporter and clean that mess
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]18- reporters
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]19 - output to file
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]20 - a system for writing custom reporters
21 - ability to use multiple reporters at once (but only 1 to stdout)
22 - a compact reporter
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]23 - a progress reporter - or maybe just an option for the console reporter
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]24 - an xml reporter
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]25 - xUnit reporter
26 - a listener interface - similar to a reporter - look at Catch
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]27- ability to have no output when everything succeeds
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]28- option to output summary only
29- log levels - like in [boost test](http://www.boost.org/doc/libs/1_63_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/log_level.html)
30- matchers - should investigate what they are - look at google test/mock and Catch (also predicates and boost test)
31- convolution support for the assertion macros (with a predicate)
32- Value-Parameterized test cases
33- generators? - look at Catch - and investigate what they are
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]34- proper conan package
35 - https://github.com/uilianries/conan-catch
36 - https://github.com/philsquared/Catch/pull/938
37 - https://stackoverflow.com/questions/40311363/conan-io-header-only-package
38 - https://github.com/conan-io/docs/issues/67
39 - https://github.com/agauniyal/rang
40 - https://github.com/conan-io/conan/issues/324
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]41- look at property based testing
42 - [rapidcheck](https://github.com/emil-e/rapidcheck)
43 - [autocheck](https://github.com/thejohnfreeman/autocheck)
44 - [CppQuickCheck](https://github.com/grogers0/CppQuickCheck)
onqtam6ff3d352017-09-11 14:35:06 +0300[diff] [blame]45- support for LibIdentify
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]46- IDE integration
47 - https://blogs.msdn.microsoft.com/vcblog/2017/05/10/unit-testing-and-the-future-announcing-the-test-adapter-for-google-test/
48 - https://www.reddit.com/r/cpp/comments/65c0f1/run_cpp_unit_tests_from_xcode_and_visual_studio/
49 - https://github.com/k-brac/CUTI
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]50 - https://github.com/csoltenborn/GoogleTestAdapter
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]51 - MSTest
52 - http://accu.org/index.php/journals/1851
53 - https://msdn.microsoft.com/en-us/library/hh270865.aspx
54 - https://msdn.microsoft.com/en-us/library/hh598953.aspx
55 - https://blogs.msdn.microsoft.com/vcblog/2017/04/19/cpp-testing-in-visual-studio/
56 - https://msdn.microsoft.com/en-us/library/hh419385.aspx
57 - XCode - https://github.com/philsquared/Catch/pull/454
58 - CLion
59 - https://www.jetbrains.com/clion/features/unit-testing.html
60 - https://blog.jetbrains.com/clion/2017/03/clion-2017-1-released/#catch
61
62### For 1.4:
63
64- running tests a [few times](https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.html#repeating-the-tests)
65- test execution in [separate processes](https://github.com/philsquared/Catch/issues/853) - ```fork()``` for UNIX and [this](https://github.com/nemequ/munit/issues/2) for Windows
66- killing a test that exceeds a time limit (will perhaps require threading or processes)
67- [symbolizer](https://github.com/facebook/folly/tree/master/folly/experimental/symbolizer) - for a stack trace - when an assertion fails - and it's in a user function with some deep callstack away from the current test case - how to know the exact code path that lead to the failing assert
68- ability to make the framework not capture unexpected exceptions - as requested [here](https://github.com/onqtam/doctest/issues/12#issuecomment-235334585)
69- add Approx ability to compare with absolute epsilon - [Catch PR](https://github.com/philsquared/Catch/pull/538)
70- ability to customize the colors in the console output (may also use styles - based on [this](https://github.com/agauniyal/rang))
71- implement breaking into the debugger under linux - see [here](https://github.com/philsquared/Catch/pull/585) and [here](https://github.com/scottt/debugbreak)
72- better testing of the library
73 - unit test the String class
74 - should unit test internals - currently even if a bug is caught by different output it's very difficult to track the reason
75 - should test stuff that should not compile
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]76 - https://github.com/ldionne/dyno/blob/master/cmake/CompileFailTest.cmake
77 - see slide 38 here - https://github.com/boostcon/cppnow_presentations_2017/blob/master/05-19-2017_friday/effective_cmake__daniel_pfeifer__cppnow_05-19-2017.pdf
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]78 - should test crash handling
79 - should test more config options
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]80 - don't cheat for maxing out code coverage (see [coverage_maxout.cpp](../../examples/coverage_maxout.cpp)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]81 - should test C++11 stuff - perhaps inspect the CMAKE_CXX_FLAGS for -std=c++11 on the CI and add more targets/tests
82 - test tricky stuff like expressions with commas in asserts
83
84### For 2.0:
85
86- remove C++98 support
87 - remove the config identifiers for C++11 features
88 - use variadic templates where appropriate
89 - update type lists to C++11
90 - update traits - use declval, etc.
91 - move initialization of fields from initializer lists to class bodies
92 - update static code analysis - less warning suppressing
93
94### For 3.0:
95
96- use modules - use ```std::string``` and whatever else comes from the standard - no more hand rolled traits and classes
97- minimize the use of the preprocessor
98
99### Things that are being considered but not part of the roadmap yet:
100
101- FakeIt mocking integration - like [catch](https://github.com/eranpeer/FakeIt/tree/master/config/catch)
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]102- look into https://github.com/cpp-testing/GUnit - https://www.youtube.com/watch?v=NVrZjT5lW5o
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]103- consider the following 2 properties for the MSVC static code analyzer: EnableCppCoreCheck, EnableExperimentalCppCoreCheck
104- rpm package? like this: https://github.com/vietjtnguyen/argagg/blob/master/packaging/rpm/argagg.spec
105- get the current test case/section path - https://github.com/philsquared/Catch/issues/522
106- when no assertion is encountered in a test case it should fail
107- failure reporting should print out previous SECTIONs for data-driven testing - as requested [here](https://github.com/philsquared/Catch/issues/734)
108- ```Bitwise()``` class that has overloaded operators for comparison - to be used to check objects bitwise against each other
109- detect floating point exceptions
110- checkpoint/passpoint - like in [boost test](http://www.boost.org/doc/libs/1_63_0/libs/test/doc/html/boost_test/test_output/test_tools_support_for_logging/checkpoints.html) (also make all assert/subcase/logging macros to act as passpoints and print the last one on crashes or exceptions)
111- queries for the current test case - name (and probably decorators)
112- thread safety - asserts/subcases/captures should be safe to be used by multiple threads simultaneously
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]113- support for running tests in parallel in multiple threads
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]114- death tests - as in [google test](https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.html#death-tests)
115- config options
116 - test case name uniqueness - reject the ones with identical names
117- command line options
118 - ability to specify ASC/DESC for the order option
119 - global timeout option (per test or per entire session?)
120 - command line error handling/reporting
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]121 - option to not print context info when the --success option is used
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]122 - ability for the user to extend the command line - as requested [here](https://github.com/philsquared/Catch/issues/622)
123 - option to list files in which there are test cases who match the current filters
124 - option for filters to switch from "match any" to "match all" mode
125 - option to list test suites and test cases in a tree view
126 - add a "wait key" option - as requested [here](https://github.com/philsquared/Catch/issues/477#issuecomment-256417686)
127- decorators for test cases and test suites- like in boost test
128 - depends_on
129 - precondition
130 - fixture
131 - label (tag) - with the ability to have multiple labels (tags) for a test case and also the ability to list them
132 - run X times (should also multiply with (or just override) the global test run times)
133 - throw an exception when incompatible decorators are given in the same list of decorators - like may_fail and should_fail
134- setup / teardown support
135 - global setup / teardown - can be currently achieved by providing a custom main function
136 - per test suite (block? only? and not all blocks of the same test suite?)
137 - as decorators
138 - see how it's done in boost test - with the fixture decorator
139 - perhaps for fixtures in addition to the constructor / destructor - since throwing in the destructor might terminate the program
140 - or just ignore all of this this - it would require globals or classes and inheritance - and we already have subcases
141- doctest in a GUI environment? with no console? APIs for attaching a console? querying if there is one? [investigate...](https://github.com/philsquared/Catch/blob/master/docs/configuration.html#stdout)
142- runtime performance
143 - startup - the set holding all registered tests should use a specialized allocator to minimize program startup time
144 - failing - optimize createStream/freeStream to reuse a pool of ostringstream objects
145- ability to provide a temp folder that is cleared between each test case
146- make the _MESSAGE assert macros work with variadic arguments - and maybe write the ones for binary/unary/fast asserts as well
147- move from operator "<<" to "<=" for capturing the left operand when decomposing binary expressions with templates
148- think about silencing warnings about unused variables when DOCTEST_CONFIG_DISABLE is used - see commit 6b61e8aa3818c5ea100cedc1bb48a60ea10df6e8 or issue #61
149- think about optionally using ```<typeinfo>``` and libcxxabi for demangling so users don't have to use ```TYPE_TO_STRING()```
150- handle more complex expressions - ```CHECK(foo() == 1 || bar() == 2);```
151- think about using a string view of some sorts
152- benchmark against google test and boost test
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]153
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]154### Things that are very unlikely to enter the roadmap:
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]155
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]156- think about removing the binary asserts (leaving only the fast binary asserts) because normal asserts + no try/catch in asserts are almost the same
157- move the "react()" part (the one that throws for REQUIRE asserts - or for when "abort-after=<int>" is reached) to a function call in the while() part of the asserts
158- stop using underscores for the begining of identifiers - the anonymous variables - against the standard...
159- templated fixture test cases
160- test with missed warning flags for GCC
161 - https://github.com/Barro/compiler-warnings
162 - http://stackoverflow.com/a/34971392/3162383
163- utf8 / unicode ???
164 - https://github.com/philsquared/Catch/pull/903
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]165- handle ```wchar``` strings???
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]166- hierarchical test suites - using a stack for the pushed ones
167- ability to specify the width of the terminal in terms of characters (for example 60 - less than 80 - the default)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]168- ability to re-run only newly compiled tests based on time stamps using ```__DATE__``` and ```__TIME__``` - stored in some file
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]169- add underscores to all preprocessor identifiers not intended for use by the user
170- put everything from the ```detail``` namespace also in a nested anonymous namespace to make them with internal linkage
171- ability to put everything from doctest into an anonymous namespace - to allow the use of multiple different versions of **doctest** within the same binary (executable/dll) - like the [**stb**](https://github.com/nothings/stb) libraries can
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]172
173---------------
174
175[Home](readme.html#reference)
176
177
178</xmp>
179<script src="strapdown.js/strapdown.js"></script>
180</html>