Gitiles
Code Review Sign In
gerrit.cesnet.cz / github / onqtam / doctest / 7cd2fabc778b9e7c968acfcd0f26a48e449c3f97 / . / doc / html_generated / features.html
blob: 27eaf4244b1ab202f973a77cb6338f33f01c2df5 [file] [log] [blame]
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]1<!DOCTYPE html>
2<html>
3<title>features</title>
4<xmp theme="united" style="display:none;">
5
6## Features and design goals
7
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]8There are many C++ testing frameworks - [Catch](https://github.com/philsquared/Catch), [Boost.Test](http://www.boost.org/doc/libs/1_64_0/libs/test/doc/html/index.html), [UnitTest++](https://github.com/unittest-cpp/unittest-cpp), [cpputest](https://github.com/cpputest/cpputest), [googletest](https://github.com/google/googletest) and many [other](https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B).
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]9
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]10What makes **doctest** different is that it is ultra light on compile times (by [**orders of magnitude**](benchmarks.html#benchmarks)) and is unintrusive.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]11
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]12## Unintrusive (transparent):
13
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]14- everything testing-related can be removed from the binary executable by defining the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) identifier
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]15- very small and easy to integrate - single header
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]16- **Extremely** low footprint on compile times - [**around 25ms**](benchmarks.html#cost-of-including-the-header) of compile time overhead for including the header in a file
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]17- The [**fastest possible**](benchmarks.html#cost-of-an-assertion-macro) assertion macros - 50k asserts can compile for under 30 seconds (even under 10 sec)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]18- doesn't drag any headers when included (except for in the translation unit where the library gets implemented)
19- everything is in the ```doctest``` namespace (and the implementation details are in a nested ```detail``` namespace)
20- all macros have prefixes - some by default have unprefixed versions as well but that is optional - see [**configuration**](configuration.html)
21- 0 warnings even with the most aggresive flags (on all tested compilers!!!)
22 - ```-Weverything -pedantic``` for **clang**
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]23 - ```-Wall -Wextra -pedantic``` and **>> over 35 <<** other warnings **not** covered by these flags for **GCC**!!! - see [**here**](../../scripts/common.cmake#L77)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]24 - ```/W4``` for **MSVC** (```/Wall``` is too much there - even their own headers produce **thousands** of warnings with that option)
25- doesn't error on unrecognized [**command line**](commandline.html) options and supports prefixes for interop with client command line parsing
26- can set options [**procedurally**](main.html) and not deal with passing ```argc```/```argv``` from the command line
27- doesn't leave warnings disabled after itself
28
29## Extremely portable:
30
31- Standards compliant **C++98** code - should work with any **C++98** compiler
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]32- tested with **GCC**: **4.4**, **4.5**, **4.6**, **4.7**, **4.8**, **4.9**, **5**, **6**
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]33- tested with **Clang**: **3.4**, **3.5**, **3.6**, **3.7**, **3.8**, **3.9**
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]34- tested with **MSVC**: **2008**, **2010**, **2012**, **2013**, **2015**, **2017**
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]35- per-commit tested on **travis** and **appveyor** CI services
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]36 - warnings as errors even on the most aggressive warning levels - see [**here**](../../scripts/common.cmake#L71)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]37 - statically analyzed - [**Cppcheck**](http://cppcheck.sourceforge.net/) / [**Clang-Tidy**](http://oclint.org/) / [**Coverity Scan**](https://scan.coverity.com/) / [**OCLint**](https://scan.coverity.com/) / [**Visual Studio Analyzer**](https://docs.microsoft.com/en-us/visualstudio/code-quality/analyzing-c-cpp-code-quality-by-using-code-analysis)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]38 - all tests have their output compared to reference output of a previous known good run
39 - all tests built and ran in **Debug**/**Release** and also in **32**/**64** bit modes
40 - all tests ran through **valgrind** under **Linux**/**OSX**
41 - all tests ran through **address** and **UB** sanitizers under **Linux**/**OSX**
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]42 - tests are ran in more than **300** different configurations on UNIX (Linux + OSX) on **travis** CI
43 - tests are ran in a total of **24** different configurations on Windows on **appveyor** CI
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]44
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]45------
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]46
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]47This allows the framework to be used in more ways than any other - tests can be written directly in the production code!
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]48
49- This makes the barrier for writing tests **much lower** - you don't have to: **1.** make a separate source file **2.** include a bunch of stuff in it **3.** add it to the build system and **4.** add it to source control - You can just write the tests for a class or a piece of functionality at the bottom of its source file - or even header file!
50- Tests in the production code can be thought of as documentation or up-to-date comments - showing how an API is used
51- Testing internals that are not exposed through the public API and headers becomes easier!
52- [**Test-driven development**](https://en.wikipedia.org/wiki/Test-driven_development) in C++ has never been easier!
53
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]54The library can be used like any other if you don't like the idea of mixing production code and tests - see features below.
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]55
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]56## Other features:
57
58- really easy to get started - it's just 1 header file - see the [**tutorial**](tutorial.html)
59- **very** light, unintrusive and portable - see the sections above - and also the [**benchmarks**](benchmarks.html)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]60- offers a way to remove **everything** testing-related from the binary with the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) macro
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]61- tests are registered automatically - no need to add them to a collection manually
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]62- [**Subcases**](tutorial.html#test-cases-and-subcases) - an intuitive way to share common setup and teardown code for test cases (alternative to [**test fixtures**](testcases.html#test-fixtures) which are also supported)
63- [**templated test cases**](parameterized-tests.html#templated-test-cases---parameterized-by-type) - parameterized by type
64- supports [**logging macros**](logging.html) for capturing local variables and strings - as a message for when an assert fails - with lazy stringification and no allocations when possible!
65- crash handling support - uses signals for UNIX and SEH for Windows
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]66- output from all compilers on all platforms is the same - byte by byte
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]67- binaries (exe/dll) can use the test runner of another binary - so tests end up in a single registry - [**example**](../../examples/executable_dll_and_plugin/)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]68- supports [**BDD style**](testcases.html) tests
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]69- one core [**assertion macro**](assertions.html) for comparisons - standard C++ operators are used for the comparison (less than, equal, greater than...) - yet the full expression is decomposed and left and right values of the expression are logged
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]70- assertion macros for [**exceptions**](assertions.html) - if something should or shouldn't throw
71- floating point comparison support - see the [**```Approx()```**](assertions.html#floating-point-comparisons) helper
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]72- powerful mechanism for [**stringification**](stringification.html) of user types - including [**exceptions**](stringification.html#translating-exceptions)!
73- tests can be grouped in [**test suites**](testcases.html#test-suites)
74- test case [**decorators**](testcases.html#decorators) such as ```description``` / ```skip``` / ```may_fail``` / ```should_fail``` / ```expected_failures``` / ```timeout```
75- can be used without exceptions and rtti - checkout [**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](configuration.html#doctest_config_no_exceptions)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]76- powerful [**command line**](commandline.html) with lots of options
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]77- can report the duration of test cases
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]78- tests can be [**filtered**](commandline.html) based on their name/file/test suite using wildcards
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]79- can [**filter**](commandline.html) subcases using wildcards and by specifying the nesting levels for which those filters should work
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]80- failures can (optionally) break into the debugger on Windows and Mac
81- integration with the output window of Visual Studio for failing tests
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]82- a ```main()``` can be provided when implementing the library with the [**```DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN```**](main.html#doctest_config_implement_with_main) identifier
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]83- can write tests in headers - they will still be registered only once in the executable/shared object
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]84- [**range-based**](commandline.html) execution of tests - see the [**example python script**](../../examples/range_based_execution.py)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]85- colored output in the console
86- controlling the order of test execution
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]87- different ```doctest::Context```s can be created and ran many times within a single execution of the program
88- ability to query if code is currently being ran in a test - ```doctest::isRunningInTest()```
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]89
90There is a list of planned features which are all important and big - see the [**roadmap**](roadmap.html).
91
92---------------
93
94[Home](readme.html#reference)
95
96
97</xmp>
98<script src="strapdown.js/strapdown.js"></script>
99</html>