Gitiles
Code Review Sign In
gerrit.cesnet.cz / github / onqtam / doctest / 2d97b85a31912e9bb69e51c668e17b17dcfbe350 / . / doc / html_generated / faq.html
blob: 285ca4716ef12c12e486f2ed6150eaa0f6b1ad48 [file] [log] [blame]
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]1<!DOCTYPE html>
2<html>
3<title>faq</title>
4<xmp theme="united" style="display:none;">
5
6## FAQ
7
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]8- [**How is doctest different from Catch?**](#how-is-doctest-different-from-catch)
9- [**How to get the best compile-time performance with the framework?**](#how-to-get-the-best-compile-time-performance-with-the-framework)
10- [**Is doctest thread-aware?**](#is-doctest-thread-aware)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]11- [**Is mocking supported?**](#is-mocking-supported)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]12- [**Why are my tests in a static library not getting registered?**](#why-are-my-tests-in-a-static-library-not-getting-registered)
13- [**Why is comparing C strings (```char*```) actually comparing pointers?**](#why-is-comparing-c-strings-char-actually-comparing-pointers)
14- [**How to write tests in header-only libraries?**](#how-to-write-tests-in-header-only-libraries)
15- [**Does the framework use exceptions?**](#does-the-framework-use-exceptions)
16- [**Why do I get compiler errors in STL headers when including the doctest header?**](#why-do-i-get-compiler-errors-in-stl-headers-when-including-the-doctest-header)
17- [**Can different versions of the framework be used within the same binary (executable/dll)?**](#can-different-versions-of-the-framework-be-used-within-the-same-binary-executabledll)
18- [**Why is doctest using macros?**](#why-is-doctest-using-macros)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]19
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]20### How is **doctest** different from Catch?
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]21
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]22Pros of **doctest**:
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]23
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]24- **doctest** is [**thread-safe**](faq.html#is-doctest-thread-aware)
onqtam2d97b852018-11-30 18:06:17 +0200[diff] [blame^]25- asserts can be used [**outside of a testing context**](assertions.html#using-asserts-out-of-a-testing-context)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]26- including the **doctest** header is [**over 20 times lighter**](benchmarks.html#cost-of-including-the-header) on compile times than that of [**Catch**](https://github.com/philsquared/Catch)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]27- the asserts in **doctest** can be [**many times lighter**](benchmarks.html#cost-of-an-assertion-macro) on compile times than those of [**Catch**](https://github.com/philsquared/Catch)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]28- **doctest** executes tests [**many times faster**](benchmarks.html#runtime-benchmarks) than [**Catch**](https://github.com/philsquared/Catch)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]29- everything testing-related can be removed from the binary by defining the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) identifier
30- doesn't drag any headers when included (except for in the translation unit where the library gets implemented)
onqtam4aff18c2017-05-17 04:10:03 +0300[diff] [blame]31- 0 warnings even on the [**most aggressive**](../../scripts/cmake/common.cmake#L84) warning levels for MSVC/GCC/Clang
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]32- per commit tested with 180+ builds on [**much more compilers**](features.html#extremely-portable) - and through valgrind/sanitizers/analyzers
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]33- test cases can be written in headers - the framework will still register the tests only once - no duplicates
onqtam2d97b852018-11-30 18:06:17 +0200[diff] [blame^]34- binaries (exe/dll) can use the test runner of another binary - so tests end up in a single registry - [**example**](../../examples/executable_dll_and_plugin/)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]35
onqtam2d97b852018-11-30 18:06:17 +0200[diff] [blame^]36Aside from everything mentioned so far doctest has some [**features**](features.html#other-features) (like [**test suites**](testcases.html#test-suites) and [**decorators**](testcases.html#decorators)) which [**Catch**](https://github.com/philsquared/Catch) doesn't.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]37
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]38Missing stuff:
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]39
onqtam9639cee2018-10-24 17:46:32 +0300[diff] [blame]40- xml reporter
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]41- matchers and generators
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]42- other small stuff
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]43
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]44But these things (and more!) are planned in the [**roadmap**](roadmap.html)!
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]45
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]46**doctest** can be thought of as a very polished, light, stable and clean subset (or reimplementation) of [**Catch**](https://github.com/philsquared/Catch) but this might change in the future as more features are added.
47
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]48Also checkout [this table](https://github.com/martinmoene/catch-lest-other-comparison) that compares **doctest** / [**Catch**](https://github.com/philsquared/Catch) / [**lest**](https://github.com/martinmoene/lest).
49
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]50A quick and easy way to migrate most of your Catch tests to doctest is to change the ```TEST_CASE``` (if using tags) and ```SECTION``` macros as follows:
51
52```
53#include "path/to/doctest.h"
54
55#undef TEST_CASE
56#define TEST_CASE(name, tags) DOCTEST_TEST_CASE(tags " " name) // will concatenate the tags and test name string literals to one
57#define SECTION(name) DOCTEST_SUBCASE(name)
58using doctest::Approx; // catch exposes this by default outside of its namespace
59
60```
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]61
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]62### How to get the best compile-time performance with the framework?
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]63
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]64Using the [**fast**](assertions.html#fast-asserts) asserts in combination with [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](configuration.html#doctest_config_super_fast_asserts) yelds the [**fastest**](benchmarks.html#cost-of-an-assertion-macro) compile times.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]65
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]66There are only 2 drawbacks of this approach:
onqtamb6c69672016-09-21 15:41:52 +0300[diff] [blame]67
onqtam4aff18c2017-05-17 04:10:03 +0300[diff] [blame]68- using fast asserts (60-90% [**faster**](benchmarks.html#cost-of-an-assertion-macro) than ```CHECK(a==b)```) means that there is no ```try/catch``` block in each assert so if an expression throws the whole test case ends.
69- defining the [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](configuration.html#doctest_config_super_fast_asserts) config identifier will result in even [**faster**](benchmarks.html#cost-of-an-assertion-macro) fast asserts (50-80%) at the cost of only one thing: when an assert fails and a debugger is present - the framework will break inside a doctest function so the user will have to go 1 level up in the callstack to see where the actual assert is in the source code.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]70
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]71These 2 things can be considered negligible if you are dealing mainly with arithmetic (expressions are unlikely to throw exceptions) and all the tests usually pass (you don't need to often navigate to a failing assert with a debugger attached)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]72
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]73If you want better aliases for the asserts instead of the long ones you could use [**```DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES```**](configuration.html#doctest_config_no_short_macro_names) and then define your aliases like this: ```#define CHECK_EQ DOCTEST_FAST_CHECK_EQ``` (example in [**here**](../../examples/all_features/alternative_macros.cpp) and [**here**](../../examples/all_features/doctest_proxy.h)).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]74
75### Is doctest thread-aware?
76
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]77Most macros/functionality is safe to use in a multithreaded context: [**assertion**](assertions.html) and [**logging**](logging.html) macros can be safely used from multiple threads spawned from a single test case. This however does not mean that multiple test cases can be ran in parallel - test cases are still ran serially. [**Subcases**](tutorial.html#test-cases-and-subcases) should also be used only from the test runner thread - not following these instructions will lead to crashes (example in [**here**](../../examples/all_features/concurrency.cpp)). Also note that logged context in one thread will not be used/printed when asserts from another thread fail - logged context is thread-local.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]78
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]79There is also an option to run a [**range**](commandline.html) of tests from an executable - so tests can be ran in parallel by invoking the process multiple times with different ranges - see [**the example python script**](../../examples/range_based_execution.py).
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]80
81### Is mocking supported?
82
83**doctest** doesn't support mocking but should be easy to integrate with third-party libraries such as:
84
85- [trompeloeil](https://github.com/rollbear/trompeloeil) - integration shown [here](https://github.com/rollbear/trompeloeil/blob/master/docs/CookBook.html#adapt_doctest)
86- [googlemock](https://github.com/google/googletest/tree/master/googlemock) - for integration check [this](https://github.com/google/googletest/blob/master/googlemock/docs/ForDummies.html#using-google-mock-with-any-testing-framework)
87- [FakeIt](https://github.com/eranpeer/FakeIt) - integration might be similar to that of [catch](https://github.com/eranpeer/FakeIt/tree/master/config/catch) but this has not been looked into
88
89by using the [**logging**](logging.html#messages-which-can-optionally-fail-test-cases) macros such as ```ADD_FAIL_AT(file, line, message)```
90
91<!--
92Not sure how to integrate with these:
93https://github.com/dascandy/hippomocks
94https://github.com/tpounds/mockitopp
95-->
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]96
97### Why are my tests in a static library not getting registered?
98
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]99This is a [**common problem among libraries with self-registering code**](https://groups.google.com/forum/#!msg/catch-forum/FV0Qo62DvgY/jxEO6c9_q3kJ) and it affects all modern compilers on all platforms.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]100
onqtamec103be2016-10-09 13:46:25 +0300[diff] [blame]101The problem is that when a static library is being linked to a binary (executable or dll) - only object files from the static library that define a symbol being required from the binary will get pulled in (this is a linker/dependency optimization).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]102
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]103A way to solve this in CMake is to use object libraries instead of static libraries - like this:
104
105```cmake
106add_library(with_tests OBJECT src_1.cpp src_2.cpp src_3.cpp ...)
107
108add_library(dll SHARED $<TARGET_OBJECTS:with_tests> dll_src_1.cpp ...)
109add_executable(exe $<TARGET_OBJECTS:with_tests> exe_src_1.cpp ...)
110```
111
112Thanks to [pthom](https://github.com/pthom) for suggesting this.
113
114As an alternative I have created a CMake function that forces every object file from a static library to be linked into a binary target - it is called [**```doctest_force_link_static_lib_in_target()```**](../../examples/exe_with_static_libs/doctest_force_link_static_lib_in_target.cmake). It is unintrusive - no source file gets changed - everything is done with compiler flags per source files. An example project using it can be found [**here**](../../examples/exe_with_static_libs) - the commented part of the CMakeLists.txt file.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]115
116It doesn't work in 2 scenarios:
onqtamb6c69672016-09-21 15:41:52 +0300[diff] [blame]117
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]118- either the target or the library uses a precompiled header - see [**this**](https://github.com/onqtam/doctest/issues/21#issuecomment-247001423) issue for details
119- either the target or the library is an imported target (pre-built) and not built within the current cmake tree
120
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]121You can also checkout this repository for a different solution: [**pthom/doctest_registerlibrary**](https://github.com/pthom/doctest_registerlibrary).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]122
onqtam8cec9172018-02-06 23:34:14 +0200[diff] [blame]123A compiler-specific solution for MSVC is to use the [```/OPT:NOREF```](https://msdn.microsoft.com/en-us/library/bxwfs976.aspx) linker flag (thanks to [lectem](https://github.com/Lectem) for [reporting](https://github.com/onqtam/doctest/issues/106) it!)
124
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]125### Why is comparing C strings (```char*```) actually comparing pointers?
126
127**doctest** by default treats ```char*``` as normal pointers. Using the [**```DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING```**](configuration.html#doctest_config_treat_char_star_as_string) changes that.
128
129### How to write tests in header-only libraries?
130
131There are 2 options:
132
133- just include the doctest header in your headers and write the tests - the doctest header should be shipped with your headers and the user will have to implement the doctest runner in one of his source files.
134- don't include the doctest header and guard your test cases with ```#ifdef DOCTEST_LIBRARY_INCLUDED``` and ```#endif``` - that way your tests will be compiled and registered if the user includes the doctest header before your headers (and he will also have to implement the test runner somewhere).
135
136Also note that it would be a good idea to add a tag in your test case names (like this: ```TEST_CASE("[the_lib] testing foo")```) so the user can easily filter them out with ```--test-case-exclude=*the_lib*``` if he wishes to.
137
138### Does the framework use exceptions?
139
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]140Yes - but they can be disabled - see the [**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](configuration.html#doctest_config_no_exceptions) config identifier.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]141
142### Why do I get compiler errors in STL headers when including the doctest header?
143
144Try using the [**```DOCTEST_CONFIG_USE_IOSFWD```**](configuration.html#doctest_config_use_iosfwd) configuration identifier.
145
146### Can different versions of the framework be used within the same binary (executable/dll)?
147
148Currently no. Single header libraries like [**stb**](https://github.com/nothings/stb) have this as an option (everything gets declared static - making it with internal linkage) but it isn't very logical for **doctest** - the main point is to write tests in any source file of the project and have the test runner implemented in only one source file.
149
150### Why is doctest using macros?
151
152Aren't they evil and not *modern*? - Check out the answer Phil Nash gives to this question [**here**](http://accu.org/index.php/journals/2064) (the creator of [**Catch**](https://github.com/philsquared/Catch)).
153
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]154---------------
155
156[Home](readme.html#reference)
157
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]158<p align="center"><img src="../../scripts/data/logo/icon_2.svg"></p>
159
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]160
161</xmp>
162<script src="strapdown.js/strapdown.js"></script>
163</html>