Gitiles
Code Review Sign In
gerrit.cesnet.cz / github / onqtam / doctest / 8ee354591f81f0159a2cb9b2795fae9999844df5 / . / doc / html_generated / faq.html
blob: 0dcef437319a48d78baf39684cd433a9507cf54e [file] [log] [blame]
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]1<!DOCTYPE html>
2<html>
3<title>faq</title>
4<xmp theme="united" style="display:none;">
5
6## FAQ
7
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]8- [**How is doctest different from Catch?**](#how-is-doctest-different-from-catch)
onqtam50146342019-08-12 22:29:59 +0300[diff] [blame]9- [**How is doctest different from Google Test?**](#how-is-doctest-different-from-google-test)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]10- [**How to get the best compile-time performance with the framework?**](#how-to-get-the-best-compile-time-performance-with-the-framework)
11- [**Is doctest thread-aware?**](#is-doctest-thread-aware)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]12- [**Is mocking supported?**](#is-mocking-supported)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]13- [**Why are my tests in a static library not getting registered?**](#why-are-my-tests-in-a-static-library-not-getting-registered)
14- [**Why is comparing C strings (```char*```) actually comparing pointers?**](#why-is-comparing-c-strings-char-actually-comparing-pointers)
15- [**How to write tests in header-only libraries?**](#how-to-write-tests-in-header-only-libraries)
16- [**Does the framework use exceptions?**](#does-the-framework-use-exceptions)
17- [**Why do I get compiler errors in STL headers when including the doctest header?**](#why-do-i-get-compiler-errors-in-stl-headers-when-including-the-doctest-header)
18- [**Can different versions of the framework be used within the same binary (executable/dll)?**](#can-different-versions-of-the-framework-be-used-within-the-same-binary-executabledll)
19- [**Why is doctest using macros?**](#why-is-doctest-using-macros)
onqtamad21bfc2021-12-10 23:40:00 +0200[diff] [blame]20- [**How to use with multiple files?**](#how-to-use-with-multiple-files)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]21
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]22### How is **doctest** different from Catch?
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]23
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]24Pros of **doctest**:
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]25
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]26- **doctest** is [**thread-safe**](faq.html#is-doctest-thread-aware)
onqtam2d97b852018-11-30 18:06:17 +0200[diff] [blame]27- asserts can be used [**outside of a testing context**](assertions.html#using-asserts-out-of-a-testing-context)
onqtamd52859a2019-03-02 18:28:14 +0200[diff] [blame]28- including the **doctest** header is [**over 20 times lighter**](benchmarks.html#cost-of-including-the-header) on compile times than that of [**Catch**](https://github.com/catchorg/Catch2)
29- the asserts in **doctest** can be [**many times lighter**](benchmarks.html#cost-of-an-assertion-macro) on compile times than those of [**Catch**](https://github.com/catchorg/Catch2)
30- **doctest** executes tests [**many times faster**](benchmarks.html#runtime-benchmarks) than [**Catch**](https://github.com/catchorg/Catch2)
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]31- everything testing-related can be removed from the binary by defining the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) identifier
32- doesn't drag any headers when included (except for in the translation unit where the library gets implemented)
onqtam4aff18c2017-05-17 04:10:03 +0300[diff] [blame]33- 0 warnings even on the [**most aggressive**](../../scripts/cmake/common.cmake#L84) warning levels for MSVC/GCC/Clang
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]34- per commit tested with 180+ builds on [**much more compilers**](features.html#extremely-portable) - and through valgrind/sanitizers/analyzers
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]35- test cases can be written in headers - the framework will still register the tests only once - no duplicates
onqtam2d97b852018-11-30 18:06:17 +0200[diff] [blame]36- binaries (exe/dll) can use the test runner of another binary - so tests end up in a single registry - [**example**](../../examples/executable_dll_and_plugin/)
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]37
onqtamd52859a2019-03-02 18:28:14 +0200[diff] [blame]38Aside from everything mentioned so far doctest has some [**features**](features.html#other-features) (like [**test suites**](testcases.html#test-suites) and [**decorators**](testcases.html#decorators)) which [**Catch**](https://github.com/catchorg/Catch2) doesn't.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]39
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]40Missing stuff:
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]41
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]42- matchers and generators
onqtam50146342019-08-12 22:29:59 +0300[diff] [blame]43- micro benchmarking support - nonius is used in [**Catch**](https://github.com/catchorg/Catch2)
onqtamcdb3c402019-12-17 00:17:12 +0200[diff] [blame]44- other small stuff such as tags - can be easily emulated/migrated from - see below
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]45
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]46But these things (and more!) are planned in the [**roadmap**](roadmap.html)!
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]47
onqtamd52859a2019-03-02 18:28:14 +0200[diff] [blame]48**doctest** can be thought of as a very polished, light, stable and clean subset (or reimplementation) of [**Catch**](https://github.com/catchorg/Catch2) but this might change in the future as more features are added.
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]49
onqtamd52859a2019-03-02 18:28:14 +0200[diff] [blame]50Also checkout [this table](https://github.com/martinmoene/catch-lest-other-comparison) that compares **doctest** / [**Catch**](https://github.com/catchorg/Catch2) / [**lest**](https://github.com/martinmoene/lest).
onqtam6a5da422017-09-04 17:56:08 +0300[diff] [blame]51
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]52A quick and easy way to migrate most of your Catch tests to doctest is to change the ```TEST_CASE``` (if using tags) and ```SECTION``` macros as follows:
53
54```
55#include "path/to/doctest.h"
56
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]57#define SECTION(name) DOCTEST_SUBCASE(name)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]58
onqtam72ca33b2018-12-05 17:34:33 +0200[diff] [blame]59// only if tags are used: will concatenate them to the test name string literal
60#undef TEST_CASE
61#define TEST_CASE(name, tags) DOCTEST_TEST_CASE(tags " " name)
62
63// catch exposes this by default outside of its namespace
64using doctest::Approx;
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]65```
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]66
onqtam50146342019-08-12 22:29:59 +0300[diff] [blame]67### How is **doctest** different from Google Test?
68
69Here are a couple of differences:
70
onqtam77af2002019-09-22 21:17:20 +0300[diff] [blame]71- the main one is that only doctest from the C++ frameworks is usable next to your production code (speed of compilation, ability to remove the tests from the binary, ability to execute tests/code/both, ability to have tests in multiple shared objects and still a single registry for all of them)
72- doctest is a single header - Google Test has to be built as a separate static library and linked against.
onqtam8ee35452021-12-15 15:42:40 +0200[diff] [blame^]73- doctest has the concept of [**Subcases**](https://github.com/doctest/doctest/blob/master/doc/markdown/tutorial.html#test-cases-and-subcases) which is a much cleaner way to share setup and teardown code between tests compared to fixtures and class inheritance - Google Test is quite verbose!
onqtam77af2002019-09-22 21:17:20 +0300[diff] [blame]74- doctest compiles faster and probably runs faster (although the runtime becomes an issue only when you have millions of asserts)
75- doctest asserts are thread-safe even on Windows (Google Test uses pthreads so thread-safe asserts are available only on UNIX)
76- doctest overall has a simpler API
onqtam50146342019-08-12 22:29:59 +0300[diff] [blame]77
78but there are also some areas in which doctest is lacking:
79
onqtam77af2002019-09-22 21:17:20 +0300[diff] [blame]80- value-parameterized tests
81- death tests (where you check if calling a certain function doesn’t simply throw but if it crashes the process)
82- doctest has some integration with mocking libraries but Google Test works perfectly with Google Mock (although doctest should in theory work with it as well)
onqtam50146342019-08-12 22:29:59 +0300[diff] [blame]83
onqtam77af2002019-09-22 21:17:20 +0300[diff] [blame]84The areas where doctest is behind are planned for improvement in the future. There are many other smaller differences - it would be impractical to cover them all.
onqtam50146342019-08-12 22:29:59 +0300[diff] [blame]85
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]86### How to get the best compile-time performance with the framework?
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]87
warmsocks33cc7f92019-09-12 01:24:04 -0700[diff] [blame]88The [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](configuration.html#doctest_config_super_fast_asserts) config option yields the [**fastest possible**](benchmarks.html#cost-of-an-assertion-macro) compile times (up to 31-91%). Also the expression-decomposing template machinery can be skipped by using the [**binary**](assertions.html#binary-and-unary-asserts) asserts.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]89
onqtam72ca33b2018-12-05 17:34:33 +0200[diff] [blame]90There are only 2 tiny drawbacks of using this config option:
onqtamb6c69672016-09-21 15:41:52 +0300[diff] [blame]91
onqtam72ca33b2018-12-05 17:34:33 +0200[diff] [blame]92- there is no ```try/catch``` block in each assert so if an expression is thrown the whole test case ends (but is still caught and reported).
93- when an assert fails and a debugger is present - the framework will break inside a doctest function so the user will have to go 1 level up in the callstack to see where the actual assert is in the source code.
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]94
onqtam72ca33b2018-12-05 17:34:33 +0200[diff] [blame]95These 2 things can be considered negligible and totally worth it if you are dealing mainly with expressions unlikely to throw exceptions and all the tests usually pass (you don't need to navigate often to a failing assert with a debugger attached).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]96
97### Is doctest thread-aware?
98
onqtamad21bfc2021-12-10 23:40:00 +0200[diff] [blame]99Most macros/functionality is safe to use in a multithreaded context: [**assertion**](assertions.html) and [**logging**](logging.html) macros can be safely used from multiple threads spawned from a single test case. This however does not mean that multiple test cases can be ran in parallel - test cases are still ran serially. [**Subcases**](tutorial.html#test-cases-and-subcases) should also be used only from the test runner thread and all threads spawned in a subcase ought to be joined before the end of that subcase and no new subcases should be entered while other threads with doctest assertions in them are still running - not following these instructions will lead to crashes (example in [**here**](../../examples/all_features/concurrency.cpp)). Also note that logged context in one thread will not be used/printed when asserts from another thread fail - logged context is thread-local.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]100
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]101There is also an option to run a [**range**](commandline.html) of tests from an executable - so tests can be ran in parallel by invoking the process multiple times with different ranges - see [**the example python script**](../../examples/range_based_execution.py).
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]102
103### Is mocking supported?
104
105**doctest** doesn't support mocking but should be easy to integrate with third-party libraries such as:
106
107- [trompeloeil](https://github.com/rollbear/trompeloeil) - integration shown [here](https://github.com/rollbear/trompeloeil/blob/master/docs/CookBook.html#adapt_doctest)
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]108- [FakeIt](https://github.com/eranpeer/FakeIt) - integration might be similar to that of [catch](https://github.com/eranpeer/FakeIt/tree/master/config/catch) but this has not been looked into
109
110by using the [**logging**](logging.html#messages-which-can-optionally-fail-test-cases) macros such as ```ADD_FAIL_AT(file, line, message)```
111
112<!--
113Not sure how to integrate with these:
114https://github.com/dascandy/hippomocks
115https://github.com/tpounds/mockitopp
116-->
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]117
118### Why are my tests in a static library not getting registered?
119
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]120This is a [**common problem among libraries with self-registering code**](https://groups.google.com/forum/#!msg/catch-forum/FV0Qo62DvgY/jxEO6c9_q3kJ) and it affects all modern compilers on all platforms.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]121
onqtamec103be2016-10-09 13:46:25 +0300[diff] [blame]122The problem is that when a static library is being linked to a binary (executable or dll) - only object files from the static library that define a symbol being required from the binary will get pulled in (this is a linker/dependency optimization).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]123
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]124A way to solve this in CMake is to use object libraries instead of static libraries - like this:
125
126```cmake
127add_library(with_tests OBJECT src_1.cpp src_2.cpp src_3.cpp ...)
128
129add_library(dll SHARED $<TARGET_OBJECTS:with_tests> dll_src_1.cpp ...)
130add_executable(exe $<TARGET_OBJECTS:with_tests> exe_src_1.cpp ...)
131```
132
133Thanks to [pthom](https://github.com/pthom) for suggesting this.
134
135As an alternative I have created a CMake function that forces every object file from a static library to be linked into a binary target - it is called [**```doctest_force_link_static_lib_in_target()```**](../../examples/exe_with_static_libs/doctest_force_link_static_lib_in_target.cmake). It is unintrusive - no source file gets changed - everything is done with compiler flags per source files. An example project using it can be found [**here**](../../examples/exe_with_static_libs) - the commented part of the CMakeLists.txt file.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]136
137It doesn't work in 2 scenarios:
onqtamb6c69672016-09-21 15:41:52 +0300[diff] [blame]138
onqtam8ee35452021-12-15 15:42:40 +0200[diff] [blame^]139- either the target or the library uses a precompiled header - see [**this**](https://github.com/doctest/doctest/issues/21#issuecomment-247001423) issue for details
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]140- either the target or the library is an imported target (pre-built) and not built within the current cmake tree
141
onqtamb8220c52017-05-16 00:21:15 +0300[diff] [blame]142You can also checkout this repository for a different solution: [**pthom/doctest_registerlibrary**](https://github.com/pthom/doctest_registerlibrary).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]143
onqtam8ee35452021-12-15 15:42:40 +0200[diff] [blame^]144A compiler-specific solution for MSVC is to use the [```/OPT:NOREF```](https://msdn.microsoft.com/en-us/library/bxwfs976.aspx) linker flag (thanks to [lectem](https://github.com/Lectem) for [reporting](https://github.com/doctest/doctest/issues/106) it!). Another option is to look at [```/wholearchive```](https://docs.microsoft.com/en-us/cpp/build/reference/wholearchive-include-all-library-object-files?view=vs-2019) for MSVC.
onqtam8cec9172018-02-06 23:34:14 +0200[diff] [blame]145
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]146### Why is comparing C strings (```char*```) actually comparing pointers?
147
148**doctest** by default treats ```char*``` as normal pointers. Using the [**```DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING```**](configuration.html#doctest_config_treat_char_star_as_string) changes that.
149
150### How to write tests in header-only libraries?
151
152There are 2 options:
153
Michael Letterled33d1e82021-11-04 09:01:07 -0400[diff] [blame]154- just include the doctest header in your headers and write the tests - the doctest header should be shipped with your headers and the user will have to implement the doctest runner in one of their source files.
155- don't include the doctest header and guard your test cases with ```#ifdef DOCTEST_LIBRARY_INCLUDED``` and ```#endif``` - that way your tests will be compiled and registered if the user includes the doctest header before your headers (and they will also have to implement the test runner somewhere).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]156
Michael Letterled33d1e82021-11-04 09:01:07 -0400[diff] [blame]157Also note that it would be a good idea to add a tag in your test case names (like this: ```TEST_CASE("[the_lib] testing foo")```) so the user can easily filter them out with ```--test-case-exclude=*the_lib*``` if they wish to.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]158
159### Does the framework use exceptions?
160
onqtame34600e2016-11-15 02:16:56 +0200[diff] [blame]161Yes - but they can be disabled - see the [**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](configuration.html#doctest_config_no_exceptions) config identifier.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]162
163### Why do I get compiler errors in STL headers when including the doctest header?
164
onqtam92dce5b2019-03-23 14:24:47 +0200[diff] [blame]165Try using the [**```DOCTEST_CONFIG_USE_STD_HEADERS```**](configuration.html#doctest_config_use_std_headers) configuration identifier.
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]166
167### Can different versions of the framework be used within the same binary (executable/dll)?
168
169Currently no. Single header libraries like [**stb**](https://github.com/nothings/stb) have this as an option (everything gets declared static - making it with internal linkage) but it isn't very logical for **doctest** - the main point is to write tests in any source file of the project and have the test runner implemented in only one source file.
170
171### Why is doctest using macros?
172
onqtamd52859a2019-03-02 18:28:14 +0200[diff] [blame]173Aren't they evil and not *modern*? - Check out the answer Phil Nash gives to this question [**here**](http://accu.org/index.php/journals/2064) (the creator of [**Catch**](https://github.com/catchorg/Catch2)).
onqtam1435c012016-09-21 15:29:11 +0300[diff] [blame]174
onqtamad21bfc2021-12-10 23:40:00 +0200[diff] [blame]175### How to use with multiple files?
176
177All you need to do is define either [**```DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN```**](configuration.html#doctest_config_implement_with_main) or [**```DOCTEST_CONFIG_IMPLEMENT```**](configuration.html#doctest_config_implement) in only ONE of the source files just before including the doctest header - in all other source files you just include the header and use the framework. The difference between the two is that one of them provides a `main()` entry point - for more info on that please refer to [`The main() entry point`](main.html).
178
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]179---------------
180
181[Home](readme.html#reference)
182
onqtamf8d57192018-08-23 16:02:12 +0300[diff] [blame]183<p align="center"><img src="../../scripts/data/logo/icon_2.svg"></p>
184
onqtam8126b562016-05-27 17:01:15 +0300[diff] [blame]185
186</xmp>
187<script src="strapdown.js/strapdown.js"></script>
188</html>