version 1.2 almost out!
diff --git a/README.md b/README.md
index 3392267..abd09f8 100644
--- a/README.md
+++ b/README.md
@@ -81,7 +81,7 @@
[](https://raw.githubusercontent.com/onqtam/doctest/master/doctest/doctest.h)
[](https://bestpractices.coreinfrastructure.org/projects/503)
[](https://gitter.im/onqtam/doctest?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
-[](http://melpon.org/wandbox/permlink/RL0lY1YxOlGF7CYN)
+[](https://wandbox.org/permlink/SJrRfwy3aaLwjndl)
<!--
[](https://isocpp.org/)
[](https://github.com/onqtam/doctest/blob/master/doc/markdown/readme.md#reference)
diff --git a/doc/html_generated/assertions.html b/doc/html_generated/assertions.html
index 6ca9bbe..11eaf8d 100644
--- a/doc/html_generated/assertions.html
+++ b/doc/html_generated/assertions.html
@@ -7,7 +7,7 @@
Most test frameworks have a large collection of assertion macros to capture all possible conditional forms (```_EQUALS```, ```_NOTEQUALS```, ```_GREATER_THAN``` etc).
-**doctest** is different (but it's like [**Catch**](https://github.com/philsquared/Catch) in this regard). Because it decomposes comparison expressions most of these forms are reduced to one or two that you will use all the time. That said there is a rich set of auxiliary macros as well.
+**doctest** is different (but it's like [**Catch**](https://github.com/philsquared/Catch) in this regard). Because it decomposes comparison expressions most of these forms are reduced to one or two that you will use all the time. That said, there is a rich set of auxiliary macros as well.
There are 3 levels of assert severity for all assertion macros:
@@ -19,6 +19,8 @@
All asserts evaluate the expressions only once and if they fail - the values are [**stringified**](stringification.html) properly.
+Note that the ```REQUIRE``` level of asserts uses exceptions to end the current test case. It might be dangerous to use this level of asserts inside destructors of user-defined classes - if a destructor is called during stack unwinding due to an exception and a ```REQUIRE``` assert fails then the program will terminate. Also since C++11 all destructors are by default ```noexcept(true)``` unless specified otherwise so such an assert will lead to ```std::terminate()``` being called.
+
## Expression decomposing asserts
These are of the form ```CHECK(expression)``` (Same for ```REQUIRE``` and ```WARN```).
@@ -45,6 +47,20 @@
REQUIRE_FALSE(thisReturnsFalse());
```
+Note that these asserts also have a ```_MESSAGE``` form - like ```CHECK_MESSAGE(expression, message)``` which is basically a code block ```{}``` with a scoped [**```INFO()```**](logging.html#info) logging macro together with the ```CHECK``` macro - that way the message will be relevant only to that assert. All the other binary/unary/fast asserts don't have this variation.
+
+Examples:
+
+```
+INFO("this is relevant to all asserts, and here is some var: " << local);
+
+CHECK_MESSAGE(a < b, "relevant only to this assert " << other_local << "more text!");
+
+CHECK(b < c); // here only the first INFO() will be relevant
+```
+
+For more information about the ```INFO()``` macro and logging with the streaming ```operator<<``` visit the [logging page](logging.html).
+
## Binary and unary asserts
These asserts don't use templates to decompose the comparison expressions for the left and right parts.
@@ -93,31 +109,39 @@
Expects that an exception of the _specified type_ is thrown during evaluation of the expression.
+Note that the exception type is used as-is - no const or reference is added - so users are expected to specify them since the standard practice for exceptions in C++ is ```Throw by value, catch by reference```.
+
+```
+CHECK_THROWS_AS(func(), const std::exception&); // note the reference and the const
+```
+
* ```<LEVEL>_NOTHROW(expression)```
Expects that no exception is thrown during evaluation of the expression.
+Note that these asserts also have a ```_MESSAGE``` form - like ```CHECK_THROWS_MESSAGE(expression, message)``` - these work identically to the ```_MESSAGE``` form of the normal macros (```CHECK_MESSAGE(a < b, "this shouldn't fail")```) described earlier.
+
## Floating point comparisons
When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations.
-**doctest** provides a way to perform tolerant comparisons of floating point values through use of a wrapper class called ```doctest::Approx```. ```doctest::Approx``` can be used on either side of a comparison expression. It overloads the comparisons operators to take a tolerance into account. Here's a simple example:
+**doctest** provides a way to perform tolerant comparisons of floating point values through the use of a wrapper class called ```doctest::Approx```. ```doctest::Approx``` can be used on either side of a comparison expression. It overloads the comparisons operators to take a relative tolerance into account. Here's a simple example:
```
REQUIRE(performComputation() == doctest::Approx(2.1));
```
-By default a small epsilon value is used that covers many simple cases of rounding errors. When this is insufficient the epsilon value (the amount within which a difference either way is ignored) can be specified by calling the ```epsilon()``` method on the ```doctest::Approx``` instance. e.g.:
+By default a small epsilon value (relative - in percentages) is used that covers many simple cases of rounding errors. When this is insufficient the epsilon value (the amount within which a difference either way is ignored) can be specified by calling the ```epsilon()``` method on the ```doctest::Approx``` instance. e.g.:
```
-REQUIRE(22.0/7 == doctest::Approx(3.141).epsilon(0.01));
+REQUIRE(22.0/7 == doctest::Approx(3.141).epsilon(0.01)); // allow for a 1% error
```
When dealing with very large or very small numbers it can be useful to specify a scale, which can be achieved by calling the ```scale()``` method on the ```doctest::Approx``` instance.
--------
-- Check out the [**example**](../../examples/assertion_macros/main.cpp) which shows many of these macros
+- Check out the [**example**](../../examples/all_features/assertion_macros.cpp) which shows many of these macros
- Do not wrap assertion macros in ```try```/```catch``` - the REQUIRE macros throw exceptions to end the test case execution!
---------------
diff --git a/doc/html_generated/benchmarks.html b/doc/html_generated/benchmarks.html
index f82a623..82f28ca 100644
--- a/doc/html_generated/benchmarks.html
+++ b/doc/html_generated/benchmarks.html
@@ -3,28 +3,31 @@
<title>benchmarks</title>
<xmp theme="united" style="display:none;">
-# Compile time benchmarks
+# Benchmarks
-The benchmarks are done with [**this**](../../scripts/bench/bench.py) script using CMake. There are 2 benchmarking scenarios:
+The benchmarks are done with [**this**](../../scripts/bench/bench.py) script using CMake. There are 3 benchmarking scenarios:
- [the cost of including the header](#cost-of-including-the-header)
- [the cost of an assertion macro](#cost-of-an-assertion-macro)
+- [runtime speed of lots of asserts](#runtime-benchmarks)
Compilers used:
-- WINDOWS: Microsoft Visual Studio Community 2015 - Version 14.0.25431.01 Update 3
-- WINDOWS: gcc 6.2.0 (x86_64-posix-seh-rev0, Built by MinGW-W64 project)
-- LINUX: gcc 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu2)
-- LINUX: clang 3.6.2-1 (tags/RELEASE_362/final) (based on LLVM 3.6.2) Ubuntu - Target: x86_64-pc-linux-gnu
+- WINDOWS: Microsoft Visual Studio Community 2017 - Version 15.1.26403.7
+- WINDOWS: gcc 7.1.0 (x86_64-posix-seh-rev0, Built by MinGW-W64 project)
+- LINUX: gcc 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2)
+- LINUX: clang 4.0.0-1 (tags/RELEASE_400/rc1) Target: x86_64-pc-linux-gnu
Environment used (Intel i7 3770k, 16g RAM):
- Windows 7 - on an SSD
-- Ubuntu 15.10 in a VirtualBox VM - on a HDD
+- Ubuntu 17.04 in a VirtualBox VM - on a HDD
-**doctest** version: 1.1.0 (released on 2016.09.21)
+**doctest** version: 1.2.0 (released on 2017.05.16)
-[**Catch**](https://github.com/philsquared/Catch) version: 1.5.6 (released on 2016.06.09)
+[**Catch**](https://github.com/philsquared/Catch) version: 1.9.3 (released on 2017.04.25)
+
+# Compile time benchmarks
## Cost of including the header
@@ -35,51 +38,54 @@
- **baseline** - how much time the source files need for a single threaded build with ```msbuild```/```make```
- **+ implement** - only in ```main.cpp``` the header is included with a ```#define``` before it so the test runner gets implemented:
- ```
+```
#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
- ```
+```
- **+ header everywhere** - the framework header is also included in all the other source files
- **+ disabled** - **doctest** specific - only this framework can remove everything related to it from the binary
| doctest | baseline | + implement | + header everywhere | + disabled |
|---------------------|----------|-------------|---------------------|------------|
-| MSVC Debug | 5.9 | 7.1 | 8.3 | 7.0 |
-| MSVC Release | 5.4 | 6.9 | 8.7 | 6.5 |
-| MinGW GCC Debug | 9.4 | 11.7 | 14.4 | 11.1 |
-| MinGW GCC Release | 9.6 | 12.3 | 14.9 | 11.4 |
-| Linux GCC Debug | 6.3 | 7.1 | 10.2 | 7.4 |
-| Linux GCC Release | 6.5 | 8.4 | 10.8 | 7.8 |
-| Linux Clang Debug | 6.9 | 7.6 | 10.6 | 8.2 |
-| Linux Clang Release | 7.2 | 8.4 | 11.4 | 8.4 |
+| MSVC Debug | 7.17 | 8.50 | 12.50 | 9.42 |
+| MSVC Release | 6.72 | 8.47 | 12.51 | 8.97 |
+| MinGW GCC Debug | 10.65 | 13.56 | 18.59 | 13.34 |
+| MinGW GCC Release | 10.81 | 14.53 | 19.21 | 14.11 |
+| Linux GCC Debug | 4.98 | 6.55 | 11.22 | 6.94 |
+| Linux GCC Release | 4.49 | 7.27 | 11.92 | 7.87 |
+| Linux Clang Debug | 8.52 | 9.42 | 15.25 | 11.14 |
+| Linux Clang Release | 9.01 | 11.85 | 18.18 | 11.75 |
| Catch | baseline | + implement | + header everywhere |
|---------------------|----------|-------------|---------------------|
-| MSVC Debug | 5.9 | 8.5 | 102 |
-| MSVC Release | 5.4 | 10.3 | 96 |
-| MinGW GCC Debug | 9.4 | 24.5 | 125 |
-| MinGW GCC Release | 9.6 | 18.4 | 113 |
-| Linux GCC Debug | 6.3 | 10.4 | 59 |
-| Linux GCC Release | 6.5 | 14.1 | 64 |
-| Linux Clang Debug | 6.9 | 9.8 | 64 |
-| Linux Clang Release | 7.2 | 12.8 | 67 |
+| MSVC Debug | 7.17 | 10.60 | 128.02 |
+| MSVC Release | 6.72 | 10.90 | 119.25 |
+| MinGW GCC Debug | 10.65 | 26.12 | 127.00 |
+| MinGW GCC Release | 10.81 | 19.95 | 114.15 |
+| Linux GCC Debug | 4.98 | 9.37 | 105.66 |
+| Linux GCC Release | 4.49 | 13.04 | 105.57 |
+| Linux Clang Debug | 8.52 | 11.33 | 70.57 |
+| Linux Clang Release | 9.01 | 16.59 | 75.85 |
+
+<img src="../../scripts/data/benchmarks/header.png" width="430" align="right">
+<img src="../../scripts/data/benchmarks/implement.png" width="430">
### Conclusion
#### doctest
-- instantiating the test runner in one source file costs ~1.5 seconds ```implement - baseline```
-- the inclusion of ```doctest.h``` in one source file costs below 9ms ```(header_everywhere - implement) / 200```
-- including the library everywhere but everything disabled costs less than 2 seconds ```disabled - baseline``` for 200 files
+- instantiating the test runner in one source file costs ~1.5-3 seconds ```implement - baseline```
+- the inclusion of ```doctest.h``` in one source file costs between 20ms - 30ms ```(header_everywhere - implement) / 200```
+- including the library everywhere but everything disabled costs less than 3 seconds ```disabled - baseline``` for 200 files
#### [Catch](https://github.com/philsquared/Catch)
-- instantiating the test runner in one source file costs ~5 second ```implement - baseline```
-- the inclusion of ```catch.hpp``` in one source file costs around 430ms ```(header_everywhere - implement) / 200```
+- instantiating the test runner in one source file costs ~4-8 seconds ```implement - baseline```
+- the inclusion of ```catch.hpp``` in one source file costs between 300ms - 575ms ```(header_everywhere - implement) / 200```
----------
-So if ```doctest.h``` costs 8ms and ```catch.hpp``` costs 430ms on MSVC - then the **doctest** header is >> **54** << times lighter!
+So if ```doctest.h``` costs 20ms and ```catch.hpp``` costs 560ms on MSVC - then the **doctest** header is >> **28** << times lighter (for MSVC)!
----------
@@ -101,41 +107,101 @@
- **+faster** - will add [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](configuration.html#doctest_config_super_fast_asserts) which speeds up ```FAST_CHECK_EQ(a,b)``` even more
- **+disabled** - all test case and assert macros will be disabled with [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable)
+[**Catch**](https://github.com/philsquared/Catch) specific:
+
+- **+faster** - will add [**```CATCH_CONFIG_FAST_COMPILE```**](https://github.com/philsquared/Catch/blob/master/docs/configuration.html#catch_config_fast_compile) which speeds up the compilation of the normal asserts ```CHECK(a==b)```
+
| doctest | baseline | ```CHECK(a==b)``` | ```CHECK_EQ(a,b)``` | ```FAST_CHECK_EQ(a,b)``` | +faster | +disabled |
|---------------------|----------|-------------------|---------------------|--------------------------|---------|-----------|
-| MSVC Debug | 2.5 | 21 | 16.2 | 6.7 | 4.4 | 2.2 |
-| MSVC Release | 2.6 | 64 | 55 | 63 | 5.3 | 1.8 |
-| MinGW GCC Debug | 3.2 | 77 | 52 | 29.5 | 12.2 | 1.6 |
-| MinGW GCC Release | 3.9 | 425 | 295 | 81 | 18.6 | 1.9 |
-| Linux GCC Debug | 1.3 | 72 | 48 | 20.3 | 9.5 | 0.9 |
-| Linux GCC Release | 2.3 | 339 | 210 | 42 | 18.3 | 1.3 |
-| Linux Clang Debug | 1.3 | 70 | 46 | 18.8 | 7.0 | 0.9 |
-| Linux Clang Release | 1.8 | 205 | 136 | 30 | 10.8 | 1.1 |
+| MSVC Debug | 3.22 | 24.17 | 18.54 | 8.32 | 5.75 | 2.34 |
+| MSVC Release | 3.63 | 41.10 | 23.20 | 10.89 | 6.87 | 2.31 |
+| MinGW GCC Debug | 4.09 | 91.98 | 60.19 | 25.51 | 12.61 | 1.82 |
+| MinGW GCC Release | 4.74 | 240.58 | 156.79 | 50.16 | 19.72 | 2.53 |
+| Linux GCC Debug | 2.06 | 81.32 | 52.14 | 18.07 | 10.15 | 1.16 |
+| Linux GCC Release | 3.28 | 207.21 | 126.89 | 33.17 | 19.92 | 2.03 |
+| Linux Clang Debug | 1.75 | 79.41 | 51.20 | 17.78 | 7.65 | 1.20 |
+| Linux Clang Release | 3.73 | 140.79 | 82.61 | 21.19 | 12.64 | 1.46 |
-And here are [**Catch**](https://github.com/philsquared/Catch) and **doctest 1.0** which only have normal ```CHECK(a==b)``` asserts:
+And here is [**Catch**](https://github.com/philsquared/Catch) which only has normal ```CHECK(a==b)``` asserts:
-| Catch | baseline | ```CHECK(a==b)``` | | doctest 1.0 | ```CHECK(a==b)``` |
-|---------------------|----------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------------|
-| MSVC Debug | 8.4 | 34 | | MSVC Debug | 58 |
-| MSVC Release | 9.7 | 77 | | MSVC Release | 367 |
-| MinGW GCC Debug | 20.5 | 115 | | MinGW GCC Debug | 202 |
-| MinGW GCC Release | 15.1 | 496 | | MinGW GCC Release | 1257 |
-| Linux GCC Debug | 7.3 | 101 | | Linux GCC Debug | 204 |
-| Linux GCC Release | 10.3 | 435 | | Linux GCC Release | 1090 |
-| Linux Clang Debug | 6.0 | 91 | | Linux Clang Debug | 167 |
-| Linux Clang Release | 8.5 | 159 | | Linux Clang Release | 494 |
+| Catch | baseline | ```CHECK(a==b)``` | +faster |
+|---------------------|----------|-------------------|---------|
+| MSVC Debug | 9.94 | 40.14 | 36.66 |
+| MSVC Release | 10.66 | 231.60 | 81.90 |
+| MinGW GCC Debug | 21.20 | 129.26 | 110.95 |
+| MinGW GCC Release | 14.59 | 297.04 | 207.75 |
+| Linux GCC Debug | 10.05 | 115.53 | 98.84 |
+| Linux GCC Release | 13.29 | 294.26 | 218.37 |
+| Linux Clang Debug | 6.38 | 103.06 | 85.02 |
+| Linux Clang Release | 11.15 | 195.62 | 156.04 |
+
+<img src="../../scripts/data/benchmarks/asserts.png">
### Conclusion
-**doctest 1.1**:
+**doctest**:
-- improves the compile time of it's ```CHECK(a==b)``` asserts by roughly 3 times compared to **doctest** 1.0 (released 2016.05.22)
-- is around 25% faster than [**Catch**](https://github.com/philsquared/Catch) when using expression decomposing ```CHECK(a==b)``` asserts
-- adds asserts of the form ```CHECK_EQ(a,b)``` with no expression decomposition - around 20% faster than ```CHECK(a==b)```
-- adds fast asserts like ```FAST_CHECK_EQ(a,b)``` with no ```try/catch``` blocks - around 30-70% faster than ```CHECK_EQ(a,b)```
-- adds the [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](configuration.html#doctest_config_super_fast_asserts) identifier which makes the fast assertions even faster by another 35-80%
+- is around 30% faster than [**Catch**](https://github.com/philsquared/Catch) when using normal expression decomposing ```CHECK(a==b)``` asserts
+- asserts of the form ```CHECK_EQ(a,b)``` with no expression decomposition - around 25%-45% faster than ```CHECK(a==b)```
+- fast asserts like ```FAST_CHECK_EQ(a,b)``` with no ```try/catch``` blocks - around 60-80% faster than ```CHECK_EQ(a,b)```
+- the [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](configuration.html#doctest_config_super_fast_asserts) identifier which makes the fast assertions even faster by another 50-80%
- using the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) identifier the assertions just disappear as if they were never written
+[**Catch**](https://github.com/philsquared/Catch):
+
+- using [**```CATCH_CONFIG_FAST_COMPILE```**](https://github.com/philsquared/Catch/blob/master/docs/configuration.html#catch_config_fast_compile) results in 10%-40% faster build times for asserts.
+
+## Runtime benchmarks
+
+The runtime benchmarks consist of a single test case with a loop of 10 million iterations performing the task - a single normal assert (using expression decomposition) or the assert + the logging of the loop iterator ```i```:
+
+```
+for(int i = 0; i < 10000000; ++i)
+ CHECK(i == i);
+```
+
+or
+
+```
+for(int i = 0; i < 10000000; ++i) {
+ INFO(i);
+ CHECK(i == i);
+}
+```
+
+Note that the assert always passes - the goal should be to optimize for the common case - lots of passing test cases and a few that maybe fail.
+
+| doctest | assert | + info |
+|---------------------|---------|---------|
+| MSVC Debug | 5.29 | 14.32 |
+| MSVC Release | 0.77 | 1.62 |
+| MinGW GCC Debug | 2.25 | 4.71 |
+| MinGW GCC Release | 0.39 | 0.90 |
+| Linux GCC Debug | 2.84 | 5.13 |
+| Linux GCC Release | 0.30 | 0.69 |
+| Linux Clang Debug | 2.47 | 5.02 |
+| Linux Clang Release | 0.41 | 0.75 |
+
+| Catch | assert | + info |
+|---------------------|---------|---------|
+| MSVC Debug | 365.37 | 621.78 |
+| MSVC Release | 7.04 | 18.64 |
+| MinGW GCC Debug | 9.22 | 21.89 |
+| MinGW GCC Release | 7.29 | 13.95 |
+| Linux GCC Debug | 11.17 | 24.79 |
+| Linux GCC Release | 6.45 | 12.68 |
+| Linux Clang Debug | 10.40 | 22.64 |
+| Linux Clang Release | 5.81 | 13.83 |
+
+<img src="../../scripts/data/benchmarks/runtime_info.png" width="430" align="right">
+<img src="../../scripts/data/benchmarks/runtime_assert.png" width="430">
+
+### Conclusion
+
+**doctest** is significantly faster - between 4 and 40 times.
+
+In these particular cases **doctest** makes 0 allocations when the assert doesn't fail - it uses lazy stringification (meaning it stringifies the expression or the logged loop counter only if it has to) and a small-buffer optimized string class to achieve these results.
+
----------
If you want a benchmark that is not synthetic - check out [**this blog post**](http://baptiste-wicht.com/posts/2016/09/blazing-fast-unit-test-compilation-with-doctest-11.html) of [**Baptiste Wicht**](https://github.com/wichtounet) who tested the compile times of the asserts in the 1.1 release with his [**Expression Templates Library**](https://github.com/wichtounet/etl)!
diff --git a/doc/html_generated/build-systems.html b/doc/html_generated/build-systems.html
new file mode 100644
index 0000000..2d10c3d
--- /dev/null
+++ b/doc/html_generated/build-systems.html
@@ -0,0 +1,86 @@
+<!DOCTYPE html>
+<html>
+<title>build-systems</title>
+<xmp theme="united" style="display:none;">
+
+## Build systems
+
+The latest released version of doctest can be obtained from here: https://raw.githubusercontent.com/onqtam/doctest/master/doctest/doctest.h
+
+You can substitute ```master``` with ```dev``` or a tag like ```1.2.0``` for a specific version in the URL above.
+
+### CMake
+
+- **doctest** is easiest to use as a single file inside your own repository. Then the following minimal example will work:
+
+```cmake
+cmake_minimum_required(VERSION 3.0)
+project(cmake_test)
+
+# Prepare doctest for other targets to use
+add_library(doctest INTERFACE)
+target_include_directories(doctest INTERFACE path/to/doctest)
+
+# Make test executable
+add_executable(tests main.cpp)
+target_link_libraries(tests doctest)
+```
+
+- You can also use the following CMake snippet to automatically fetch the entire **doctest** repository from github and configure it as an external project:
+
+```cmake
+include(ExternalProject)
+find_package(Git REQUIRED)
+
+ExternalProject_Add(
+ doctest
+ PREFIX ${CMAKE_BINARY_DIR}/doctest
+ GIT_REPOSITORY https://github.com/onqtam/doctest.git
+ TIMEOUT 10
+ UPDATE_COMMAND ${GIT_EXECUTABLE} pull
+ CONFIGURE_COMMAND ""
+ BUILD_COMMAND ""
+ INSTALL_COMMAND ""
+ LOG_DOWNLOAD ON
+)
+
+# Expose required variable (DOCTEST_INCLUDE_DIR) to parent scope
+ExternalProject_Get_Property(doctest source_dir)
+set(DOCTEST_INCLUDE_DIR ${source_dir}/doctest CACHE INTERNAL "Path to include folder for doctest")
+```
+
+And later you'll be able to use the doctest include directory like this:
+
+```cmake
+# add it globally
+include_directories(${DOCTEST_INCLUDE_DIR})
+
+# or per target
+target_include_directories(my_target PUBLIC ${DOCTEST_INCLUDE_DIR})
+```
+
+- If you have the entire doctest repository available (as a submodule or just as files) you could also include it in your CMake build by using ```add_subdirectory(path/to/doctest)``` and then you could use it like this:
+
+```cmake
+add_executable(my_tests src_1.cpp src_2.cpp ...)
+target_link_libraries(my_tests doctest)
+```
+
+- The ```CMakeLists.txt``` file of the doctest repository has ```install()``` commands so you could also use doctest as a package.
+
+### Package managers
+
+**doctest** is available through the following package managers:
+
+- vcpkg
+- hunter
+- conan
+
+---
+
+[Home](readme.html#reference)
+
+
+</xmp>
+<script src="strapdown.js/strapdown.js"></script>
+</html>
diff --git a/doc/html_generated/commandline.html b/doc/html_generated/commandline.html
index 8ffefdf..e31f852 100644
--- a/doc/html_generated/commandline.html
+++ b/doc/html_generated/commandline.html
@@ -7,7 +7,7 @@
**doctest** works quite nicely without any command line options at all - but for more control a bunch are available.
-**Query flags** - after the result is printed the program quits without executing any test cases (and if the framework is integrated into a client codebase which [**supplies it's own ```main()``` entry point**](main.html) - the program should check the result of ```doctest::Context::shouldExit()``` after calling ```doctest::Context::run()``` and should exit - this is left up to the user).
+**Query flags** - after the result is printed the program quits without executing any test cases (and if the framework is integrated into a client codebase which [**supplies it's own ```main()``` entry point**](main.html) - the program should check the result of ```shouldExit()``` method after calling ```run()``` on a ```doctest::Context``` object and should exit - this is left up to the user).
**Int/String options** - they require a value after the ```=``` sign - without spaces! For example: ```--order-by=rand```.
@@ -32,15 +32,18 @@
| ```-sfe``` ```--source-file-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on the file in which test cases are written |
| ```-ts``` ```--test-suite=<filters>``` | Same as ```--test-case=<filters>``` but filters based on the test suite in which test cases are in |
| ```-tse``` ```--test-suite-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on the test suite in which test cases are in |
+| ```-sc``` ```--subcase=<filters>``` | Same as ```--test-case=<filters>``` but filters subcases based on their names |
+| ```-sce``` ```--subcase-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on subcase names |
| ```-ob``` ```--order-by=<string>``` | Test cases will be sorted before being executed either by **the file in which they are** / **the test suite they are in** / **their name** / **random**. The possible values of ```<string>``` are ```file```/```suite```/```name```/```rand```. The default is ```file``` |
| ```-rs``` ```--rand-seed=<int>``` | The seed for random ordering |
-| ```-f``` ```--first=<int>``` | The **first** test case to execute which passes the current filters - for range-based execution - see [**the example**](../../examples/range_based_execution/) (the **run.py** script) |
-| ```-l``` ```--last=<int>``` | The **last** test case to execute which passes the current filters - for range-based execution - see [**the example**](../../examples/range_based_execution/) (the **run.py** script) |
-| ```-aa``` ```--abort-after=<int>``` | The testing framework will stop executing test cases/assertions after this many failed assertions. The default is 0 which means don't stop at all |
+| ```-f``` ```--first=<int>``` | The **first** test case to execute which passes the current filters - for range-based execution - see [**the example python script**](../../examples/range_based_execution.py) |
+| ```-l``` ```--last=<int>``` | The **last** test case to execute which passes the current filters - for range-based execution - see [**the example python script**](../../examples/range_based_execution.py) |
+| ```-aa``` ```--abort-after=<int>``` | The testing framework will stop executing test cases/assertions after this many failed assertions. The default is 0 which means don't stop at all. Note that the framework uses an exception to stop the current test case regardless of the level of the assert (```CHECK```/```REQUIRE```) - so be careful with asserts in destructors... |
+| ```-scfl``` ```--subcase-filter-levels=<int>``` | Apply subcase filters only for the first ```<int>``` levels of nested subcases and just run the ones nested deeper. Default is a very high number which means *filter any subcase* |
| **Bool Options** | <hr> |
| ```-s``` ```--success=<bool>``` | To include successful assertions in the output |
| ```-cs``` ```--case-sensitive=<bool>``` | Filters being treated as case sensitive |
-| ```-e``` ```--exit=<bool>``` | Exits after the tests finish - this is meaningful only when the client has [**provided the ```main()``` entry point**](main.html) - the program should check ```doctest::Context::shouldExit()``` after calling ```doctest::Context::run()``` and should exit - this is left up to the user. The idea is to be able to execute just the tests in a client program and to not continue with it's execution |
+| ```-e``` ```--exit=<bool>``` | Exits after the tests finish - this is meaningful only when the client has [**provided the ```main()``` entry point**](main.html) - the program should check the ```shouldExit()``` method after calling ```run()``` on a ```doctest::Context``` object and should exit - this is left up to the user. The idea is to be able to execute just the tests in a client program and to not continue with it's execution |
| ```-nt``` ```--no-throw=<bool>``` | Skips [**exceptions-related assertion**](assertions.html#exceptions) checks |
| ```-ne``` ```--no-exitcode=<bool>``` | Always returns a successful exit code - even if a test case has failed |
| ```-nr``` ```--no-run=<bool>``` | Skips all runtime **doctest** operations (except the test registering which happens before the program enters ```main()```). This is useful if the testing framework is integrated into a client codebase which has [**provided the ```main()``` entry point**](main.html) and the user wants to skip running the tests and just use the program |
@@ -48,7 +51,9 @@
| ```-nc``` ```--no-colors=<bool>``` | Disables colors in the output |
| ```-fc``` ```--force-colors=<bool>``` | Forces the use of colors even when a tty cannot be detected |
| ```-nb``` ```--no-breaks=<bool>``` | Disables breakpoints in debuggers when an assertion fails |
+| ```-ns``` ```--no-skip=<bool>``` | Don't skip test cases marked as skip with a decorator |
| ```-npf``` ```--no-path-filenames=<bool>``` | Paths are removed from the output when a filename is printed - useful if you want the same output from the testing framework on different environments |
+| ```-nln``` ```--no-line-numbers=<bool>``` | Line numbers are replaced with ```0``` in the output when a source location is printed - useful if you want the same output from the testing framework even when test positions change within a source file |
| | |
All the flags/options also come with a prefixed version (with ```--dt-``` at the front) - for example ```--version``` can be used also with ```--dt-version``` or ```--dt-v```.
@@ -57,7 +62,63 @@
This is done for easy interoperability with client command line option handling when the testing framework is integrated within a client codebase - all **doctest** related flags/options can be prefixed so there are no clashes and so that the user can exclude everything starting with ```--dt-``` from their option parsing.
-If there isn't an option to exclude everything starting with ```--dt-``` then the [**removing_doctest_options**](../../examples/removing_doctest_options/) example might help.
+If there isn't an option to exclude everything starting with ```--dt-``` then the ```dt_removed``` helper class might help to filter them out:
+
+```
+#define DOCTEST_CONFIG_NO_UNPREFIXED_OPTIONS
+#define DOCTEST_CONFIG_IMPLEMENT
+#include "doctest.h"
+
+class dt_removed {
+ std::vector<const char*> vec;
+public:
+ dt_removed(const char** argv_in) {
+ for(; *argv_in; ++argv_in)
+ if(strncmp(*argv_in, "--dt-", strlen("--dt-")) != 0)
+ vec.push_back(*argv_in);
+ vec.push_back(NULL);
+ }
+
+ int argc() { return static_cast<int>(vec.size()) - 1; }
+ const char** argv() { return &vec[0]; } // Note: non-const char **:
+};
+
+int program(int argc, const char** argv);
+
+int main(int argc, const char** argv) {
+ doctest::Context context(argc, argv);
+ int test_result = context.run(); // run queries, or run tests unless --no-run
+
+ if(context.shouldExit()) // honor query flags and --exit
+ return test_result;
+
+ dt_removed args(argv);
+ int app_result = program(args.argc(), args.argv());
+
+ return test_result + app_result; // combine the 2 results
+}
+
+int program(int argc, const char** argv) {
+ printf("Program: %d arguments received:\n", argc - 1);
+ while(*++argv)
+ printf("'%s'\n", *argv);
+ return EXIT_SUCCESS;
+}
+```
+
+When ran like this:
+
+```
+program.exe --dt-test-case=math* --my-option -s --dt-no-breaks
+```
+
+Will output this:
+
+```
+Program: 2 arguments received:
+'--my-option'
+'-s'
+```
---------------
diff --git a/doc/html_generated/configuration.html b/doc/html_generated/configuration.html
index 2bf7532..62f8863 100644
--- a/doc/html_generated/configuration.html
+++ b/doc/html_generated/configuration.html
@@ -9,10 +9,14 @@
The identifiers should be defined before the inclusion of the framework header.
+Defining something ```globally``` means for every source file of the binary (executable / shared object).
+
- [**```DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN```**](#doctest_config_implement_with_main)
- [**```DOCTEST_CONFIG_IMPLEMENT```**](#doctest_config_implement)
- [**```DOCTEST_CONFIG_DISABLE```**](#doctest_config_disable)
+- [**```DOCTEST_CONFIG_IMPLEMENTATION_IN_DLL```**](#doctest_config_implementation_in_dll)
- [**```DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES```**](#doctest_config_no_short_macro_names)
+- [**```DOCTEST_CONFIG_NUM_CAPTURES_ON_STACK```**](#doctest_config_num_captures_on_stack)
- [**```DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING```**](#doctest_config_treat_char_star_as_string)
- [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](#doctest_config_super_fast_asserts)
- [**```DOCTEST_CONFIG_USE_IOSFWD```**](#doctest_config_use_iosfwd)
@@ -25,9 +29,22 @@
- [**```DOCTEST_CONFIG_COLORS_NONE```**](#doctest_config_colors_none)
- [**```DOCTEST_CONFIG_COLORS_WINDOWS```**](#doctest_config_colors_windows)
- [**```DOCTEST_CONFIG_COLORS_ANSI```**](#doctest_config_colors_ansi)
+- [**```DOCTEST_CONFIG_WINDOWS_SEH```**](#doctest_config_windows_seh)
+- [**```DOCTEST_CONFIG_NO_WINDOWS_SEH```**](#doctest_config_no_windows_seh)
+- [**```DOCTEST_CONFIG_POSIX_SIGNALS```**](#doctest_config_posix_signals)
+- [**```DOCTEST_CONFIG_NO_POSIX_SIGNALS```**](#doctest_config_no_posix_signals)
+
+Detection of modern C++ features:
+
+- [**```DOCTEST_CONFIG_WITH_DELETED_FUNCTIONS```**](#doctest_config_with_deleted_functions)
+- [**```DOCTEST_CONFIG_WITH_RVALUE_REFERENCES```**](#doctest_config_with_rvalue_references)
+- [**```DOCTEST_CONFIG_WITH_VARIADIC_MACROS```**](#doctest_config_with_variadic_macros)
- [**```DOCTEST_CONFIG_WITH_NULLPTR```**](#doctest_config_with_nullptr)
- [**```DOCTEST_CONFIG_WITH_LONG_LONG```**](#doctest_config_with_long_long)
- [**```DOCTEST_CONFIG_WITH_STATIC_ASSERT```**](#doctest_config_with_static_assert)
+- [**```DOCTEST_CONFIG_NO_DELETED_FUNCTIONS```**](#doctest_config_no_deleted_functions)
+- [**```DOCTEST_CONFIG_NO_RVALUE_REFERENCES```**](#doctest_config_no_rvalue_references)
+- [**```DOCTEST_CONFIG_NO_VARIADIC_MACROS```**](#doctest_config_no_variadic_macros)
- [**```DOCTEST_CONFIG_NO_NULLPTR```**](#doctest_config_no_nullptr)
- [**```DOCTEST_CONFIG_NO_LONG_LONG```**](#doctest_config_no_long_long)
- [**```DOCTEST_CONFIG_NO_STATIC_ASSERT```**](#doctest_config_no_static_assert)
@@ -55,12 +72,37 @@
This should be defined globally.
+### **```DOCTEST_CONFIG_IMPLEMENTATION_IN_DLL```**
+
+This will affect the public interface of doctest - all necessary forward declarations for writing tests will be turned into imported symbols. That way the test runner doesn't have to be implemented in the binary (executable / shared object) and can be reused from another binary where it is built and exported.
+
+To export the test runner from a binary simply use [**```DOCTEST_CONFIG_IMPLEMENTATION_IN_DLL```**](#doctest_config_implementation_in_dll) together with [**```DOCTEST_CONFIG_IMPLEMENT```**](#doctest_config_implement) (or [**```DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN```**](#doctest_config_implement_with_main) but then the other binaries will have to link to the executable) in whatever source file the test runner gets implemented into. Note that this identifier should not be defined in the other source files of the binary which exports the doctest test runner - or there will be linker conflicts - having the same symbols as both imported and exported within the same binary.
+
+Checkout the [**example**](../../examples/executable_dll_and_plugin/) - it shows how to have the test runner implemented in a dll (and there are even tests in a plugin which is dynamically loaded).
+
+This should be defined globally in binaries that import the symbols.
+
+This should be defined only in the source file where the library is implemented for binaries that export the test runner.
+
### **```DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES```**
-This will remove all macros from **doctest** that don't have the **```DOCTEST_```** prefix - like **```CHECK```**, **```TEST_CASE```** and **```SUBCASE```**. Then only the full macro names will be available - **```DOCTEST_CHECK```**, **```DOCTEST_TEST_CASE```** and **```DOCTEST_SUBCASE```**. The user is free to make his own short versions of these macros - [**example**](../../examples/alternative_macros/).
+This will remove all macros from **doctest** that don't have the **```DOCTEST_```** prefix - like **```CHECK```**, **```TEST_CASE```** and **```SUBCASE```**. Then only the full macro names will be available - **```DOCTEST_CHECK```**, **```DOCTEST_TEST_CASE```** and **```DOCTEST_SUBCASE```**. The user is free to make his own short versions of these macros - [**example**](../../examples/all_features/alternative_macros.cpp).
This can be defined both globally and in specific source files only.
+### **```DOCTEST_CONFIG_NUM_CAPTURES_ON_STACK```**
+
+With this identifier the user may configure the number of captures on the stack by the ```INFO()``` [**logging macros**](logging.html) (read there for more info). The default is ```5``` - which means that for a call like this: ```INFO(var1 << "la la" << var2);``` all 3 logged variables will be captured on the stack (with the ability to hold 2 more - so no heap allocation unless an assert fails later in the same scope) - and a total of ```5 * (sizeof(void*) * 2))``` bytes are used on the stack for captures. A subsequent call to ```INFO()``` will have it's own stack space. Note that ```0``` is an invalid value. Examples:
+
+```
+#define DOCTEST_CONFIG_NUM_CAPTURES_ON_STACK 10
+#include <doctest.h>
+```
+
+or through the command line: ```-DDOCTEST_CONFIG_NUM_CAPTURES_ON_STACK=10```
+
+This should be defined globally.
+
### **```DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING```**
By default ```char*``` is being treated as a pointer. With this option comparing ```char*``` pointers will switch to using ```strcmp()``` for comparisons and when stringified the string will be printed instead of the pointer value.
@@ -89,7 +131,6 @@
- g++/clang ```-Wsign-conversion```
- g++/clang ```-Wsign-compare```
-- g++/clang ```-Wdouble-promotion```
- msvc ```C4389``` 'operator' : signed/unsigned mismatch
- msvc ```C4018``` 'expression' : signed/unsigned mismatch
@@ -108,7 +149,7 @@
This will remove all ```try``` / ```catch``` sections from:
- the [normal asserts](assertions.html#expression-decomposing-asserts)
-- the [binary and unary asserts](assertions.html#binary-and-unary-asserts)
+- the [binary and unary asserts](assertions.html#binary-and-unary-asserts) - making them functionally the same as the [**fast asserts**](assertions.html#fast-asserts) (but not for compile time speed)
so exceptions thrown while evaluating the expression in an assert will terminate the current test case.
@@ -127,13 +168,14 @@
The ```REQUIRE``` family of asserts uses exceptions to terminate the current test case when they fail. An exception is used instead of a simple ```return;``` because asserts can be used not only in a test case but also in functions called by a test case.
+Also some of the [**logging macros**](logging.html#messages-which-can-optionally-fail-test-cases) which act like a ```REQUIRE``` assert (terminating the test case) - like ```FAIL()``` - start to work differently - like a ```FAIL_CHECK()```.
+
[**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](#doctest_config_no_exceptions) implies [**```DOCTEST_CONFIG_NO_TRY_CATCH_IN_ASSERTS```**](#doctest_config_no_try_catch_in_asserts)
If you wish to use asserts that deal with exceptions and only sometimes build without exceptions - check the [**```DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS```**](#doctest_config_no_exceptions_but_with_all_asserts) config option.
This should be defined globally.
-
### **```DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS```**
When building with no exceptions (see [**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](#doctest_config_no_exceptions)) ```REQUIRE``` asserts and the ones about dealing with exceptions are gone.
@@ -153,8 +195,8 @@
template<typename T> struct type_traits { static const bool value = false; };
// unless DOCTEST_CONFIG_ASSERTION_PARAMETERS_BY_VALUE is defined the following assertion
-// will lead to a linker error if type_traits<T>::value isn't defined in a translation unit
-CHECK(type_traits::value == false);
+// will lead to a linker error if type_traits<int>::value isn't defined in a translation unit
+CHECK(type_traits<int>::value == false);
```
This can be defined both globally and in specific source files only.
@@ -177,6 +219,48 @@
This should be defined only in the source file where the library is implemented (it's relevant only there).
+### **```DOCTEST_CONFIG_WINDOWS_SEH```**
+
+This will enable SEH handling on Windows. Currently enabled only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support. The user may choose to enable this explicitly - it is known to work with the MinGW-w64 project.
+
+This should be defined only in the source file where the library is implemented (it's relevant only there).
+
+### **```DOCTEST_CONFIG_NO_WINDOWS_SEH```**
+
+This can be used to disable **```DOCTEST_CONFIG_WINDOWS_SEH```** when it is auto-selected by the library.
+
+This should be defined only in the source file where the library is implemented (it's relevant only there).
+
+### **```DOCTEST_CONFIG_POSIX_SIGNALS```**
+
+This will enable the use of signals under UNIX for handling crashes. On by default.
+
+This should be defined only in the source file where the library is implemented (it's relevant only there).
+
+### **```DOCTEST_CONFIG_NO_POSIX_SIGNALS```**
+
+This can be used to disable **```DOCTEST_CONFIG_POSIX_SIGNALS```** when it is auto-selected by the library.
+
+This should be defined only in the source file where the library is implemented (it's relevant only there).
+
+### **```DOCTEST_CONFIG_WITH_DELETED_FUNCTIONS```**
+
+doctest tries to detect if c++11 deleted functions are available but if it doesn't detect it - the user might define this.
+
+This should be defined globally.
+
+### **```DOCTEST_CONFIG_WITH_RVALUE_REFERENCES```**
+
+doctest tries to detect if c++11 rvalue references are available but if it doesn't detect it - the user might define this.
+
+This should be defined globally.
+
+### **```DOCTEST_CONFIG_WITH_VARIADIC_MACROS```**
+
+doctest tries to detect if c++11 variadic macros are available but if it doesn't detect it - the user might define this.
+
+This can be defined both globally and in specific source files only.
+
### **```DOCTEST_CONFIG_WITH_NULLPTR```**
doctest tries to detect if c++11 ```nullptr``` is available but if it doesn't detect it - the user might define this.
@@ -195,6 +279,24 @@
This should be defined globally.
+### **```DOCTEST_CONFIG_NO_DELETED_FUNCTIONS```**
+
+If doctest detects c++11 deleted functions support as available but the user knows better - this can be defined to disable it.
+
+This should be defined globally.
+
+### **```DOCTEST_CONFIG_NO_RVALUE_REFERENCES```**
+
+If doctest detects c++11 rvalue references support as available but the user knows better - this can be defined to disable it.
+
+This should be defined globally.
+
+### **```DOCTEST_CONFIG_NO_VARIADIC_MACROS```**
+
+If doctest detects c++11 variadic macros support as available but the user knows better - this can be defined to disable it.
+
+This can be defined both globally and in specific source files only.
+
### **```DOCTEST_CONFIG_NO_NULLPTR```**
If doctest detects c++11 ```nullptr``` support as available but the user knows better - this can be defined to disable it.
diff --git a/doc/html_generated/contributing.html b/doc/html_generated/contributing.html
deleted file mode 100644
index d060420..0000000
--- a/doc/html_generated/contributing.html
+++ /dev/null
@@ -1,39 +0,0 @@
-<!DOCTYPE html>
-<html>
-<title>contributing</title>
-<xmp theme="united" style="display:none;">
-
-## Contributing
-
-This library is free, and will stay free but needs your support to sustain its development. There are lots of [**new features**](roadmap.html) and maintenance to do. If you work for a company using **doctest** or have the means to do so, please consider financial support.
-
-[](https://pledgie.com/campaigns/31280)
-[](http://www.patreon.com/onqtam)
-[](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3K423Q6TK48BN)
-
-## Pull requests
-
-Consider opening an issue for a discussion before making a pull request to make sure the contribution goes smoothly.
-
-All pull requests should be made against the ```dev``` branch because the ```master``` is the stable one with the latest release.
-
-If you're going to change something in the library itself - make sure you don't modify ```doctest/doctest.h``` because it's generated from ```doctest/parts/doctest_fwd.h``` and ```doctest/parts/doctest_impl.h``` - they get concatenated by CMake - so make sure you do a CMake build after you modify them so the ```assemble_single_header``` target gets built.
-
-This framework has some design goals which must be kept. Make sure you have read the [**features and design goals**](features.html) page.
-
-If your changes also change the output of the library - you should also update the reference output for the tests or otherwise the CI builds (```travis``` and ```appveyor```) will fail when they compare the latest output to the outdated reference output (which is committed in the repository). To do this run CMake with the ```TEST_MODE``` variable set to ```COLLECT``` (making the new reference output) and then run ```ctest``` and commit the changed ```.txt``` files too. The default ```TEST_MODE``` is ```COMPARE```.
-
-Example: ```cmake -DTEST_MODE=COLLECT path/to/sources && ctest```
-
-Code should be formatted with a recent-enough ```clang-format``` using the config file in the root of the repo (or I will do it...)
-
-Testing with compilers different from GCC/Clang/MSVC (and more platforms) is something the project would benefit from.
-
----------------
-
-[Home](readme.html#reference)
-
-
-</xmp>
-<script src="strapdown.js/strapdown.js"></script>
-</html>
diff --git a/doc/html_generated/faq.html b/doc/html_generated/faq.html
index bd9283b..c4fd2c1 100644
--- a/doc/html_generated/faq.html
+++ b/doc/html_generated/faq.html
@@ -8,6 +8,7 @@
- [**How is doctest different from Catch?**](#how-is-doctest-different-from-catch)
- [**How to get the best compile-time performance with the framework?**](#how-to-get-the-best-compile-time-performance-with-the-framework)
- [**Is doctest thread-aware?**](#is-doctest-thread-aware)
+- [**Is mocking supported?**](#is-mocking-supported)
- [**Why are my tests in a static library not getting registered?**](#why-are-my-tests-in-a-static-library-not-getting-registered)
- [**Why is comparing C strings (```char*```) actually comparing pointers?**](#why-is-comparing-c-strings-char-actually-comparing-pointers)
- [**How to write tests in header-only libraries?**](#how-to-write-tests-in-header-only-libraries)
@@ -15,36 +16,44 @@
- [**Why do I get compiler errors in STL headers when including the doctest header?**](#why-do-i-get-compiler-errors-in-stl-headers-when-including-the-doctest-header)
- [**Can different versions of the framework be used within the same binary (executable/dll)?**](#can-different-versions-of-the-framework-be-used-within-the-same-binary-executabledll)
- [**Why is doctest using macros?**](#why-is-doctest-using-macros)
-- [**How are subcases implemented?**](#how-are-subcases-implemented)
### How is **doctest** different from Catch?
Pros of **doctest**:
-- including the **doctest** header is [**over 50 times lighter**](benchmarks.html#cost-of-including-the-header) on compile times than that of [**Catch**](https://github.com/philsquared/Catch)
+- including the **doctest** header is [**over 20 times lighter**](benchmarks.html#cost-of-including-the-header) on compile times than that of [**Catch**](https://github.com/philsquared/Catch)
- the asserts in **doctest** can be [**many times lighter**](benchmarks.html#cost-of-an-assertion-macro) on compile times than those of [**Catch**](https://github.com/philsquared/Catch)
+- **doctest** executes tests [**many times faster**](benchmarks.html#runtime-benchmarks) than [**Catch**](https://github.com/philsquared/Catch)
- everything testing-related can be removed from the binary by defining the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) identifier
- doesn't drag any headers when included (except for in the translation unit where the library gets implemented)
- 0 warnings even on the [**most aggressive**](../../scripts/common.cmake#L71) warning levels for MSVC/GCC/Clang
-- per commit tested with 220+ builds on [**much more compilers**](features.html#extremely-portable) - and through valgrind/sanitizers
+- per commit tested with 300+ builds on [**much more compilers**](features.html#extremely-portable) - and through valgrind/sanitizers
+- statically analyzed - [**Cppcheck**](http://cppcheck.sourceforge.net/) / [**Clang-Tidy**](http://oclint.org/) / [**Coverity Scan**](https://scan.coverity.com/) / [**OCLint**](https://scan.coverity.com/) / [**Visual Studio Analyzer**](https://docs.microsoft.com/en-us/visualstudio/code-quality/analyzing-c-cpp-code-quality-by-using-code-analysis)
- test cases can be written in headers - the framework will still register the tests only once - no duplicates
-- more actively maintained on GitHub
-- everything is documented (an undocumented feature is as good as a missing feature)
-Aside from everything mentioned so far doctest has some [**small features**](features.html#other-features) which [**Catch**](https://github.com/philsquared/Catch) doesn't but currently nothing big.
+Aside from everything mentioned so far doctest has some [**features**](features.html#other-features) (like [templated test cases](parameterized-tests.html#templated-test-cases---parameterized-by-type)) which [**Catch**](https://github.com/philsquared/Catch) doesn't.
Missing stuff:
-- exception translation - derivatives of ```std::exception``` don't get their ```.what()``` method called
-- no support for tags (the user can add *tags* in the test case names like this: ```TEST_CASE("[myTag] test name")```)
-- a reporter system - to a file, to xml, ability for the user to write their own reporter, etc.
-- measuring how much time a test case executes
-- adding contextual info to asserts with ```INFO```/```CONTEXT```
+- a reporter/listener system - to a file, to xml, ability for the user to write their own reporter, etc.
- matchers and generators
+- other small stuff
-But all these things (and more!) are planned in the [**roadmap**](roadmap.html)!
+But these things (and more!) are planned in the [**roadmap**](roadmap.html)!
-**doctest** can be thought of as a very polished, light, stable and clean subset (or reimplementation) of [**Catch**](https://github.com/philsquared/Catch) but this will change in the future as more features are added.
+**doctest** can be thought of as a very polished, light, stable and clean subset (or reimplementation) of [**Catch**](https://github.com/philsquared/Catch) but this might change in the future as more features are added.
+
+A quick and easy way to migrate most of your Catch tests to doctest is to change the ```TEST_CASE``` (if using tags) and ```SECTION``` macros as follows:
+
+```
+#include "path/to/doctest.h"
+
+#undef TEST_CASE
+#define TEST_CASE(name, tags) DOCTEST_TEST_CASE(tags " " name) // will concatenate the tags and test name string literals to one
+#define SECTION(name) DOCTEST_SUBCASE(name)
+using doctest::Approx; // catch exposes this by default outside of its namespace
+
+```
### How to get the best compile-time performance with the framework?
@@ -57,7 +66,7 @@
These 2 things can be considered negligible if you are dealing mainly with arithmetic (expressions are unlikely to throw exceptions) and all the tests usually pass (you don't need to often navigate to a failing assert with a debugger attached)
-If you want better aliases for the asserts instead of the long ones you could use [**```DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES```**](configuration.html#doctest_config_no_short_macro_names) and then define your aliases like this: ```#define CHECK_EQ DOCTEST_FAST_CHECK_EQ``` (example in [**here**](../../examples/alternative_macros)).
+If you want better aliases for the asserts instead of the long ones you could use [**```DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES```**](configuration.html#doctest_config_no_short_macro_names) and then define your aliases like this: ```#define CHECK_EQ DOCTEST_FAST_CHECK_EQ``` (example in [**here**](../../examples/all_features/alternative_macros.cpp)).
### Is doctest thread-aware?
@@ -65,22 +74,49 @@
For now tests are ran serially and doing asserts in multiple user threads will lead to crashes.
-There is an option to run a [**range**](commandline.html) of tests from an executable - so tests can be ran in parallel with multiple process invocations - see [**the example**](../../examples/range_based_execution/) (the **run.py** script).
+There is an option to run a [**range**](commandline.html) of tests from an executable - so ran in parallel with multiple process invocations - see [**the example python script**](../../examples/range_based_execution.py).
+
+### Is mocking supported?
+
+**doctest** doesn't support mocking but should be easy to integrate with third-party libraries such as:
+
+- [trompeloeil](https://github.com/rollbear/trompeloeil) - integration shown [here](https://github.com/rollbear/trompeloeil/blob/master/docs/CookBook.html#adapt_doctest)
+- [googlemock](https://github.com/google/googletest/tree/master/googlemock) - for integration check [this](https://github.com/google/googletest/blob/master/googlemock/docs/ForDummies.html#using-google-mock-with-any-testing-framework)
+- [FakeIt](https://github.com/eranpeer/FakeIt) - integration might be similar to that of [catch](https://github.com/eranpeer/FakeIt/tree/master/config/catch) but this has not been looked into
+
+by using the [**logging**](logging.html#messages-which-can-optionally-fail-test-cases) macros such as ```ADD_FAIL_AT(file, line, message)```
+
+<!--
+Not sure how to integrate with these:
+https://github.com/dascandy/hippomocks
+https://github.com/tpounds/mockitopp
+-->
### Why are my tests in a static library not getting registered?
-This is a [**common problem**](https://groups.google.com/forum/#!msg/catch-forum/FV0Qo62DvgY/jxEO6c9_q3kJ) and it affects all modern compilers on all platforms.
+This is a [**common problem among libraries with self-registering code**](https://groups.google.com/forum/#!msg/catch-forum/FV0Qo62DvgY/jxEO6c9_q3kJ) and it affects all modern compilers on all platforms.
The problem is that when a static library is being linked to a binary (executable or dll) - only object files from the static library that define a symbol being required from the binary will get pulled in (this is a linker/dependency optimization).
-I have created a CMake function that forces every object file from a static library to be linked into a binary target - it is called [**```doctest_force_link_static_lib_in_target()```**](../../examples/exe_with_static_libs/doctest_force_link_static_lib_in_target.cmake). It is unintrusive - no source file gets changed - everything is done with compiler flags per source files. An example project using it can be found [**here**](../../examples/exe_with_static_libs).
+A way to solve this in CMake is to use object libraries instead of static libraries - like this:
+
+```cmake
+add_library(with_tests OBJECT src_1.cpp src_2.cpp src_3.cpp ...)
+
+add_library(dll SHARED $<TARGET_OBJECTS:with_tests> dll_src_1.cpp ...)
+add_executable(exe $<TARGET_OBJECTS:with_tests> exe_src_1.cpp ...)
+```
+
+Thanks to [pthom](https://github.com/pthom) for suggesting this.
+
+As an alternative I have created a CMake function that forces every object file from a static library to be linked into a binary target - it is called [**```doctest_force_link_static_lib_in_target()```**](../../examples/exe_with_static_libs/doctest_force_link_static_lib_in_target.cmake). It is unintrusive - no source file gets changed - everything is done with compiler flags per source files. An example project using it can be found [**here**](../../examples/exe_with_static_libs) - the commented part of the CMakeLists.txt file.
It doesn't work in 2 scenarios:
- either the target or the library uses a precompiled header - see [**this**](https://github.com/onqtam/doctest/issues/21#issuecomment-247001423) issue for details
- either the target or the library is an imported target (pre-built) and not built within the current cmake tree
-For an alternative you can checkout the [**pthom/doctest_registerlibrary**](https://github.com/pthom/doctest_registerlibrary) repository.
+You can also checkout this repository for a different solution: [**pthom/doctest_registerlibrary**](https://github.com/pthom/doctest_registerlibrary).
### Why is comparing C strings (```char*```) actually comparing pointers?
@@ -111,10 +147,6 @@
Aren't they evil and not *modern*? - Check out the answer Phil Nash gives to this question [**here**](http://accu.org/index.php/journals/2064) (the creator of [**Catch**](https://github.com/philsquared/Catch)).
-### How are subcases implemented?
-
-Look at [**this example**](../../scripts/how_subcases_work/main.cpp) in the repository.
-
---------------
[Home](readme.html#reference)
diff --git a/doc/html_generated/features.html b/doc/html_generated/features.html
index d4e50fd..27eaf42 100644
--- a/doc/html_generated/features.html
+++ b/doc/html_generated/features.html
@@ -5,22 +5,22 @@
## Features and design goals
-**doctest** has been designed from the start to be as **light** and **unintrusive** as possible. These key features should be kept.
+There are many C++ testing frameworks - [Catch](https://github.com/philsquared/Catch), [Boost.Test](http://www.boost.org/doc/libs/1_64_0/libs/test/doc/html/index.html), [UnitTest++](https://github.com/unittest-cpp/unittest-cpp), [cpputest](https://github.com/cpputest/cpputest), [googletest](https://github.com/google/googletest) and many [other](https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B).
-It is modeled after [**Catch**](https://github.com/philsquared/Catch) which is currently the most popular alternative for testing in C++ - check out [**the differences**](faq.html#how-is-doctest-different-from-catch).
+What makes **doctest** different is that it is ultra light on compile times (by [**orders of magnitude**](benchmarks.html#benchmarks)) and is unintrusive.
## Unintrusive (transparent):
- everything testing-related can be removed from the binary executable by defining the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) identifier
- very small and easy to integrate - single header
-- **Extremely** low footprint on compile times - [**below 10ms**](benchmarks.html#cost-of-including-the-header) of compile time overhead for including the header in a file
+- **Extremely** low footprint on compile times - [**around 25ms**](benchmarks.html#cost-of-including-the-header) of compile time overhead for including the header in a file
- The [**fastest possible**](benchmarks.html#cost-of-an-assertion-macro) assertion macros - 50k asserts can compile for under 30 seconds (even under 10 sec)
- doesn't drag any headers when included (except for in the translation unit where the library gets implemented)
- everything is in the ```doctest``` namespace (and the implementation details are in a nested ```detail``` namespace)
- all macros have prefixes - some by default have unprefixed versions as well but that is optional - see [**configuration**](configuration.html)
- 0 warnings even with the most aggresive flags (on all tested compilers!!!)
- ```-Weverything -pedantic``` for **clang**
- - ```-Wall -Wextra -pedantic``` and **>> over 50 <<** other warnings **not** covered by these flags for **GCC**!!! - see [**here**](../../scripts/common.cmake#L77)
+ - ```-Wall -Wextra -pedantic``` and **>> over 35 <<** other warnings **not** covered by these flags for **GCC**!!! - see [**here**](../../scripts/common.cmake#L77)
- ```/W4``` for **MSVC** (```/Wall``` is too much there - even their own headers produce **thousands** of warnings with that option)
- doesn't error on unrecognized [**command line**](commandline.html) options and supports prefixes for interop with client command line parsing
- can set options [**procedurally**](main.html) and not deal with passing ```argc```/```argv``` from the command line
@@ -31,26 +31,27 @@
- Standards compliant **C++98** code - should work with any **C++98** compiler
- tested with **GCC**: **4.4**, **4.5**, **4.6**, **4.7**, **4.8**, **4.9**, **5**, **6**
- tested with **Clang**: **3.4**, **3.5**, **3.6**, **3.7**, **3.8**, **3.9**
-- tested with **MSVC**: **2008**, **2010**, **2012**, **2013**, **2015**
+- tested with **MSVC**: **2008**, **2010**, **2012**, **2013**, **2015**, **2017**
- per-commit tested on **travis** and **appveyor** CI services
- warnings as errors even on the most aggressive warning levels - see [**here**](../../scripts/common.cmake#L71)
+ - statically analyzed - [**Cppcheck**](http://cppcheck.sourceforge.net/) / [**Clang-Tidy**](http://oclint.org/) / [**Coverity Scan**](https://scan.coverity.com/) / [**OCLint**](https://scan.coverity.com/) / [**Visual Studio Analyzer**](https://docs.microsoft.com/en-us/visualstudio/code-quality/analyzing-c-cpp-code-quality-by-using-code-analysis)
- all tests have their output compared to reference output of a previous known good run
- all tests built and ran in **Debug**/**Release** and also in **32**/**64** bit modes
- all tests ran through **valgrind** under **Linux**/**OSX**
- all tests ran through **address** and **UB** sanitizers under **Linux**/**OSX**
- - tests are ran in more than **200** different configurations on UNIX (Linux + OSX) on **travis** CI
- - tests are ran in a total of **20** different configurations on Windows on **appveyor** CI
+ - tests are ran in more than **300** different configurations on UNIX (Linux + OSX) on **travis** CI
+ - tests are ran in a total of **24** different configurations on Windows on **appveyor** CI
----------------
+------
-This allows the library to be used in more ways than any other - tests can be written directly in the production code!
+This allows the framework to be used in more ways than any other - tests can be written directly in the production code!
- This makes the barrier for writing tests **much lower** - you don't have to: **1.** make a separate source file **2.** include a bunch of stuff in it **3.** add it to the build system and **4.** add it to source control - You can just write the tests for a class or a piece of functionality at the bottom of its source file - or even header file!
- Tests in the production code can be thought of as documentation or up-to-date comments - showing how an API is used
- Testing internals that are not exposed through the public API and headers becomes easier!
- [**Test-driven development**](https://en.wikipedia.org/wiki/Test-driven_development) in C++ has never been easier!
-The library can be used like any other if you don't like the idea of mixing production code and tests.
+The library can be used like any other if you don't like the idea of mixing production code and tests - see features below.
## Other features:
@@ -58,23 +59,33 @@
- **very** light, unintrusive and portable - see the sections above - and also the [**benchmarks**](benchmarks.html)
- offers a way to remove **everything** testing-related from the binary with the [**```DOCTEST_CONFIG_DISABLE```**](configuration.html#doctest_config_disable) macro
- tests are registered automatically - no need to add them to a collection manually
-- supports [**subcases**](testcases.html) for easy setup/teardown of tests (also supports the retro [**test fixtures**](testcases.html#) with classes)
+- [**Subcases**](tutorial.html#test-cases-and-subcases) - an intuitive way to share common setup and teardown code for test cases (alternative to [**test fixtures**](testcases.html#test-fixtures) which are also supported)
+- [**templated test cases**](parameterized-tests.html#templated-test-cases---parameterized-by-type) - parameterized by type
+- supports [**logging macros**](logging.html) for capturing local variables and strings - as a message for when an assert fails - with lazy stringification and no allocations when possible!
+- crash handling support - uses signals for UNIX and SEH for Windows
- output from all compilers on all platforms is the same - byte by byte
+- binaries (exe/dll) can use the test runner of another binary - so tests end up in a single registry - [**example**](../../examples/executable_dll_and_plugin/)
- supports [**BDD style**](testcases.html) tests
-- only one core [**assertion macro**](assertions.html) for comparisons - standard C++ operators are used for the comparison (less than, equal, greater than...) - yet the full expression is decomposed and left and right values of the expression are logged
+- one core [**assertion macro**](assertions.html) for comparisons - standard C++ operators are used for the comparison (less than, equal, greater than...) - yet the full expression is decomposed and left and right values of the expression are logged
- assertion macros for [**exceptions**](assertions.html) - if something should or shouldn't throw
- floating point comparison support - see the [**```Approx()```**](assertions.html#floating-point-comparisons) helper
-- powerful mechanism for [**stringification**](stringification.html) of user types
-- tests can be grouped in [**test suites**](testcases.html)
+- powerful mechanism for [**stringification**](stringification.html) of user types - including [**exceptions**](stringification.html#translating-exceptions)!
+- tests can be grouped in [**test suites**](testcases.html#test-suites)
+- test case [**decorators**](testcases.html#decorators) such as ```description``` / ```skip``` / ```may_fail``` / ```should_fail``` / ```expected_failures``` / ```timeout```
+- can be used without exceptions and rtti - checkout [**```DOCTEST_CONFIG_NO_EXCEPTIONS```**](configuration.html#doctest_config_no_exceptions)
- powerful [**command line**](commandline.html) with lots of options
+- can report the duration of test cases
- tests can be [**filtered**](commandline.html) based on their name/file/test suite using wildcards
+- can [**filter**](commandline.html) subcases using wildcards and by specifying the nesting levels for which those filters should work
- failures can (optionally) break into the debugger on Windows and Mac
- integration with the output window of Visual Studio for failing tests
- a ```main()``` can be provided when implementing the library with the [**```DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN```**](main.html#doctest_config_implement_with_main) identifier
- can write tests in headers - they will still be registered only once in the executable/shared object
-- [**range-based**](commandline.html) execution of tests - see the [**range_based_execution**](../../examples/range_based_execution/) example (the **run.py** script)
+- [**range-based**](commandline.html) execution of tests - see the [**example python script**](../../examples/range_based_execution.py)
- colored output in the console
- controlling the order of test execution
+- different ```doctest::Context```s can be created and ran many times within a single execution of the program
+- ability to query if code is currently being ran in a test - ```doctest::isRunningInTest()```
There is a list of planned features which are all important and big - see the [**roadmap**](roadmap.html).
diff --git a/doc/html_generated/logging.html b/doc/html_generated/logging.html
new file mode 100644
index 0000000..04c9583
--- /dev/null
+++ b/doc/html_generated/logging.html
@@ -0,0 +1,95 @@
+<!DOCTYPE html>
+<html>
+<title>logging</title>
+<xmp theme="united" style="display:none;">
+
+## Logging macros
+
+Additional messages can be logged during a test case.
+
+## INFO()
+
+The ```INFO()``` macro allows heterogenous sequences of values to be streamed using the insertion operator (```<<```) in the same way that ```std::ostream```, ```std::cout```, etc support it.
+
+```
+INFO("The number is " << i);
+```
+
+This message will be relevant to all asserts after it in the current scope or in scopes nested in the current one and will be printed later only if an assert fails.
+
+Note that there is no initial ```<<``` - instead the insertion sequence is placed in parentheses.
+
+The message is **NOT** constructed right away - instead it gets lazily stringified only when needed. This means that rvalues (temporaries) cannot be passed to the ```INFO()``` macro. All literals are treated as rvalue references by the standard - except for C string literals (```"like this one"```). That means that even normal integer literals cannot be used directly - they need to be stored in a variable/constant before being passed to ```INFO()```. If C++14 is used (or Visual Studio 2017+) doctest provides the ```TO_LVALUE()``` macro to help in this regard - it turns any literal or constexpr value to an lvalue and can be used like this:
+
+```
+constexpr int foo() { return 42; }
+TEST_CASE("playing with literals and constexpr values") {
+ INFO(TO_LVALUE(6) << " and this is a C string literal " << TO_LVALUE(foo()));
+ CHECK(false);
+}
+```
+
+```TO_LVALUE()``` can also help in contexts where you might want to avoid a ```static constexpr``` member to be ODR-used - see [**```DOCTEST_CONFIG_ASSERTION_PARAMETERS_BY_VALUE```**](configuration.html#doctest_config_assertion_parameters_by_value).
+
+Some notes:
+
+- the lazy stringification means the values will be stringified when an assert fails and not at the point of capture - so the value might have changed
+- if the library is built with C++11 rvalue reference support (see [**```DOCTEST_CONFIG_WITH_RVALUE_REFERENCES```**](configuration.html#doctest_config_with_rvalue_references)) then deleted overloads are provided to prohibit rvalues from being captured in an **```INFO()```** call - since the lazy stringification actually caches pointers to the objects. For C++98 temporaries will again not work but there will be horrible compilation errors
+- the [**```DOCTEST_CONFIG_NUM_CAPTURES_ON_STACK```**](configuration.html#doctest_config_num_captures_on_stack) config identifier can be used to control how much stack space is used to avoid heap allocations for the streaming macros
+- stream manipulators (from ```<iomanip>```) can be used but need to be created as local variables and used as lvalues
+- refer to the [**stringification**](stringification.html) page for information on how to teach doctest to stringify your types
+
+The lazy stringification and the stack usage means that in the common case when no asserts fail the code runs super fast. This makes it suitable even in loops - perhaps to log the iteration.
+
+There is also the **```CAPTURE()```** macro which is a convenience wrapper of **```INFO()```**:
+
+```
+CAPTURE(some_variable)
+```
+
+This will handle the stringification of the variable name for you (actually it works with any expression, not just variables).
+
+This would log something like:
+
+```
+ some_variable := 42
+```
+
+## Messages which can optionally fail test cases
+
+There are a few other macros for logging information:
+
+- ```MESSAGE(message)```
+- ```FAIL_CHECK(message)```
+- ```FAIL(message)```
+
+```FAIL()``` is like a ```REQUIRE``` assert - fails the test case and exits it. ```FAIL_CHECK()``` acts like a ```CHECK``` assert - fails the test case but continues with the execution. ```MESSAGE()``` just prints a message.
+
+In all these macros the messages are again composed using the ```<<``` streaming operator - like this:
+
+```
+FAIL("This is not supposed to happen! some var: " << var);
+```
+
+Also there is no lazy stringification here - strings are always constructed and printed and thus there are no limitations to the values being logged - temporaries and rvalues are accepted - unlike with the ```INFO()``` macro.
+
+There are also a few more intended for use by third party libraries such as mocking frameworks:
+
+- ```ADD_MESSAGE_AT(file, line, message)```
+- ```ADD_FAIL_CHECK_AT(file, line, message)```
+- ```ADD_FAIL_AT(file, line, message)```
+
+They can be useful when integrating asserts from a different framework with doctest.
+
+------
+
+- Check out the [**example**](../../examples/all_features/logging.cpp) which shows how all of these are used.
+
+---
+
+[Home](readme.html#reference)
+
+
+</xmp>
+<script src="strapdown.js/strapdown.js"></script>
+</html>
diff --git a/doc/html_generated/main.html b/doc/html_generated/main.html
index e56840b..d77acc4 100644
--- a/doc/html_generated/main.html
+++ b/doc/html_generated/main.html
@@ -16,19 +16,21 @@
However if you need more control - want to set options with code to the execution context or want to integrate the framework in your production code - then the default ```main()``` just won't do the job. In that case use [**```DOCTEST_CONFIG_IMPLEMENT```**](configuration.html#doctest_config_implement).
-All the [**command line**](commandline.html) options can be set like this (flags cannot because it wouldn't make sense). Filters can only be appended or cleared with ```Context::clearFilters()``` - the user cannot remove a specific filter with code.
+All the [**command line**](commandline.html) options can be set like this (flags cannot because it wouldn't make sense). Filters can only be appended or cleared with the ```addFilter()``` or ```clearFilters()``` method of a ```doctest::Context``` object - the user cannot remove a specific filter with code.
```
#define DOCTEST_CONFIG_IMPLEMENT
#include "doctest.h"
int main(int argc, char** argv) {
- doctest::Context context; // initialize
+ doctest::Context context;
+
+ // !!! THIS IS JUST AN EXAMPLE SHOWING HOW DEFAULTS/OVERRIDES ARE SET !!!
// defaults
context.addFilter("test-case-exclude", "*math*"); // exclude test cases with "math" in their name
context.setOption("abort-after", 5); // stop test execution after 5 failed assertions
- context.setOption("sort", "name"); // sort the test cases by their name
+ context.setOption("order-by", "name"); // sort the test cases by their name
context.applyCommandLine(argc, argv);
@@ -52,9 +54,11 @@
### Dealing with shared objects (DLLs)
-When integrating the framework in production code which gets built as a shared object (dll) everything still works. Many shared objects and an executable can have tests in them and can even use different versions of the **doctest** framework.
+The framework can be used separately in binaries (executables / shared objects) with each having it's own test runner - this way even different versions of doctest can be used - but there will be no simple way to execute the tests from all loaded binaries and have the results aggregated and summarized.
-Check out [**this**](../../examples/dll_and_executable/) example which showcases how to call the tests in a shared object from the executable (and it also showcases that if a file with a test case is included both in the shared object and the executable then the test is registered in both places).
+There is also an option to have the test runner (implementation) built in a binary and shared with others (so there is a single test registry) by exporting it's public symbols (the ones needed for writing tests by the user - all the forward declarations of the framework).
+
+For more info on that checkout the [**```DOCTEST_CONFIG_IMPLEMENTATION_IN_DLL```**](configuration.html#doctest_config_implementation_in_dll) config identifier and [**this example**](../../examples/executable_dll_and_plugin/).
---------------
diff --git a/doc/html_generated/parameterized-tests.html b/doc/html_generated/parameterized-tests.html
new file mode 100644
index 0000000..1108e02
--- /dev/null
+++ b/doc/html_generated/parameterized-tests.html
@@ -0,0 +1,151 @@
+<!DOCTYPE html>
+<html>
+<title>parameterized-tests</title>
+<xmp theme="united" style="display:none;">
+
+## Parameterized test cases
+
+Test cases can be parameterized easily by type and indirectly by value.
+
+## Value-parameterized test cases
+
+There will be proper support for this in the future. For now there are 2 ways of doing data-driven testing in doctest:
+
+- extracting the asserts in a helper function and calling it with a user-constructed array of data:
+
+ ```
+ void doChecks(int data) {
+ // do asserts with data
+ }
+
+ TEST_CASE("test name") {
+ std::vector<int> data {1, 2, 3, 4, 5, 6};
+
+ for(auto& i : data) {
+ CAPTURE(i); // log the current input data
+ doChecks(i);
+ }
+ }
+ ```
+
+ This has several drawbacks:
+ - in case of an exception (or a ```REQUIRE``` assert failing) the entire test case ends and the checks are not done for the rest of the input data
+ - the user has to manually log the data with calls to ```CAPTURE()``` ( or ```INFO()```)
+ - more boilerplate - doctest should supply primitives for generating data but currently doesnt - so the user has to write his own data generation
+
+- using subcases to initialize data differently:
+
+ ```
+ TEST_CASE("test name") {
+ int data;
+ SUBCASE("") { data = 1; }
+ SUBCASE("") { data = 2; }
+
+ CAPTURE(data);
+
+ // do asserts with data
+ }
+ ```
+
+ This has the following drawbacks:
+ - doesn't scale well - it is very impractical to write such code for more than a few different inputs
+ - the user has to manually log the data with calls to ```CAPTURE()``` (or ```INFO()```)
+
+Stay tuned for proper value-parameterization in doctest!
+
+## Templated test cases - parameterized by type
+
+Suppose you have multiple implementations of the same interface and want to make sure that all of them satisfy some common requirements. Or, you may have defined several types that are supposed to conform to the same "concept" and you want to verify it. In both cases, you want the same test logic repeated for different types.
+
+While you can write one ```TEST_CASE``` for each type you want to test (and you may even factor the test logic into a function template that you invoke from the test case), it's tedious and doesn't scale: if you want ```M``` tests over ```N``` types, you'll end up writing ```M * N``` tests.
+
+Templated tests allow you to repeat the same test logic over a list of types. You only need to write the test logic once.
+
+There are 2 ways to do it:
+
+- directly pass the list of types to the templated test case
+
+ ```
+ typedef doctest::Types<char, short, int, long long int> the_types;
+
+ TEST_CASE_TEMPLATE("signed integers stuff", T, the_types) {
+ T var = T();
+ --var;
+ CHECK(var == -1);
+ }
+ ```
+
+- define the templated test case with a specific unique name (identifier) for later instantiation
+
+ ```
+ TEST_CASE_TEMPLATE_DEFINE("signed integer stuff", T, test_id) {
+ T var = T();
+ --var;
+ CHECK(var == -1);
+ }
+
+ typedef doctest::Types<char, short, int, long long int> the_types;
+ TEST_CASE_TEMPLATE_INSTANTIATE(test_id, the_types);
+
+ typedef doctest::Types<float, double> the_types_2;
+ TEST_CASE_TEMPLATE_INSTANTIATE(test_id, the_types_2);
+ ```
+ If you are designing an interface or concept, you can define a suite of type-parameterized tests to verify properties that any valid implementation of the interface/concept should have. Then, the author of each implementation can just instantiate the test suite with his type to verify that it conforms to the requirements, without having to write similar tests repeatedly.
+
+
+A test case named ```signed integers stuff``` instantiated for type ```int``` will yield the following test case name:
+
+```
+signed integers stuff<int>
+```
+
+By default all primitive types (fundamental - ```int```, ```bool```, ```float```...) have stringification provided by the library. For all other types the user will have to use the ```TYPE_TO_STRING(type)``` macro - like this:
+
+```
+TYPE_TO_STRING(std::vector<int>);
+```
+
+The ```TYPE_TO_STRING``` macro has an effect only in the current source file and thus needs to be used in some header if the same type will be used in separate source files for templated test cases.
+
+Other testing frameworks use the header ```<typeinfo>``` in addition to demangling to get the string for types automatically but doctest cannot afford to include any header in it's forward declaration part (the public one) of the header - so the user has to teach the framework for each type. This is done to achieve [maximal compile time performance](benchmarks.html).
+
+Some notes:
+
+- types are NOT filtered for uniqueness - the same templated test case can be instantiated multiple times for the same type - preventing that is left up to the user
+- you don't need to provide stringification for every type as that plays a role only in the test case name - the default is ```<>``` - the tests will still work and be distinct
+- the ```doctest::Types<>``` template accepts up to 60 type arguments
+- if variadic macros are enabled (see [**```DOCTEST_CONFIG_WITH_VARIADIC_MACROS```**](configuration.html#doctest_config_with_variadic_macros)) the typedefs can be skipped and the type lists can be constructed directly in the macros - otherwise the compiler will think that each comma in the type list introduces a new macro argument. With variadic macro support the ```TYPE_TO_STRING``` macro will also be able to work with types such as ```std::pair<int, float>```.
+- if you need parameterization on more than 1 type you can package multiple types in a single one like this:
+
+ ```
+ template <typename first, typename second>
+ struct TypePair
+ {
+ typedef first A;
+ typedef second B;
+ };
+
+ typedef Types<
+ TypePair<int, char>,
+ TypePair<char, int>
+ > pairs;
+
+ TEST_CASE_TEMPLATE("multiple types", T, pairs) {
+ typedef typename T::A T1;
+ typedef typename T::B T2;
+ // use T1 and T2 types
+ }
+ ```
+
+------
+
+- Check out the [**example**](../../examples/all_features/templated_test_cases.cpp) which shows how all of these are used.
+
+---
+
+[Home](readme.html#reference)
+
+
+</xmp>
+<script src="strapdown.js/strapdown.js"></script>
+</html>
diff --git a/doc/html_generated/readme.html b/doc/html_generated/readme.html
index 61073ce..90109fc 100644
--- a/doc/html_generated/readme.html
+++ b/doc/html_generated/readme.html
@@ -8,28 +8,31 @@
Project:
-- [Features and design goals](features.html) - the complete list of features
+- [Features and design goals](features.html) - the complete list of features and rationale behind the design
- [Roadmap](roadmap.html) - upcoming features
-- [Benchmarks](benchmarks.html) - compile time supremacy
-- [Contributing](contributing.html) - how to make a proper pull request
+- [Benchmarks](benchmarks.html) - compile-time and runtime supremacy
+- [Contributing](../../CONTRIBUTING.html) - how to make a proper pull request
+- [Changelog](../../CHANGELOG.html) - generated changelog based on closed issues/PRs
Usage:
- [Tutorial](tutorial.html) - make sure you have read it before the other parts of the documentation
- [Assertion macros](assertions.html)
- [Test cases, subcases and test fixtures](testcases.html)
+- [Parameterized test cases](parameterized-tests.html)
+- [Logging macros](logging.html)
- [Command line](commandline.html)
- [```main()``` entry point](main.html)
- [Configuration](configuration.html)
- [String conversions](stringification.html)
- [FAQ](faq.html)
+- [Build systems](build-systems.html)
- [Examples](../../examples)
This library is free, and will stay free but needs your support to sustain its development. There are lots of [**new features**](roadmap.html) and maintenance to do. If you work for a company using **doctest** or have the means to do so, please consider financial support.
-[](https://pledgie.com/campaigns/31280)
[](http://www.patreon.com/onqtam)
-[](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3K423Q6TK48BN)
+[](https://www.paypal.me/onqtam)
</xmp>
diff --git a/doc/html_generated/roadmap.html b/doc/html_generated/roadmap.html
index 198fd87..f0bd5cb 100644
--- a/doc/html_generated/roadmap.html
+++ b/doc/html_generated/roadmap.html
@@ -7,81 +7,151 @@
This library is free, and will stay free but needs your support to sustain its development. There are lots of [**new features**](roadmap.html) and maintenance to do. If you work for a company using **doctest** or have the means to do so, please consider financial support.
-
-[](https://pledgie.com/campaigns/31280)
[](http://www.patreon.com/onqtam)
-[](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3K423Q6TK48BN)
+[](https://www.paypal.me/onqtam)
-This is a list of planned features for future releases (not in the given order).
+Planned features for future releases - order changes constantly...
-- option to output summary only
-- ability to customize the colors in the console output
-- add support for <=, >= to Approx - like [requested](https://github.com/philsquared/Catch/pull/715) with [catch](https://github.com/philsquared/Catch/issues/651)
-- the set holding all registered tests should use a specialized allocator to minimize program startup time
-- pool allocator for the ```String``` class - currently very unoptimized
-- a mechanism for translating exceptions - users should be able to teach the framework about their types (look at Catch)
-- support for ```std::exception``` and derivatives (mainly for calling the ```.what()``` method when caught unexpectedly)
-- crash handling: signals on UNIX platforms or structured exceptions on Windows (should also have DOCTEST_CONFIG_NO_SIGNAL_CATCHING)
-- support for tags
- - may fail tag
- - invisible tag
- - look at Catch - https://github.com/philsquared/Catch/blob/master/docs/test-cases-and-sections.html#special-tags
-- output to file
+### For 1.3:
+
+- move from printf to using streams in the current console reporter and clean that mess
- reporters
+ - output to file
- a system for writing custom reporters
- ability to use multiple reporters at once (but only 1 to stdout)
- a compact reporter
+ - a progress reporter - or maybe just an option for the console reporter
- an xml reporter
- - jUnit/xUnit reporters
-- add the ability to query if code is currently being ran in a test - ```doctest::isRunningInTest()```
-- convolution support for the assertion macros (with a predicate)
-- time stuff
- - reporting running time of tests
- - restricting duration of test cases
- - killing a test that exceeds a time limit (will perhaps require threading or processes)
-- adding contextual info to asserts (logging) - with an ```INFO```/```CONTEXT``` /```TRACEPOINT``` macro (also look at [this](https://github.com/philsquared/Catch/issues/601))
-- add ```ERROR```/```FAIL``` macros (also ```ADD_FAILURE_AT(file, line);``` and extend the asserts to have ```_AT``` variants)
-- running tests a few times
-- marking a test to run X times (should also multiply with the global test run times)
-- test execution in separate processes - ```fork()``` for UNIX and [this](https://github.com/nemequ/munit/issues/2) for Windows
-- ability to provide a temp folder that is cleared between each test case
-- detect floating point exceptions
-- ```Bitwise()``` class that has overloaded operators for comparison - to be used to check objects bitwise against each other
-- look into MSTest integration
- - http://accu.org/index.php/journals/1851
- - https://msdn.microsoft.com/en-us/library/hh270865.aspx
- - https://msdn.microsoft.com/en-us/library/hh598953.aspx
- - also look into similar Xcode integration - https://github.com/philsquared/Catch/pull/454
-- matchers - should investigate what they are - look at google test and Catch
-- generators? - look at Catch - and investigate what they are (also in [boost](http://www.boost.org/doc/libs/1_61_0/libs/test/doc/html/boost_test/tests_organization/test_cases/test_case_generation.html))
-- mocking - investigate google mock assertion macros and interop with doctest (also [mockitopp](https://github.com/tpounds/mockitopp) and [trompeloeil](https://github.com/rollbear/trompeloeil)) - and write in FAQ
-- look at property based testing (for example [rapidcheck](https://github.com/emil-e/rapidcheck)) - and write in FAQ
-- [symbolizer](https://github.com/facebook/folly/tree/master/folly/experimental/symbolizer) - for a stack trace - when an assertion fails - and it's in a user function with some deep callstack away from the current test case - how to know the exact code path that lead to the failing assert
-
-And here is a list of things that are being considered but not part of the roadmap yet:
-
+ - xUnit reporter
+ - a listener interface - similar to a reporter - look at Catch
- ability to have no output when everything succeeds
-- integrate static analysis on the CI: **msvc**, **clang**, **cppcheck**
-- extend Approx for types that have operator double - see [here](https://github.com/philsquared/Catch/issues/652) and [here](https://github.com/philsquared/Catch/pull/658)
-- option to list files in which there are test cases who match the current filters
-- support for doing assertions in multiple threads - synchronize their access to shared doctest state
+- option to output summary only
+- log levels - like in [boost test](http://www.boost.org/doc/libs/1_63_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/log_level.html)
+- matchers - should investigate what they are - look at google test/mock and Catch (also predicates and boost test)
+- convolution support for the assertion macros (with a predicate)
+- Value-Parameterized test cases
+- generators? - look at Catch - and investigate what they are
+- look at property based testing
+ - [rapidcheck](https://github.com/emil-e/rapidcheck)
+ - [autocheck](https://github.com/thejohnfreeman/autocheck)
+ - [CppQuickCheck](https://github.com/grogers0/CppQuickCheck)
+- IDE integration
+ - https://blogs.msdn.microsoft.com/vcblog/2017/05/10/unit-testing-and-the-future-announcing-the-test-adapter-for-google-test/
+ - https://www.reddit.com/r/cpp/comments/65c0f1/run_cpp_unit_tests_from_xcode_and_visual_studio/
+ - https://github.com/k-brac/CUTI
+ - MSTest
+ - http://accu.org/index.php/journals/1851
+ - https://msdn.microsoft.com/en-us/library/hh270865.aspx
+ - https://msdn.microsoft.com/en-us/library/hh598953.aspx
+ - https://blogs.msdn.microsoft.com/vcblog/2017/04/19/cpp-testing-in-visual-studio/
+ - https://msdn.microsoft.com/en-us/library/hh419385.aspx
+ - XCode - https://github.com/philsquared/Catch/pull/454
+ - CLion
+ - https://www.jetbrains.com/clion/features/unit-testing.html
+ - https://blog.jetbrains.com/clion/2017/03/clion-2017-1-released/#catch
+
+### For 1.4:
+
+- running tests a [few times](https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.html#repeating-the-tests)
+- test execution in [separate processes](https://github.com/philsquared/Catch/issues/853) - ```fork()``` for UNIX and [this](https://github.com/nemequ/munit/issues/2) for Windows
+- killing a test that exceeds a time limit (will perhaps require threading or processes)
+- [symbolizer](https://github.com/facebook/folly/tree/master/folly/experimental/symbolizer) - for a stack trace - when an assertion fails - and it's in a user function with some deep callstack away from the current test case - how to know the exact code path that lead to the failing assert
+- ability to make the framework not capture unexpected exceptions - as requested [here](https://github.com/onqtam/doctest/issues/12#issuecomment-235334585)
+- add Approx ability to compare with absolute epsilon - [Catch PR](https://github.com/philsquared/Catch/pull/538)
+- ability to customize the colors in the console output (may also use styles - based on [this](https://github.com/agauniyal/rang))
+- implement breaking into the debugger under linux - see [here](https://github.com/philsquared/Catch/pull/585) and [here](https://github.com/scottt/debugbreak)
+- better testing of the library
+ - unit test the String class
+ - should unit test internals - currently even if a bug is caught by different output it's very difficult to track the reason
+ - should test stuff that should not compile
+ - should test crash handling
+ - should test more config options
+ - should test C++11 stuff - perhaps inspect the CMAKE_CXX_FLAGS for -std=c++11 on the CI and add more targets/tests
+ - test tricky stuff like expressions with commas in asserts
+
+### For 2.0:
+
+- remove C++98 support
+ - remove the config identifiers for C++11 features
+ - use variadic templates where appropriate
+ - update type lists to C++11
+ - update traits - use declval, etc.
+ - move initialization of fields from initializer lists to class bodies
+ - update static code analysis - less warning suppressing
+
+### For 3.0:
+
+- use modules - use ```std::string``` and whatever else comes from the standard - no more hand rolled traits and classes
+- minimize the use of the preprocessor
+
+### Things that are being considered but not part of the roadmap yet:
+
+- FakeIt mocking integration - like [catch](https://github.com/eranpeer/FakeIt/tree/master/config/catch)
+- consider the following 2 properties for the MSVC static code analyzer: EnableCppCoreCheck, EnableExperimentalCppCoreCheck
+- rpm package? like this: https://github.com/vietjtnguyen/argagg/blob/master/packaging/rpm/argagg.spec
+- get the current test case/section path - https://github.com/philsquared/Catch/issues/522
+- when no assertion is encountered in a test case it should fail
+- failure reporting should print out previous SECTIONs for data-driven testing - as requested [here](https://github.com/philsquared/Catch/issues/734)
+- ```Bitwise()``` class that has overloaded operators for comparison - to be used to check objects bitwise against each other
+- detect floating point exceptions
+- checkpoint/passpoint - like in [boost test](http://www.boost.org/doc/libs/1_63_0/libs/test/doc/html/boost_test/test_output/test_tools_support_for_logging/checkpoints.html) (also make all assert/subcase/logging macros to act as passpoints and print the last one on crashes or exceptions)
+- queries for the current test case - name (and probably decorators)
+- thread safety - asserts/subcases/captures should be safe to be used by multiple threads simultaneously
- support for running tests in parallel in multiple threads
-- a progress reporter
-- ability to filter not just TEST_CASE names but also SUBCASE names (and maybe tags when they are introduced)
-- doctest in a GUI environment? with no console? APIs for attaching a console? querying if there is one? investigate...
-- ability to specify ASC/DESC for the order option
-- command line error handling/reporting
-- ability for the user to extend the command line - as requested [here](https://github.com/philsquared/Catch/issues/622)
+- death tests - as in [google test](https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.html#death-tests)
+- config options
+ - test case name uniqueness - reject the ones with identical names
+- command line options
+ - ability to specify ASC/DESC for the order option
+ - global timeout option (per test or per entire session?)
+ - command line error handling/reporting
+ - ability for the user to extend the command line - as requested [here](https://github.com/philsquared/Catch/issues/622)
+ - option to list files in which there are test cases who match the current filters
+ - option for filters to switch from "match any" to "match all" mode
+ - option to list test suites and test cases in a tree view
+ - add a "wait key" option - as requested [here](https://github.com/philsquared/Catch/issues/477#issuecomment-256417686)
+- decorators for test cases and test suites- like in boost test
+ - depends_on
+ - precondition
+ - fixture
+ - label (tag) - with the ability to have multiple labels (tags) for a test case and also the ability to list them
+ - run X times (should also multiply with (or just override) the global test run times)
+ - throw an exception when incompatible decorators are given in the same list of decorators - like may_fail and should_fail
+- setup / teardown support
+ - global setup / teardown - can be currently achieved by providing a custom main function
+ - per test suite (block? only? and not all blocks of the same test suite?)
+ - as decorators
+ - see how it's done in boost test - with the fixture decorator
+ - perhaps for fixtures in addition to the constructor / destructor - since throwing in the destructor might terminate the program
+ - or just ignore all of this this - it would require globals or classes and inheritance - and we already have subcases
+- doctest in a GUI environment? with no console? APIs for attaching a console? querying if there is one? [investigate...](https://github.com/philsquared/Catch/blob/master/docs/configuration.html#stdout)
+- runtime performance
+ - startup - the set holding all registered tests should use a specialized allocator to minimize program startup time
+ - failing - optimize createStream/freeStream to reuse a pool of ostringstream objects
+- ability to provide a temp folder that is cleared between each test case
+- make the _MESSAGE assert macros work with variadic arguments - and maybe write the ones for binary/unary/fast asserts as well
+- move from operator "<<" to "<=" for capturing the left operand when decomposing binary expressions with templates
+- think about silencing warnings about unused variables when DOCTEST_CONFIG_DISABLE is used - see commit 6b61e8aa3818c5ea100cedc1bb48a60ea10df6e8 or issue #61
+- think about optionally using ```<typeinfo>``` and libcxxabi for demangling so users don't have to use ```TYPE_TO_STRING()```
+- handle more complex expressions - ```CHECK(foo() == 1 || bar() == 2);```
+- think about using a string view of some sorts
+- benchmark against google test and boost test
-The following list is with things that are very unlikely to enter the roadmap:
+### Things that are very unlikely to enter the roadmap:
-- test with missed warning flags for GCC - look into https://github.com/Barro/compiler-warnings
-- utf8???
+- think about removing the binary asserts (leaving only the fast binary asserts) because normal asserts + no try/catch in asserts are almost the same
+- move the "react()" part (the one that throws for REQUIRE asserts - or for when "abort-after=<int>" is reached) to a function call in the while() part of the asserts
+- stop using underscores for the begining of identifiers - the anonymous variables - against the standard...
+- templated fixture test cases
+- test with missed warning flags for GCC
+ - https://github.com/Barro/compiler-warnings
+ - http://stackoverflow.com/a/34971392/3162383
+- utf8 / unicode ???
+ - https://github.com/philsquared/Catch/pull/903
- handle ```wchar``` strings???
-- print a warning when no assertion is encountered in a test case
-- hierarchical test suites - using a stack for the pushed ones - should be easy
+- hierarchical test suites - using a stack for the pushed ones
+- ability to specify the width of the terminal in terms of characters (for example 60 - less than 80 - the default)
- ability to re-run only newly compiled tests based on time stamps using ```__DATE__``` and ```__TIME__``` - stored in some file
-- add option to not report line numbers - getting annoyed of re-committing reference output files with changed line reports from a tiny change...
- add underscores to all preprocessor identifiers not intended for use by the user
- put everything from the ```detail``` namespace also in a nested anonymous namespace to make them with internal linkage
- ability to put everything from doctest into an anonymous namespace - to allow the use of multiple different versions of **doctest** within the same binary (executable/dll) - like the [**stb**](https://github.com/nothings/stb) libraries can
diff --git a/doc/html_generated/stringification.html b/doc/html_generated/stringification.html
index c42c906..70c2846 100644
--- a/doc/html_generated/stringification.html
+++ b/doc/html_generated/stringification.html
@@ -14,8 +14,8 @@
```
std::ostream& operator<< (std::ostream& os, const T& value) {
- os << convertMyTypeToString(value);
- return os;
+ os << convertMyTypeToString(value);
+ return os;
}
```
@@ -27,8 +27,8 @@
```
std::ostream& T::operator<<(std::ostream& os) const {
- os << convertMyTypeToString(*this);
- return os;
+ os << convertMyTypeToString(*this);
+ return os;
}
```
@@ -40,9 +40,9 @@
namespace user {
struct udt {};
- doctest::String toString(const udt& value) {
- return convertMyTypeToString(value);
- }
+ doctest::String toString(const udt& value) {
+ return convertMyTypeToString(value);
+ }
}
```
@@ -54,18 +54,39 @@
```
namespace doctest {
- template<> struct StringMaker<T> {
- static String convert(const T& value) {
- return convertMyTypeToString(value);
+ template<> struct StringMaker<T> {
+ static String convert(const T& value) {
+ return convertMyTypeToString(value);
}
};
}
```
+## Translating exceptions
+
+By default all exceptions deriving from ```std::exception``` will be translated to strings by calling the ```what()``` method. For exception types that do not derive from ```std::exception``` - or if ```what()``` does not return a suitable string - use ```REGISTER_EXCEPTION_TRANSLATOR```. This defines a function that takes your exception type and returns a ```doctest::String```. It can appear anywhere in the code - it doesn't have to be in the same translation unit. For example:
+
+```
+REGISTER_EXCEPTION_TRANSLATOR(MyType& ex) {
+ return doctest::String(ex.message());
+}
+```
+
+Note that the exception may be accepted without a reference but it is considered bad practice in C++.
+
+An alternative way to register an exception translator is to do the following in some function - before executing any tests:
+
+```
+ // adding a lambda - the signature required is `doctest::String(exception_type)`
+ doctest::registerExceptionTranslator<int>([](int in){ return doctest::toString(in); });
+```
+
+The order of registering exception translators can be controlled - simply call the explicit function in the required order or list the exception translators with the macro in a top-to-bottom fashion in a single translation unit - everything that auto-registers in doctest works in a top-to-bottom way for a single translation unit (source file).
+
------
-- Check out the [**example**](../../examples/stringification/main.cpp) which shows how to stringify ```std::vector<T>``` and other types.
-- Note that the type ```String``` is used when specializing ```StringMaker<T>``` or overloading ```toString()``` - it is the string type **doctest** works with. ```std::string``` is not an option for the library because then it would have to drag the ```<string>``` header with it.
+- Check out the [**example**](../../examples/all_features/stringification.cpp) which shows how to stringify ```std::vector<T>``` and other types/exceptions.
+- Note that the type ```String``` is used when specializing ```StringMaker<T>``` or overloading ```toString()``` - it is the string type **doctest** works with. ```std::string``` is not an option because doctest would have to include the ```<string>``` header.
- To support the ```operator<<(std::ostream&...``` stringification the library has to offer a forward declaration of ```std::ostream``` and that is what the library does - but it is forbidden by the standard. It currently works everywhere - on all tested compilers - but if the user wishes to be 100% standards compliant - then the [**```DOCTEST_CONFIG_USE_IOSFWD```**](configuration.html#doctest_config_use_iosfwd) identifier can be used to force the inclusion of ```<iosfwd>```. The reason the header is not included by default is that on MSVC (for example) it drags a whole bunch of stuff with it - and after the preprocessor is finished the translation unit has grown to 42k lines of C++ code - while Clang and the libc++ are so well implemented that including ```<iosfwd>``` there results in 400 lines of code.
---
diff --git a/doc/html_generated/testcases.html b/doc/html_generated/testcases.html
index 43eb624..1d952eb 100644
--- a/doc/html_generated/testcases.html
+++ b/doc/html_generated/testcases.html
@@ -18,6 +18,10 @@
For examples see the [Tutorial](tutorial.html)
+Test cases can also be parameterized - see the [documentation](parameterized-tests.html)
+
+Test cases and subcases can be filtered through the use of the [**command line**](commandline.html)
+
## BDD-style test cases
In addition to **doctest**'s take on the classic style of test cases, **doctest** supports an alternative syntax that allow tests to be written as "executable specifications" (one of the early goals of [Behaviour Driven Development](http://dannorth.net/introducing-bdd/)). This set of macros map on to ```TEST_CASE```s and ```SUBCASE```s, with a little internal support to make them smoother to work with.
@@ -74,17 +78,21 @@
## Test suites
-Test cases can be grouped into test suites. This is done with the ```TEST_SUITE()``` and ```TEST_SUITE_END()``` macros.
+Test cases can be grouped into test suites. This is done with ```TEST_SUITE()``` or ```TEST_SUITE_BEGIN()``` / ```TEST_SUITE_END()```.
For example:
```
TEST_CASE("") {} // not part of any test suite
-TEST_SUITE("math");
+TEST_SUITE("math") {
+ TEST_CASE("") {} // part of the math test suite
+ TEST_CASE("") {} // part of the math test suite
+}
-TEST_CASE("") {} // part of the math test suite
-TEST_CASE("") {} // part of the math test suite
+TEST_SUITE_BEGIN("utils");
+
+TEST_CASE("") {} // part of the utils test suite
TEST_SUITE_END();
@@ -93,9 +101,62 @@
Then test cases from specific test suites can be executed with the help of filters - check out the [**command line**](commandline.html)
+## Decorators
+
+Test cases can be *decorated* with additional attributes like this:
+
+```
+TEST_CASE("name"
+ * doctest::description("shouldn't take more than 500ms")
+ * doctest::timeout(0.5)) {
+ // asserts
+}
+```
+
+Multiple decorators can be used at the same time. These are the currently supported decorators:
+
+- **```skip(bool = true)```** - marks the test case to be skipped from execution - unless the ```--no-skip``` option is used
+- **```may_fail(bool = true)```** - doesn't fail the test if any given assertion fails (but still reports it) - this can be useful to flag a work-in-progress, or a known issue that you don't want to immediately fix but still want to track in the your tests
+- **```should_fail("text")```** - like **```may_fail()```** but fails the test if it passes - his can be useful if you want to be notified of accidental, or third-party, fixes
+- **```expected_failures(int)```** - defines the number of assertions that are expected to fail within the test case - reported as failure when the number of failed assertions is different than the declared expected number of failures
+- **```timeout(double)```** - fails the test case if its execution exceeds this limit (in seconds) - but doesn't terminate it - that would require subprocess support
+- **```test_suite("name")```** - can be used on test cases to override (or just set) the test suite they are in
+- **```description("text")```** - a description of the test case
+
+The values that the decorators take are computed while registering the test cases (during global initialization) - before entering ```main()``` and not just before running them.
+
+Decorators can also be applied to test suite blocks and all test cases in that block inherit them:
+
+```
+TEST_SUITE("some TS" * doctest::description("all tests will have this")) {
+ TEST_CASE("has a description from the surrounding test suite") {
+ // asserts
+ }
+}
+TEST_SUITE("some TS") {
+ TEST_CASE("no description even though in the same test suite as the one above") {
+ // asserts
+ }
+}
+```
+
+Test cases can override the decorators that they inherit from their surrounding test suite:
+
+```
+TEST_SUITE("not longer than 500ms" * doctest::timeout(0.5)) {
+ TEST_CASE("500ms limit") {
+ // asserts
+ }
+ TEST_CASE("200ms limit" * doctest::timeout(0.2)) {
+ // asserts
+ }
+}
+```
+
------
-- Check out the [**example**](../../examples/subcases_and_bdd/main.cpp)
+- Check out the [**subcases and BDD example**](../../examples/all_features/subcases.cpp)
+- Check out the [**assertion macros example**](../../examples/all_features/assertion_macros.cpp) to see how test suites are used
- Tests are registered from top to bottom of each processed cpp after the headers have been preprocessed and included but there is no ordering between cpp files.
---------------
diff --git a/doc/markdown/benchmarks.md b/doc/markdown/benchmarks.md
index 6adf03e..746aa52 100644
--- a/doc/markdown/benchmarks.md
+++ b/doc/markdown/benchmarks.md
@@ -18,7 +18,7 @@
- Windows 7 - on an SSD
- Ubuntu 17.04 in a VirtualBox VM - on a HDD
-**doctest** version: 1.2.0 (released on 2017.05.15)
+**doctest** version: 1.2.0 (released on 2017.05.16)
[**Catch**](https://github.com/philsquared/Catch) version: 1.9.3 (released on 2017.04.25)
diff --git a/doctest/doctest.h b/doctest/doctest.h
index 400a4d7..e6567a7 100644
--- a/doctest/doctest.h
+++ b/doctest/doctest.h
@@ -102,9 +102,9 @@
#define DOCTEST_LIBRARY_INCLUDED
#define DOCTEST_VERSION_MAJOR 1
-#define DOCTEST_VERSION_MINOR 1
-#define DOCTEST_VERSION_PATCH 4
-#define DOCTEST_VERSION_STR "1.1.4"
+#define DOCTEST_VERSION_MINOR 2
+#define DOCTEST_VERSION_PATCH 0
+#define DOCTEST_VERSION_STR "1.2.0"
#define DOCTEST_VERSION \
(DOCTEST_VERSION_MAJOR * 10000 + DOCTEST_VERSION_MINOR * 100 + DOCTEST_VERSION_PATCH)
diff --git a/doctest/parts/doctest_fwd.h b/doctest/parts/doctest_fwd.h
index f0f2088..dd6a127 100644
--- a/doctest/parts/doctest_fwd.h
+++ b/doctest/parts/doctest_fwd.h
@@ -99,9 +99,9 @@
#define DOCTEST_LIBRARY_INCLUDED
#define DOCTEST_VERSION_MAJOR 1
-#define DOCTEST_VERSION_MINOR 1
-#define DOCTEST_VERSION_PATCH 4
-#define DOCTEST_VERSION_STR "1.1.4"
+#define DOCTEST_VERSION_MINOR 2
+#define DOCTEST_VERSION_PATCH 0
+#define DOCTEST_VERSION_STR "1.2.0"
#define DOCTEST_VERSION \
(DOCTEST_VERSION_MAJOR * 10000 + DOCTEST_VERSION_MINOR * 100 + DOCTEST_VERSION_PATCH)
diff --git a/examples/all_features/test_output/version.txt b/examples/all_features/test_output/version.txt
index 12266d4..eb769ef 100644
--- a/examples/all_features/test_output/version.txt
+++ b/examples/all_features/test_output/version.txt
@@ -1 +1 @@
-[doctest] doctest version is "1.1.4"
+[doctest] doctest version is "1.2.0"
diff --git a/scripts/hello_world.cpp b/scripts/hello_world.cpp
new file mode 100644
index 0000000..496e709
--- /dev/null
+++ b/scripts/hello_world.cpp
@@ -0,0 +1,12 @@
+#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
+#include "doctest.h"
+
+static int factorial(int number) { return number <= 1 ? number : factorial(number - 1) * number; }
+
+TEST_CASE("testing the factorial function") {
+ CHECK(factorial(0) == 1);
+ CHECK(factorial(1) == 1);
+ CHECK(factorial(2) == 2);
+ CHECK(factorial(3) == 6);
+ CHECK(factorial(10) == 3628800);
+}
diff --git a/scripts/update_wandbox_link.py b/scripts/update_wandbox_link.py
index a175f18..52e545a 100644
--- a/scripts/update_wandbox_link.py
+++ b/scripts/update_wandbox_link.py
@@ -5,7 +5,7 @@
# update main readme 'try it online' badge permalink
print("updating main readme with up-to-date wandbox link")
-proc = subprocess.Popen('python send_to_wandbox.py ../doctest/ ' + "../examples/hello_world/main.cpp", stdout = subprocess.PIPE)
+proc = subprocess.Popen('python send_to_wandbox.py ../doctest/ ' + "../scripts/hello_world.cpp", stdout = subprocess.PIPE)
url = proc.stdout.read().strip()
readme_contents = ""
diff --git a/scripts/version.txt b/scripts/version.txt
index 1b87bcd..867e524 100644
--- a/scripts/version.txt
+++ b/scripts/version.txt
@@ -1 +1 @@
-1.1.4
\ No newline at end of file
+1.2.0
\ No newline at end of file