| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 1 | <!DOCTYPE html> |
| 2 | <html> |
| 3 | <title>commandline</title> |
| 4 | <xmp theme="united" style="display:none;"> |
| 5 | |
| 6 | ## Command line |
| 7 | |
| onqtam | 1435c01 | 2016-09-21 15:29:11 +0300 | [diff] [blame] | 8 | **doctest** works quite nicely without any command line options at all - but for more control a bunch are available. |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 9 | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 10 | **Query flags** - after the result is printed the program quits without executing any test cases (and if the framework is integrated into a client codebase which [**supplies it's own ```main()``` entry point**](main.html) - the program should check the result of ```shouldExit()``` method after calling ```run()``` on a ```doctest::Context``` object and should exit - this is left up to the user). |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 11 | |
| 12 | **Int/String options** - they require a value after the ```=``` sign - without spaces! For example: ```--order-by=rand```. |
| 13 | |
| 14 | **Bool options** - they expect ```1```/```yes```/```on```/```true``` or ```0```/```no```/```off```/```false``` after the ```=``` sign - but they can also be used like flags and the ```=value``` part can be skipped - then ```true``` is assumed. |
| 15 | |
| 16 | **Filters** use wildcards for matching values - where ```*``` means "match any sequence" and ```?``` means "match any one character". |
| 17 | To pass a pattern with an interval use ```""``` like this: ```--test-case="*no sound*,vaguely named test number ?"```. |
| 18 | |
| onqtam | 1435c01 | 2016-09-21 15:29:11 +0300 | [diff] [blame] | 19 | All the options can also be set with code (defaults/overrides) if the user [**supplies the ```main()``` function**](main.html). |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 20 | |
| 21 | | Query Flags | Description | |
| 22 | |:------------|-------------| |
| 23 | | ```-?``` ```--help``` ```-h``` | Prints a help message listing all these flags/options | |
| 24 | | ```-v``` ```--version``` | Prints the version of the **doctest** framework | |
| 25 | | ```-c``` ```--count``` | Prints the number of test cases matching the current filters (see below) | |
| 26 | | ```-ltc``` ```--list-test-cases``` | Lists all test cases by name which match the current filters (see below) | |
| 27 | | ```-lts``` ```--list-test-suites``` | Lists all test suites by name which have at least one test case matching the current filters (see below) | |
| 28 | | **Int/String Options** | <hr> | |
| 29 | | ```-tc``` ```--test-case=<filters>``` | Filters test cases based on their name. By default all test cases match but if a value is given to this filter like ```--test-case=*math*,*sound*``` then only test cases who match atleast one of the patterns in the comma-separated list with wildcards will get executed/counted/listed | |
| 30 | | ```-tce``` ```--test-case-exclude=<filters>``` | Same as the ```-test-case=<filters>``` option but if any of the patterns in the comma-separated list of values matches - then the test case is skipped | |
| 31 | | ```-sf``` ```--source-file=<filters>``` | Same as ```--test-case=<filters>``` but filters based on the file in which test cases are written | |
| 32 | | ```-sfe``` ```--source-file-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on the file in which test cases are written | |
| 33 | | ```-ts``` ```--test-suite=<filters>``` | Same as ```--test-case=<filters>``` but filters based on the test suite in which test cases are in | |
| 34 | | ```-tse``` ```--test-suite-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on the test suite in which test cases are in | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 35 | | ```-sc``` ```--subcase=<filters>``` | Same as ```--test-case=<filters>``` but filters subcases based on their names | |
| 36 | | ```-sce``` ```--subcase-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on subcase names | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 37 | | ```-ob``` ```--order-by=<string>``` | Test cases will be sorted before being executed either by **the file in which they are** / **the test suite they are in** / **their name** / **random**. The possible values of ```<string>``` are ```file```/```suite```/```name```/```rand```. The default is ```file``` | |
| 38 | | ```-rs``` ```--rand-seed=<int>``` | The seed for random ordering | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 39 | | ```-f``` ```--first=<int>``` | The **first** test case to execute which passes the current filters - for range-based execution - see [**the example python script**](../../examples/range_based_execution.py) | |
| 40 | | ```-l``` ```--last=<int>``` | The **last** test case to execute which passes the current filters - for range-based execution - see [**the example python script**](../../examples/range_based_execution.py) | |
| 41 | | ```-aa``` ```--abort-after=<int>``` | The testing framework will stop executing test cases/assertions after this many failed assertions. The default is 0 which means don't stop at all. Note that the framework uses an exception to stop the current test case regardless of the level of the assert (```CHECK```/```REQUIRE```) - so be careful with asserts in destructors... | |
| 42 | | ```-scfl``` ```--subcase-filter-levels=<int>``` | Apply subcase filters only for the first ```<int>``` levels of nested subcases and just run the ones nested deeper. Default is a very high number which means *filter any subcase* | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 43 | | **Bool Options** | <hr> | |
| 44 | | ```-s``` ```--success=<bool>``` | To include successful assertions in the output | |
| 45 | | ```-cs``` ```--case-sensitive=<bool>``` | Filters being treated as case sensitive | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 46 | | ```-e``` ```--exit=<bool>``` | Exits after the tests finish - this is meaningful only when the client has [**provided the ```main()``` entry point**](main.html) - the program should check the ```shouldExit()``` method after calling ```run()``` on a ```doctest::Context``` object and should exit - this is left up to the user. The idea is to be able to execute just the tests in a client program and to not continue with it's execution | |
| onqtam | 6a5da42 | 2017-09-04 17:56:08 +0300 | [diff] [blame] | 47 | | ```-d``` ```--duration=<bool>``` | Prints the time each test case took in seconds | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 48 | | ```-nt``` ```--no-throw=<bool>``` | Skips [**exceptions-related assertion**](assertions.html#exceptions) checks | |
| 49 | | ```-ne``` ```--no-exitcode=<bool>``` | Always returns a successful exit code - even if a test case has failed | |
| 50 | | ```-nr``` ```--no-run=<bool>``` | Skips all runtime **doctest** operations (except the test registering which happens before the program enters ```main()```). This is useful if the testing framework is integrated into a client codebase which has [**provided the ```main()``` entry point**](main.html) and the user wants to skip running the tests and just use the program | |
| onqtam | e34600e | 2016-11-15 02:16:56 +0200 | [diff] [blame] | 51 | | ```-nv``` ```--no-version=<bool>``` | Omits the framework version in the output | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 52 | | ```-nc``` ```--no-colors=<bool>``` | Disables colors in the output | |
| onqtam | 3bb1157 | 2017-02-18 20:35:58 +0200 | [diff] [blame] | 53 | | ```-fc``` ```--force-colors=<bool>``` | Forces the use of colors even when a tty cannot be detected | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 54 | | ```-nb``` ```--no-breaks=<bool>``` | Disables breakpoints in debuggers when an assertion fails | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 55 | | ```-ns``` ```--no-skip=<bool>``` | Don't skip test cases marked as skip with a decorator | |
| onqtam | 68681e6 | 2018-05-10 16:18:17 +0300 | [diff] [blame] | 56 | | ```-gfl``` ```--gnu-file-line=<bool>``` | ```:n:``` vs ```(n):``` for line numbers in output (gnu mode is usually for linux tools/IDEs and is with the ```:``` separator) | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 57 | | ```-npf``` ```--no-path-filenames=<bool>``` | Paths are removed from the output when a filename is printed - useful if you want the same output from the testing framework on different environments | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 58 | | ```-nln``` ```--no-line-numbers=<bool>``` | Line numbers are replaced with ```0``` in the output when a source location is printed - useful if you want the same output from the testing framework even when test positions change within a source file | |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 59 | | | | |
| 60 | |
| onqtam | 1435c01 | 2016-09-21 15:29:11 +0300 | [diff] [blame] | 61 | All the flags/options also come with a prefixed version (with ```--dt-``` at the front) - for example ```--version``` can be used also with ```--dt-version``` or ```--dt-v```. |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 62 | |
| onqtam | 1435c01 | 2016-09-21 15:29:11 +0300 | [diff] [blame] | 63 | All the unprefixed versions listed here can be disabled with the [**```DOCTEST_CONFIG_NO_UNPREFIXED_OPTIONS```**](configuration.html#doctest_config_no_unprefixed_options) define. |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 64 | |
| onqtam | 1435c01 | 2016-09-21 15:29:11 +0300 | [diff] [blame] | 65 | This is done for easy interoperability with client command line option handling when the testing framework is integrated within a client codebase - all **doctest** related flags/options can be prefixed so there are no clashes and so that the user can exclude everything starting with ```--dt-``` from their option parsing. |
| 66 | |
| onqtam | b8220c5 | 2017-05-16 00:21:15 +0300 | [diff] [blame] | 67 | If there isn't an option to exclude everything starting with ```--dt-``` then the ```dt_removed``` helper class might help to filter them out: |
| 68 | |
| 69 | ``` |
| 70 | #define DOCTEST_CONFIG_NO_UNPREFIXED_OPTIONS |
| 71 | #define DOCTEST_CONFIG_IMPLEMENT |
| 72 | #include "doctest.h" |
| 73 | |
| 74 | class dt_removed { |
| 75 | std::vector<const char*> vec; |
| 76 | public: |
| 77 | dt_removed(const char** argv_in) { |
| 78 | for(; *argv_in; ++argv_in) |
| 79 | if(strncmp(*argv_in, "--dt-", strlen("--dt-")) != 0) |
| 80 | vec.push_back(*argv_in); |
| 81 | vec.push_back(NULL); |
| 82 | } |
| 83 | |
| 84 | int argc() { return static_cast<int>(vec.size()) - 1; } |
| 85 | const char** argv() { return &vec[0]; } // Note: non-const char **: |
| 86 | }; |
| 87 | |
| 88 | int program(int argc, const char** argv); |
| 89 | |
| 90 | int main(int argc, const char** argv) { |
| 91 | doctest::Context context(argc, argv); |
| 92 | int test_result = context.run(); // run queries, or run tests unless --no-run |
| 93 | |
| 94 | if(context.shouldExit()) // honor query flags and --exit |
| 95 | return test_result; |
| 96 | |
| 97 | dt_removed args(argv); |
| 98 | int app_result = program(args.argc(), args.argv()); |
| 99 | |
| 100 | return test_result + app_result; // combine the 2 results |
| 101 | } |
| 102 | |
| 103 | int program(int argc, const char** argv) { |
| 104 | printf("Program: %d arguments received:\n", argc - 1); |
| 105 | while(*++argv) |
| 106 | printf("'%s'\n", *argv); |
| 107 | return EXIT_SUCCESS; |
| 108 | } |
| 109 | ``` |
| 110 | |
| 111 | When ran like this: |
| 112 | |
| 113 | ``` |
| 114 | program.exe --dt-test-case=math* --my-option -s --dt-no-breaks |
| 115 | ``` |
| 116 | |
| 117 | Will output this: |
| 118 | |
| 119 | ``` |
| 120 | Program: 2 arguments received: |
| 121 | '--my-option' |
| 122 | '-s' |
| 123 | ``` |
| onqtam | 8126b56 | 2016-05-27 17:01:15 +0300 | [diff] [blame] | 124 | |
| 125 | --------------- |
| 126 | |
| 127 | [Home](readme.html#reference) |
| 128 | |
| 129 | |
| 130 | </xmp> |
| 131 | <script src="strapdown.js/strapdown.js"></script> |
| 132 | </html> |