# -*- rd -*-

= README --- An introduction of Cutter, a Unit Testing Framework for C

== Name

Cutter

== Author

  * Kouhei Sutou <kou@cozmixng.org>
  * Hiroyuki Ikezoe <poincare@ikezoe.net>

== License

LGPLv3

== What's this?

Cutter is a xUnit family Unit Testing Framework for C.

This is a list of features of Cutter:
  * easy to write tests.
  * outputs result with useful format for debugging.
  * tests are built as shared libraries.

See ((<FEATURES>)) for more details.

== Dependency libraries

  * GLib >= 2.16

== Get

((<URL:http://sourceforge.net/project/showfiles.php?group_id=208375>))

  % svn co https://cutter.svn.sourceforge.net/svnroot/cutter/cutter/trunk cutter

== Install

  % ./configure
  % make
  # make install

== Usage

  % cutter [Options] [Directory which has test_*.so]

Note: test_*.so are searched recursively.

=== Options

: --version

   Cutter shows its own version and exits.

: -s DIRECTORY, --source-directory=DIRECTORY

   Cutter prepends DIRECTORY to file name when test fails. This
   is for tolls (like Emacs) which have function jumping to
   error line.

: -t TEST_CASE_NAME, --test-case=TEST_CASE_NAME

   Cutter runs test cases that are matched with
   TEST_CASE_NAME. If TEST_CASE_NAME is surrounded by "/"
   (e.g. /test_/), TEST_CASE_NAME is handled as regular
   expression.

   This option can be specified n times. In the case, Cutter
   runs test cases that are matched with any
   TEST_CASE_NAME. (OR)

: -n TEST_NAME, --name=TEST_NAME

   Cutter runs tests that are matched with TEST_NAME. If
   TEST_NAME is surrounded by "/" (e.g. /test_/), TEST_NAME
   is handled as regular expression.

   This option can be specified n times. In the case, Cutter
   runs test that are matched with any TEST_NAME. (OR)

: -m, --multi-thread

   Cutter runs a test case in a new thread.

: --test-case-order=[none|name|name-desc]

   It specifies test case order. Cutter doesn't sort by
   default. (none)

   If 'name' is specified, Cutter sorts test cases by name in
   ascending order. If 'name-desc' is specified, Cutter
   sorts test cases by name in descending order.

: -u[console|gtk], --ui=[console|gtk]

   It specifies UI. The default is console UI.

: -v[s|silent|n|normal|v|verbose], --verbose=[s|silent|n|normal|v|verbose]

   It specifies verbose level.

   This option is only for console UI.

: -c[yes|true|no|false|auto], --color=[yes|true|no|false|auto]

   If 'yes' or 'true' is specified, Cutter uses colorized
   output by escape sequence. If 'no' or 'false' is
   specified, Cutter never use colorized output. If 'auto'
   or the option is omitted, Cutter uses colorized output if
   available.

   This option is only for console UI.

: --xml-report=FILE

   Cutter outputs a test report to FILE as XML format.

: -?, --help

   Cutter shows common options.

: --help-all

   Cutter shows all options.

== How to test

Executing flow of test is the following.

  (1) Write a test.
  (2) Compile it and build test_*.so.
  (3) Execute cutter. It loads test_*.so and runs them.

See ((<a tutorial|TUTORIAL>)) and sample/stack/.

== Test result

Here is an example test result:

  ..........F.................................................

  1) Failure: test_test_case_count
  <1 == cut_test_case_get_n_tests(test_object, NULL)>
  expected: <1>
   but was: <0>
  test/test-cut-test-case.c:143: test_test_case_count()

  Finished in 0.020857 seconds

  60 test(s), 253 assertion(s), 1 failure(s), 0 error(s), 0 pending(s), 0 notification(s)

=== Progress

A part that contains "." and "F" of the test result shows
test progress:

  ..........F.................................................

Each "." and "F" shows a test case (test function). "."
shows a test case that is succeeded and "F" shows a test
case that is failed. There are "E", "P" and "N". They shows
error, pending and notification respectively. Here is a
summary of test case marks:

: .
   A succeeded test

: F
   A failed test

: E
   A test that had an error

: P
   A test that is marked as pending

: N
   A test that had an notification

The above marks are showed after each test is finished. We
can confirm the test progress from the output in testing.

=== Summary of test result

Cutter outputs a summary of test result after all tests are
finished. The first of a summary is a list of a detail of
test result of non-succeeded test. In the example, cutter
outputs a detail of test result because there is a failure.

  1) Failure: test_test_case_count
  <1 == cut_test_case_get_n_tests(test_object, NULL)>
  expected: <1>
   but was: <0>
  test/test-cut-test-case.c:143: test_test_case_count()

In the example, test_test_case_count test case is failed. We
expected that cut_test_case_get_n_tests(test_object, NULL)
is 1 but was 0. The failed assertion is in
test_test_case_count() function in test/test-cut-test-case.c
at 143th line.

Elapsed time for testing is showed after a list of a detail
of test result:

  Finished in 0.020857 seconds

The last line is an summary of test result:

  60 test(s), 253 assertion(s), 1 failure(s), 0 error(s), 0 pending(s), 0 notification(s)

Here are the means of each output:

: n test(s)

   n test case(s) (test function(s)) are run.

: n assertion(s)

   n assertion(s) are passed.

: n failure(s)

   n assertion(s) are failed.

: n error(s)

   n error(s) are occurred (cut_error() is used n times)

: n pending(s)

   n test case(s) are pending (cut_pending() is used n times)

: n notification(s)

   n notification(s) are occurred (cut_notification() is used n times)

In the example, 60 test cases are run, 253 assertions are
passed and an assertion is failed. There are no error,
pending, notification.

=== XML report

Cutter reports test result as XML format if --xml-report
option is specified. A reported XML has the following
structure:

  <report>
    <result>
      <test-case>
        <name>TEST CASE NAME</name>
        <description>DESCRIPTION OF TEST CASE (if exists)</description>
      </test-case>
      <test>
        <name>TEST NAME</name>
        <description>DESCRIPTION OF TEST CASE (if exists)</description>
        <option><!-- ATTRIBUTE INFORMATION (if exists) -->
          <name>ATTRIBUTE NAME (e.g.: bug)</name>
          <value>ATTRIBUTE VALUE (e.g.: 1234)</value>
        </option>
        <option>
          ...
        </option>
      </test>
      <status>TEST RESULT ([success|failure|error|pending|notification])</status>
      <detail>DETAIL OF TEST RESULT (if exists)</detail>
      <backtrace><!-- BACKTRACE (if exists) -->
        <entry>
          <file>FILE NAME</file>
          <line>LINE</line>
          <info>ADDITIONAL INFORMATION</info>
        </entry>
        <entry>
          ...
        </entry>
      </backtrace>
      <elapsed>ELAPSED TIME (e.g.: 0.000010)</elapsed>
    </result>
    <result>
      ...
    </result>
    ...
  </report>

=== Test coverage

You can see the code coverage with Cutter if your system
have ((<LTP tools|URL:http://ltp.sourceforge.net/>)).  To
see the coverage, add the followling line in your
configure.ac and type "make coverage".

  AC_CHECK_COVERAGE

== References

=== Assertions

See ((<"cutter/cut-assertions.h"|cutter-cut-assertions.html>)).

=== Attributes

You can add attributes to your test to get more useful
information on failure. For example, you can add Bug ID like
the following

  void attributes_invalid_input(void);
  void test_invalid_input(void);

  void
  attributes_invalid_input (void)
  {
      cut_set_attributes("bug", "123");
  }

  void
  test_invalid_input (void)
  {
      cut_assert_equal("OK", get_input());
  }

In the above example, test_invalid_input test has an
attribute that the test is for Bug #123.

You need to define a function whose name is
"attributes_#{TEST_NAME - 'test_' PREFIX}" to add
attributes to a test. In the above example, attributes set
function, "attributes_invalid_input", is defined to set
"bug" attribute to "test_invalid_input" test.

=== Template

The following is a template of test.

  #include <cutter.h>
  
  #include "HEADER_FILE_OF_YOUR_PROGRAM"
  
  void test_condition(void);
  void test_strstr(void);

  static int condition = 0;
  
  void
  setup (void)
  {
      condition = 1;
  }
  
  void
  teardown (void)
  {
      condition = 0;
  }

  void
  test_condition(void)
  {
      cut_assert_equal_int(1, condition,
                           "The condition value should be set to 1 in setup()");
    ...
  }
  
  void
  test_strstr(void)
  {
      cut_assert_equal_string("sub-string",
                              strstr("string sub-string", "sub"));
      ...
  }
  
== Thanks

  * Kazumasa Matsunaga: reports a build bug.
  * Daijiro MORI: reports bugs.
