Common Lisp Package: CLUNIT

README:

CLUnit CLUnit is a Common Lisp unit testing framework. It is designed to be easy to use so that you can quickly start testing.

CLUnit provides a rich set of features aimed at improving your unit testing experience:

  • Multiple inheritance for test suites allows you to group tests into hierarchies.
  • Composes the test results of each test run into a single report.
  • Allows redefinition of inline functions and macros without having to redefine your tests.
  • Supports composable test suite fixtures.
  • Allows for an interactive testing process which gives you access to the test environment.
  • Provides visual feeback of the unit test progress.
  • Extensible test reporting. Builtin support for default reporting and TAP output.

Check out the comprehensive CLUnit Tutorial.

FUNCTION

Public

DEFINED-SUITE-P (SUITE-NAME)

Returns T if a test suite called SUITE-NAME is defined, otherwise returns NIL.

DEFINED-TEST-P (TEST-NAME)

Returns T if a test case called TEST-NAME is defined, otherwise returns NIL.

GET-CHILD-SUITES (SUITE-NAME)

Returns a list of all test suite names that are children of the suite called SUITE-NAME.

GET-CHILD-TESTS (SUITE-NAME)

Returns a list of all test case names that are children of the suite called SUITE-NAME.

GET-DEFINED-SUITES

Returns a list of all defined test suite names.

GET-DEFINED-TESTS

Returns a list of all defined test case names.

RERUN-FAILED-TESTS (&KEY USE-DEBUGGER (REPORT-PROGRESS T) STOP-ON-FAIL (LAST-REPORT *LAST-CLUNIT-REPORT*))

Reruns all failed tests recorded in LAST-REPORT which defaults to the report of the previous test run. If REPORT-PROGRESS is non-NIL, the test progress is reported. If USE-DEBUGGER is non-NIL, the debugger is invoked whenever an assertion fails. If STOP-ON-FAIL is non-NIL, the rest of the unit test is cancelled when any assertion fails or an error occurs.

RUN-SUITE (SUITE &KEY USE-DEBUGGER (REPORT-PROGRESS T) STOP-ON-FAIL)

Executes a test case called SUITE. If REPORT-PROGRESS is non-NIL, the test progress is reported. If USE-DEBUGGER is non-NIL, the debugger is invoked whenever an assertion fails. If STOP-ON-FAIL is non-NIL, the rest of the unit test is cancelled when any assertion fails or an error occurs.

RUN-TEST (TEST &KEY USE-DEBUGGER (REPORT-PROGRESS T) STOP-ON-FAIL)

Executes a test case called TEST. If REPORT-PROGRESS is non-NIL, the test progress is reported. If USE-DEBUGGER is non-NIL, the debugger is invoked whenever an assertion fails. If STOP-ON-FAIL is non-NIL, the rest of the unit test is cancelled when any assertion fails or an error occurs.

Undocumented

ASSERT-FAIL (FORMAT-STRING &REST ARGS)

Private

ASSERTION-EXPANDER (&KEY RESULT TEST RESULT-EXPRESSION REPORT-EXPRESSION EXPECTED FORMS)

Expands an assertion macro call.

DELETE-TEST-CASE (NAME)

Deletes the TEST-CASE instance associated with the key NAME in the hash table *TEST-CASES*

DELETE-TEST-SUITE (NAME)

Deletes the TEST-SUITE instance associated with the key NAME in the hash table *TEST-SUITES*

GET-TEST-CASE (NAME)

Retrieves the TEST-CASE instance associated with the key NAME in the hash table *TEST-CASES*

SETFGET-TEST-CASE (NEW-TEST-CASE NAME)

Adds NEW-TEST-CASE in the hash table *TEST-CASES* under the key NAME.

GET-TEST-SUITE (NAME)

Retrieves the TEST-SUITE instance associated with the key NAME in the hash table *TEST-SUITES*

SETFGET-TEST-SUITE (NEW-TEST-SUITE NAME)

Adds NEW-TEST-SUITE in the hash table *TEST-SUITES* under the key NAME.

HANDLE-ASSERTION (CONDITION)

Records the result of assertion tests and records any errors that occur.

TEST-CASE-EXECUTION-ACTION (TEST-CASE)

Determines the execution action for TEST-CASE. If test case has no dependencies or pending dependencies, then :RUN is returned. If test case has another test case it depends on that failed or was skipped, then :SKIP is returned. If test case depends on test cases that have not yet run or are also queued, then :QUEUE is returned.

Undocumented

EXECUTE-TEST-CASE (TEST-CASE)

EXECUTE-TEST-SUITE (TEST-SUITE)

FORM-EXPANDER (FORMS)

HANDLE-ERROR (ERROR)

PROCESS-QUEUED-TESTS

QUEUE-TEST-CASE

REPORT-ASSERTION-PROGRESS (TYPE)

REPORT-SUITE-PROGRESS (SUITE SUITE-LIST)

REPORT-TEST-PROGRESS (TEST-NAME SUITE-LIST)

SIGNAL-ASSERTION (TYPE &KEY EXPRESSION EXPECTED RETURNED FORMS MESSAGE (FORMAT-STRING ) ARGS)

SKIP-TEST-CASE

MACRO

Public

ASSERT-CONDITION (CONDITION EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if EXPRESSION signals CONDITION. FORMS and their values are printed if the test fails.

ASSERT-EQ (VALUE EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (EQ VALUE EXPRESSION) returns true. FORMS and their values are printed if the test fails.

ASSERT-EQL (VALUE EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (EQL VALUE EXPRESSION) returns true. FORMS and their values are printed if the test fails.

ASSERT-EQUAL (VALUE EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (EQUAL VALUE EXPRESSION) returns true. FORMS and their values are printed if the test fails.

ASSERT-EQUALITY (TEST VALUE EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (FUNCALL TEST VALUE EXPRESSION) returns true. FORMS and their values are printed if the test fails.

ASSERT-EQUALITY* (VALUE EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (FUNCALL *clunit-equality-test* VALUE EXPRESSION) returns true. FORMS and their values are printed if the test fails.

ASSERT-EQUALP (VALUE EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (EQUALP VALUE EXPRESSION) returns true. FORMS and their values are printed if the test fails.

ASSERT-EXPANDS (&ENVIRONMENT ENV EXPANSION EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if (EQUALP EXPANSION (MACROEXPAND-1 EXPRESSION)) returns true. FORMS and their values are printed if the test fails.

ASSERT-FALSE (EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if it returns false. FORMS and their values are printed if the test fails.

ASSERT-TRUE (EXPRESSION &BODY FORMS)

Evaluates EXPRESSION as an assertion, an assertion passes if it returns any non-NIL value. FORMS and their values are printed if the test fails. Remember in Common Lisp any non-NIL value is true, if you want a strict binary assertion test use (assert-eq t expression) instead.

DEFFIXTURE (SUITE (PLUG) &BODY BODY)

A fixture defines a code template that is wrapped around the code of each test case and test suite that are executed by test suite SUITE at runtime.. The test case body is plugged into the template at the position identified by PLUG. Fixtures are expanded at runtime, so the fixture that will wrap around a test depends on the test suite call stack.

DEFSUITE (NAME PARENTS)

Defines a test suite called NAME. If PARENTS is non-NIL the test suite is defined as a sub-suite of each of the test suites in PARENTS.

DEFTEST (NAME DECLARATIONS &BODY BODY)

Defines a test case called NAME. DECLARATIONS declares which test suites this test case is associated with as well as any other test cases that it depends on. The test case body is revaluated on each run, so any redefinition of macros and inline functions will be automatically visible without having to redefine the test.

UNDEFFIXTURE (NAME)

Remove a fixture definition.

Undocumented

UNDEFSUITE (NAME)

UNDEFTEST (NAME)

Private

WITH-ASSERT-RESTART (&BODY BODY)

WITH-ASSERT-RESTART places a restart called SKIP-ASSERTION around an assertion form.

WITH-GENSYMS (SYMS &BODY BODY)

WITH-GENSYMS takes a list of symbols SYM and a list of body forms. It returns a LET clause with each symbol in SYM bound to a gensymbol and the body forms as the body of the let clause.

Undocumented

WITH-TEST-RESTART (&BODY BODY)

GENERIC-FUNCTION

Private

EXPAND-FIXTURE (SUITE BODY)

Expands out a fixture body for the given test suite.

SLOT-ACCESSOR

Public

Undocumented

TEST-REPORT-NAME (OBJECT)

TEST-REPORT-PASSED-P (OBJECT)

TEST-REPORTS (OBJECT)

VARIABLE

Public

*CLUNIT-EQUALITY-TEST*

The equality test used to compare values by the ASSERT-EQUALITY macro.

*CLUNIT-REPORT-FORMAT*

Controls the output format of the unit test results. Possible values are :default, :tap or NIL.

Private

*CLUNIT-REPORT*

Holds a CLUNIT-REPORT object during a test run, used to store aggregated test reports.

*CLUNIT-TEST-REPORT*

Holds a CLUNIT-TEST-REPORT object for each executing test case.

*LAST-CLUNIT-REPORT*

Holds the last returned CLUNIT-REPORT object. Default optional argument to the function RERUN-FAILED-TESTS.

*QUEUED-TEST-REPORTS*

A list of queued CLUNIT-TEST-REPORT objects whose test case execution has been deferred until their dependencies requirements are met.

*REPORT-PROGRESS*

This variable switches on progress reporting if set to true.

*STOP-ON-FAIL*

If any assertion fails or an error condition occurs, stop the unit test.

*SUITE-NAME*

Holds a list of suite symbol names in their current calling order. First called suite at the front, currently executing suite at the back.

*TAB-WIDTH*

Number of tab columns to use for indentation (in the reporting functions).

*TAP-TEST-NUM*

Counter used in these printing functions

*TEST-CASE-HASHTABLE*

Holds CLUNIT-TEST-CASE instances. The key of each test case object is the symbol name.

*TEST-NAME*

Holds the name of the test currently executing.

*TEST-SUITE-HASHTABLE*

Holds CLUNIT-TEST-SUITE instances. The key of each test suite object is the symbol name.

*USE-DEBUGGER*

If set to true, the debugger is invoked whenever an assertion fails.

CLASS

Public

Undocumented

CLUNIT-REPORT

CLUNIT-TEST-REPORT

Private

Undocumented

CLUNIT-TEST-CASE

CLUNIT-TEST-SUITE

CONDITION

Private

Undocumented

ASSERTION-CONDITION

ASSERTION-ERROR

ASSERTION-FAIL-FORCED

ASSERTION-FAILED

ASSERTION-PASSED