Common Lisp Package: LISP-UNIT

README:

## lisp-unit *lisp-unit* is a Common Lisp library that supports unit testing. It is an extension of the [library written by Chris Riesbeck][orig]. There is a long history of testing packages in Lisp, usually called "regression" testers. More recent packages in Lisp and other languages have been inspired by [JUnit for Java][JUnit]. [Documentation is located on the project wiki.][wiki] ### Features * Written in portable Common Lisp * Loadable as a single file * Loadable with [ASDF][] or [Quicklisp][] * Simple to define and run tests * Redfine functions and macros without reloading tests * Test return values, printed output, macro expansions, and conditions * Fined grained control over the testing output * Store all test results in a database object that can be examined * Group tests by package for modularity * Group tests using tags * Signal test completion and return results with the condition. ### Extensions * Floating point predicates * [Test Anything Protocol][TAP] output ### How to use lisp-unit The core definitions of *lisp-unit* may be used by loading the single file 'lisp-unit.lisp'. To use the extensions, *lisp-unit* must be loaded using either [Quicklisp][] or [ASDF][]. 1. Load (or compile and load) as a single file : `(load "lisp-unit")`. 2. Load using [Quicklisp][] : `(ql:quickload :lisp-unit)`. 3. Load using [ASDF][] : `(asdf:load-system :lisp-unit)`. ## Version 0.9.5 Features No new features, improved the usability based on user feedback. The list of tests or tags to the following functions is now optional and defaults to `:ALL`. * `(remove-tests [names] [package])` * `(tagged-tests [tags] [package])` * `(remove-tags [tags] [package])` * `(run-tests [names] [package])` * `(run-tags [tags] [package])` ## Version 1 Remaining Tasks * (1.0.0) Expanded internal testing. ### Future Features * Fixtures * Test Suites * Benchmarking tools [orig]: <http://www.cs.northwestern.edu/academics/courses/325/readings/lisp-unit.html> "Original Lisp Unit" [wiki]: <https://github.com/OdonataResearchLLC/lisp-unit/wiki> "Lisp Unit Wiki" [JUnit]: <http://www.junit.org> "JUnit" [Quicklisp]: <http://www.quicklisp.org> "Quicklisp" [ASDF]: <http://common-lisp.net/project/asdf/> "ASDF" [TAP]: <http://testanything.org/> "Test Anything Protocol" ## 0.9.5 Acknowledgments * [Jesse Alama][jessealama] for usability feedback. [jessealama]: <https://github.com/jessealama> "Jesse Alama"

FUNCTION

Public

ARRAY-ERROR (ARRAY1 ARRAY2 &KEY (TEST #'NUMBER-EQUAL) (ERROR-FUNCTION #'RELATIVE-ERROR))

Return a list of the indices and error between the array elements.

COMPLEX-RANDOM (LIMIT &OPTIONAL (STATE *RANDOM-STATE*))

Return a random complex number.

LIST-TAGS (&OPTIONAL (PACKAGE *PACKAGE*))

Return a list of the tags in package.

LIST-TESTS (&OPTIONAL (PACKAGE *PACKAGE*))

Return a list of the tests in package.

LOGICALLY-EQUAL (X Y)

Return true if x and y are both false or both true.

MAKE-2D-LIST (ROWS COLUMNS &KEY (INITIAL-ELEMENT 0))

Return a nested list with INITIAL-ELEMENT.

MAKE-RANDOM-2D-ARRAY (ROWS COLUMNS &OPTIONAL (LIMIT 1.0))

Return a 2D array of random numbers.

MAKE-RANDOM-2D-LIST (ROWS COLUMNS &OPTIONAL (LIMIT 1.0))

Return a nested list of random numbers.

MAKE-RANDOM-LIST (SIZE &OPTIONAL (LIMIT 1.0))

Return a list of random numbers.

NUMBER-EQUAL (NUMBER1 NUMBER2 &OPTIONAL (EPSILON *EPSILON*) TYPE-EQ-P)

Return true if the numbers are equal within some epsilon, optionally requiring the types to be identical.

REMOVE-TAGS (&OPTIONAL (TAGS ALL) (PACKAGE *PACKAGE*))

Remove individual tags or entire sets.

REMOVE-TESTS (&OPTIONAL (NAMES ALL) (PACKAGE *PACKAGE*))

Remove individual tests or entire sets.

RUN-TAGS (&OPTIONAL (TAGS ALL) (PACKAGE *PACKAGE*))

Run the tests associated with the specified tags in package.

RUN-TESTS (&OPTIONAL (TEST-NAMES ALL) (PACKAGE *PACKAGE*))

Run the specified tests in package.

SET-EQUAL (LIST1 LIST2 &REST INITARGS &KEY KEY (TEST #'EQUAL))

Return true if every element of list1 is an element of list2 and vice versa.

SIGNAL-RESULTS (&OPTIONAL (FLAG T))

Signal the results for extensibility.

SUMMARIZE-RESULTS (RESULTS &OPTIONAL (STREAM *STANDARD-OUTPUT*))

Print a summary of all results to the stream.

TAGGED-TESTS (&OPTIONAL (TAGS ALL) (PACKAGE *PACKAGE*))

Return a list of the tests associated with the tags.

TEST-CODE (NAME &OPTIONAL (PACKAGE *PACKAGE*))

Returns the code stored for the test name.

TEST-DOCUMENTATION (NAME &OPTIONAL (PACKAGE *PACKAGE*))

Return the documentation for the test.

TEST-NAMES (TEST-RESULTS-DB)

Return a list of the test names in the database.

USE-DEBUGGER (&OPTIONAL (FLAG T))

Use the debugger when testing, or not.

WRITE-TAP (TEST-RESULTS &OPTIONAL (STREAM *STANDARD-OUTPUT*))

Write the test results to `stream` in TAP format. Returns the test results.

WRITE-TAP-TO-FILE (TEST-RESULTS PATH)

write the test results to `path` in TAP format, overwriting `path`. Returns pathname to the output file

Private

%ARRAY-ERROR (ARRAY1 ARRAY2 TEST ERRFUN)

Return a list of the indices, values and error of the elements that are not equal.

%ARRAY-INDICES (ROW-MAJOR-INDEX DIMENSIONS)

Recursively calculate the indices from the row major index.

%COMPLEX-FLOAT-RANDOM (LIMIT &OPTIONAL (STATE *RANDOM-STATE*))

Return a random complex float number.

%COMPLEX-RATIONAL-RANDOM (LIMIT &OPTIONAL (STATE *RANDOM-STATE*))

Return a random complex rational number.

%EXPANSION-EQUAL (FORM1 FORM2)

Descend into the forms checking for equality.

%FLOAT-EQUAL (DATA1 DATA2 EPSILON)

Return true if the relative error between the data is less than epsilon.

%NORM-EQUAL (SEQ1 SEQ2 EPSILON MEASURE)

Return true if the relative error norm is less than epsilon.

%NORMALIZE-FLOAT (SIGNIFICAND &OPTIONAL (EXPONENT 0))

Return the normalized floating point number and exponent.

%RELATIVE-ERROR (EXACT APPROXIMATE)

Return the relative error of the numbers.

%RELATIVE-ERROR-NORM (EXACT APPROXIMATE MEASURE)

Return the relative error norm of the sequences.

%RUN-ALL-THUNKS (&OPTIONAL (PACKAGE *PACKAGE*))

Run all of the test thunks in the package.

%RUN-THUNKS (TEST-NAMES &OPTIONAL (PACKAGE *PACKAGE*))

Run the list of test thunks in the package.

%SEQ-FLOAT-EQUAL (SEQ1 SEQ2 EPSILON)

Return true if the element-wise comparison of relative error is less than epsilon.

%SEQ-RATIONAL-EQUAL (SEQ1 SEQ2)

Return true if the sequences are equal length and at each position the corresponding elements are equal.

%SEQ-SIGFIG-EQUAL (SEQ1 SEQ2 SIGNIFICANT-FIGURES)

Return true if the element-wise comparison is equal to the specified significant figures.

%SEQUENCE-EQUAL (SEQ1 SEQ2 TEST)

Return true if the sequences are equal in length and each element is equal according to :TEST.

%SEQUENCE-ERROR (SEQUENCE1 SEQUENCE2 TEST ERROR-FUNCTION)

Return a sequence of the indice and error between the sequences.

%SIGFIG-EQUAL (FLOAT1 FLOAT2 SIGNIFICANT-FIGURES)

Return true if the floating point numbers have equal significant figures.

%TESTS-FROM-ALL-TAGS (&OPTIONAL (PACKAGE *PACKAGE*))

Return all of the tests that have been tagged.

%TESTS-FROM-TAGS (TAGS &OPTIONAL (PACKAGE *PACKAGE*))

Return the tests associated with the tags.

%WRITE-TAP-TEST-RESULT (NAME TEST-RESULT I STREAM)

Output a single test, taking care to ensure the indentation level is the same before and after invocation.

INTERNAL-ASSERT (TYPE FORM CODE-THUNK EXPECTED-THUNK EXTRAS TEST)

Perform the assertion and record the results.

PACKAGE-TAGS (PACKAGE &OPTIONAL CREATE)

Return the tags DB for the package.

PARSE-BODY (BODY &OPTIONAL DOC TAG)

Separate the components of the body.

RECORD-RESULT (TEST-NAME CODE RESULTS)

Run the test code and record the result.

RESET-COUNTERS

Reset the counters to empty lists.

RUN-CODE (CODE)

Run the code to test the assertions.

RUN-TIME-S (TEST-RESULT)

calculate the run-time of the test in seconds

SEQUENCE-ERROR (SEQUENCE1 SEQUENCE2 &KEY (TEST #'NUMBER-EQUAL) (ERROR-FUNCTION #'RELATIVE-ERROR))

Return a sequence of the indice and error between the sequence elements.

TEST-NAME-ERROR-REPORT (TEST-NAME-ERROR STREAM)

Write the test-name-error to the stream.

USE-DEBUGGER-P (CONDITION)

Debug or ignore errors.

VALID-TEST-NAME (NAME)

Signal a type-error if the test name is not a symbol.

Undocumented

PACKAGE-TABLE (PACKAGE &OPTIONAL CREATE)

RUN-TEST-THUNK (NAME CODE)

MACRO

Public

ASSERT-EQ (EXPECTED FORM &REST EXTRAS)

Assert whether expected and form are EQ.

ASSERT-EQL (EXPECTED FORM &REST EXTRAS)

Assert whether expected and form are EQL.

ASSERT-EQUAL (EXPECTED FORM &REST EXTRAS)

Assert whether expected and form are EQUAL.

ASSERT-EQUALITY (TEST EXPECTED FORM &REST EXTRAS)

Assert whether expected and form are equal according to test.

ASSERT-EQUALP (EXPECTED FORM &REST EXTRAS)

Assert whether expected and form are EQUALP.

ASSERT-ERROR (CONDITION FORM &REST EXTRAS)

Assert whether form signals condition.

ASSERT-EXPANDS (EXPANSION FORM &REST EXTRAS)

Assert whether form expands to expansion.

ASSERT-FALSE (FORM &REST EXTRAS)

Assert whether the form is false.

ASSERT-PRINTS (OUTPUT FORM &REST EXTRAS)

Assert whether printing the form generates the output.

ASSERT-TRUE (FORM &REST EXTRAS)

Assert whether the form is true.

DEFINE-TEST (NAME &BODY BODY)

Store the test in the test database.

Undocumented

ASSERT-FLOAT-EQUAL (EXPECTED FORM &REST EXTRAS)

ASSERT-NORM-EQUAL (EXPECTED FORM &REST EXTRAS)

ASSERT-NUMBER-EQUAL (EXPECTED FORM &REST EXTRAS)

ASSERT-NUMERICAL-EQUAL (EXPECTED FORM &REST EXTRAS)

ASSERT-RATIONAL-EQUAL (EXPECTED FORM &REST EXTRAS)

ASSERT-SIGFIG-EQUAL (EXPECTED FORM &REST EXTRAS)

Private

EXPAND-ASSERT (TYPE FORM BODY EXPECTED EXTRAS &KEY (TEST '#'EQL))

Expand the assertion to the internal format.

EXPAND-ERROR-FORM (FORM)

Wrap the error assertion in HANDLER-CASE.

EXPAND-EXTRAS (EXTRAS)

Expand extra forms.

EXPAND-MACRO-FORM (FORM ENV)

Expand the macro form once.

EXPAND-OUTPUT-FORM (FORM)

Capture the output of the form in a string.

GENERIC-FUNCTION

Public

DEFAULT-EPSILON (VALUE)

Return the default epsilon for the value.

FLOAT-EQUAL (DATA1 DATA2 &OPTIONAL EPSILON)

Return true if the floating point data is equal.

NORM (DATA &OPTIONAL MEASURE)

Return the element-wise norm of the data.

NORM-EQUAL (DATA1 DATA2 &OPTIONAL EPSILON MEASURE)

Return true if the norm of the data is equal.

NUMERICAL-EQUAL (RESULT1 RESULT2 &KEY TEST (TEST #'NUMBER-EQUAL))

Return true if the results are numerically equal according to :TEST.

RATIONAL-EQUAL (DATA1 DATA2)

Return true if the rational data is equal.

RELATIVE-ERROR (EXACT APPROXIMATE)

Return the relative-error between the 2 quantities.

RELATIVE-ERROR-NORM (EXACT APPROXIMATE &OPTIONAL MEASURE)

Return the relative error norm

SIGFIG-EQUAL (DATA1 DATA2 &OPTIONAL SIGNIFICANT-FIGURES)

Return true if the data have equal significant figures.

SUMP (DATA P)

Return the scaling parameter and the sum of the powers of p of the ~ data.

SUMSQ (DATA)

Return the scaling parameter and the sum of the squares of the ~ data.

Undocumented

RESULTS (CONDITION)

Private

%NORM (DATA MEASURE)

Return the norm of the data according to measure.

ASSERT-RESULT (TYPE TEST EXPECTED ACTUAL)

Return the result of the assertion.

RECORD-FAILURE (TYPE FORM ACTUAL EXPECTED EXTRAS TEST)

Record the details of the failure.

SLOT-ACCESSOR

Public

Undocumented

ERROR-TESTS (OBJECT)

SETFERROR-TESTS (NEW-VALUE OBJECT)

FAILED-TESTS (OBJECT)

SETFFAILED-TESTS (NEW-VALUE OBJECT)

MISSING-TESTS (OBJECT)

SETFMISSING-TESTS (NEW-VALUE OBJECT)

Private

RUN-TIME (OBJECT)

Test run time measured in internal time units

Undocumented

ACTUAL (OBJECT)

CODE (OBJECT)

DATABASE (OBJECT)

DOC (OBJECT)

EXERR (OBJECT)

SETFEXERR (NEW-VALUE OBJECT)

EXPECTED (OBJECT)

EXTRAS (OBJECT)

FAIL (OBJECT)

SETFFAIL (NEW-VALUE OBJECT)

FORM (OBJECT)

NAME (OBJECT)

PASS (OBJECT)

SETFPASS (NEW-VALUE OBJECT)

TEST (OBJECT)

VARIABLE

Public

*EPSILON*

Set the error epsilon if the defaults are not acceptable.

*PRINT-ERRORS*

Print error messages if non-NIL.

*PRINT-FAILURES*

Print failure messages if non-NIL.

*PRINT-SUMMARY*

Print a summary of the pass, fail, and error count if non-nil.

*SIGNIFICANT-FIGURES*

Default to 4 significant figures.

Undocumented

*MEASURE*

Private

*FAIL*

The failed assertion results.

*PASS*

The passed assertion results.

*SIGNAL-RESULTS*

Signal the result if non NIL.

*TAG-DB*

The tag database is simply a hash table.

*TEST-DB*

The unit test database is simply a hash table.

*USE-DEBUGGER*

If not NIL, enter the debugger when an error is encountered in an assertion.

CLASS

Private

BOOLEAN-RESULT

Result of a failed boolean assertion.

EQUAL-RESULT

Result of a failed equal assertion.

ERROR-RESULT

Result of a failed error assertion.

FAILURE-RESULT

Failure details of the assertion.

MACRO-RESULT

Result of a failed macro expansion assertion.

OUTPUT-RESULT

Result of a failed output assertion.

TEST-RESULT

Store the results of the unit test.

TEST-RESULTS-DB

Store the results of the tests for further evaluation.

UNIT-TEST

Organize the unit test documentation and code.

CONDITION

Public

TEST-RUN-COMPLETE

Signaled when a test run is finished.

Private

TEST-NAME-ERROR

The test name error is a type error.