Common Lisp Package: SIFT.NST

NST unit testing package

README:

FUNCTION

Public

ADD-ERROR (RESULT &KEY FORMAT ARGS)

The add-error function adds an error note to a result record. (ADD-ERROR RESULT-REPORT [ :FORMAT FORMAT-STRING ] [ :ARGS ARGUMENT-LIST ])

ADD-FAILURE (RESULT &KEY FORMAT ARGS)

The add-failure function adds a failure note to a result record. (ADD-FAILURE RESULT-REPORT [ :FORMAT FORMAT-STRING ] [ :ARGS ARGUMENT-LIST ])

ADD-INFO (RESULT ITEM)

The add-info function adds auxiliary information to a result record. (ADD-INFO RESULT-REPORT INFO-ITEM)

ASSERT-EQ (G371 G372 &REST G373 &KEY (FORMAT NIL G374) (FORMAT-ARGS NIL G375) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-EQ function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQ , and succeeds whenever that call returns non-nil. (ASSERT-EQ EXPECTED-VALUE TESTED-VALUE)

ASSERT-EQL (G436 G437 &REST G438 &KEY (FORMAT NIL G439) (FORMAT-ARGS NIL G440) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-EQL function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQL , and succeeds whenever that call returns non-nil. (ASSERT-EQL EXPECTED-VALUE TESTED-VALUE)

ASSERT-EQUAL (G501 G502 &REST G503 &KEY (FORMAT NIL G504) (FORMAT-ARGS NIL G505) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-EQUAL function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQUAL , and succeeds whenever that call returns non-nil. (ASSERT-EQUAL EXPECTED-VALUE TESTED-VALUE)

ASSERT-EQUALP (G566 G567 &REST G568 &KEY (FORMAT NIL G569) (FORMAT-ARGS NIL G570) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-EQUALP function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQUALP , and succeeds whenever that call returns non-nil. (ASSERT-EQUALP EXPECTED-VALUE TESTED-VALUE)

ASSERT-NON-NIL (G308 &REST G309 &KEY (FORMAT NIL G310) (FORMAT-ARGS NIL G311) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-NON-NIL function is a unary predicate for use within the forms evaluated for an EVAL criterion. It succeeds whenever the NULL function returns nil. (ASSERT-NON-NIL TESTED-VALUE)

ASSERT-NOT-EQ (G631 G632 &REST G633 &KEY (FORMAT NIL G634) (FORMAT-ARGS NIL G635) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-NOT-EQ function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQ , and succeeds whenever that call returns nil. (ASSERT-NOT-EQ EXPECTED-VALUE TESTED-VALUE)

ASSERT-NOT-EQL (G696 G697 &REST G698 &KEY (FORMAT NIL G699) (FORMAT-ARGS NIL G700) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-NOT-EQL function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQL , and succeeds whenever that call returns nil. (ASSERT-NOT-EQL EXPECTED-VALUE TESTED-VALUE)

ASSERT-NOT-EQUAL (G761 G762 &REST G763 &KEY (FORMAT NIL G764) (FORMAT-ARGS NIL G765) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-NOT-EQUAL function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQUAL , and succeeds whenever that call returns nil. (ASSERT-NOT-EQUAL EXPECTED-VALUE TESTED-VALUE)

ASSERT-NOT-EQUALP (G826 G827 &REST G828 &KEY (FORMAT NIL G829) (FORMAT-ARGS NIL G830) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-NOT-EQUALP function is a binary predicate for use within the forms evaluated for an EVAL criterion. It compares the expected and tested values using EQUALP , and succeeds whenever that call returns nil. (ASSERT-NOT-EQUALP EXPECTED-VALUE TESTED-VALUE)

ASSERT-NULL (G182 &REST G183 &KEY (FORMAT NIL G184) (FORMAT-ARGS NIL G185) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-NULL function is a unary predicate for use within the forms evaluated for an EVAL criterion. It succeeds whenever the NULL function returns non-nil. (ASSERT-NULL TESTED-VALUE)

ASSERT-ZERO (G245 &REST G246 &KEY (FORMAT NIL G247) (FORMAT-ARGS NIL G248) (FATAL NIL) &ALLOW-OTHER-KEYS)

The ASSERT-ZERO function is a unary predicate for use within the forms evaluated for an EVAL criterion. It succeeds whenever the ZEROP function returns non-nil. (ASSERT-ZERO TESTED-VALUE)

CHECK-CRITERION-ON-FORM (CRITERION FORM)

The check-criterion-on-form function verifies that an unevaluated form adheres to a criterion. (CHECK-CRITERION-ON-FORM CRITERION FORM)

CHECK-CRITERION-ON-VALUE (CRITERION VAL)

The check-criterion-on-value function can be called from within a criterion body to verify that a value adheres to a criterion. (CHECK-CRITERION-ON-VALUE CRITERION VALUE)

JUNIT-RESULTS-BY-GROUP (&REST ARGS &KEY VERBOSE &ALLOW-OTHER-KEYS)

The junit-results-by-group function writes the NST test results in JUnit XML format, organized by group, aligning test groups with Java classes, and individual tests with @Test methods. (JUNIT-RESULTS-BY-GROUP [ :VERBOSE FLAG ] [ :DIR DIRECTORY ] [ :FILE FILESPEC ] [ :STREAM STREAM ] [ :IF-DIR-DOES-NOT-EXIST BOOL ] [ :IF-FILE-EXISTS BOOL ]) Either :dir and :file options, or the :stream option, but not both, should be used to specify the target for XML output; if none of the three options are given, the function will write to *standard-output*.

MAKE-ERROR-REPORT (E &REST FORMAT-ARGS)

Function make-error-report produces a report of an error during test execution. (MAKE-ERROR-REPORT [ :FORMAT FORMAT-STRING ] [ :ARGS ARG-FORM-LIST ])

MAKE-FAILURE-REPORT (&KEY FORMAT ARGS INFO)

The make-failure-report function returns a report of test failure. (MAKE-FAILURE-REPORT [ :FORMAT FORMAT-STRING ] [ :ARGS ARG-FORM-LIST ]) The format-string and args are as to the Common Lisp function format. The emit-failure function is an older, deprecated version of this function.

MAKE-SUCCESS-REPORT (&REST ARGS &KEY WARNINGS INFO)

The make-success-report function indicates a successful test result. (MAKE-SUCCESS-REPORT ) Note that some older examples show (make-check-result), (emit-success) or (check-result). The former is an internal function and should not be used from outside the core NST files. The latter two are deprecated.

MAKE-WARNING-REPORT (&KEY FORMAT ARGS)

Function make-warning-report is like make-failure-report, but provides supplimentary information as a warning. (MAKE-WARNING-REPORT [ :FORMAT FORMAT-STRING ] [ :ARGS ARG-FORM-LIST ]) The emit-warning function is an older, deprecated version of this function.

WRAP-THROWN-LISP-WARNING (W)

The helper function WRAP-THROWN-LISP-WARNING creates an NST CHECK-NOTE object from a standard Lisp WARNING .

Private

CDR-OR-NIL (NAME-OR-NAME-AND-ARGS)

Return the cdr given a list, or return nil if given a symbol.

CHECK-NOTE-ARGS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-NOTE-CONTEXT (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-NOTE-FORMAT (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-NOTE-STACK (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-CHECK-NAME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-ELAPSED-TIME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-ERRING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-ERRORS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-FAILING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-FAILURES (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-GROUP-NAME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-INFO (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-PASSING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-TESTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-TIMESTAMP (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-WARNING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CHECK-RESULT-WARNINGS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

CORE-RUN-TEST (TEST)

Capture the result of the test.

DECODE-DEFCHECK-NAME-AND-ARGS (NAME-OR-NAME-AND-ARGS)

This function unpacks the information inside the first form of a def-test block, which can be either a single symbol naming the test, or a list whose first element is that symbol and whose remaining elements are options.

EQL-FOR-SIGDIGITS (DIGITS N1 N2)

Test whether two numbers are eql to the given number of significant digits.

ERROR-CHECK-NOTE-ARGS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

ERROR-CHECK-NOTE-CONTEXT (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

ERROR-CHECK-NOTE-ERROR (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

ERROR-CHECK-NOTE-FORMAT (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

ERROR-CHECK-NOTE-STACK (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GET-NAME-USE-RECORD (NAME)

Internal function for use within record-name-use --- this is not the function you want to use to read +name-use+.

GROUP-REPORT (GROUP)

Top-level function for reporting the results of a group.

GROUP-RESULT-CHECK-RESULTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-ELAPSED-TIME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-ERRING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-FAILING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-GROUP-NAME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-PASSING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-TESTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-TIMESTAMP (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

GROUP-RESULT-WARNING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

LAMBDA-LIST-NAMES (LAMBDA-LIST SUPP-P)

Pick out the names from a lambda-list, omitting the ampersand-prefixed delimiters.

LOOKUP-ARTIFACT (NAME)

Input is a symbol, output is a list.

MAKE-CHECK-RESULT (&KEY (GROUP-NAME *NST-GROUP-NAME*) (CHECK-NAME *NST-CHECK-USER-NAME*) WARNINGS FAILURES ERRORS INFO (TESTS 0) (PASSING 0) (ERRING 0) (FAILING 0) (WARNING 0) (ELAPSED-TIME 0) (TIMESTAMP (MULTIPLE-VALUE-LIST (GET-DECODED-TIME))))

Functional wrapper around the constructor for check-result structure, permitting the use of apply.

MAKE-PACKAGE-DOCUMENTATION

Write documentation for this package, using system package-doc.

MULTI-RESULTS-ELAPSED-TIME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-ERRING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-FAILING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-GROUP-REPORTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-PACKAGE-REPORTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-PASSING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-STATS-SOURCE (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-SYSTEM (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-TEST-REPORTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-TESTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-TIMESTAMP (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

MULTI-RESULTS-WARNING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

NAME-USE-FIXTURE (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

NAME-USE-GROUP (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

NAME-USE-TESTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

NOTE-ARTIFACT (NAME ARTIFACT)

Standardize the name (a string), and file the artifact.

NST-DUMP (&KEY (STREAM *NST-OUTPUT-STREAM*) (VERBOSITY *DEFAULT-REPORT-VERBOSITY*))

Spit out the full NST state.

PACKAGE-REPORT (&OPTIONAL (PACKAGE *PACKAGE*))

Top-level function for reporting the results of a package.

PACKAGE-RESULT-ELAPSED-TIME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-ERRING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-FAILING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-GROUP-RESULTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-PACKAGE-NAME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-PASSING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-TESTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-TIMESTAMP (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PACKAGE-RESULT-WARNING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

PROCESS-FIXTURE-LIST (FIXTURE-SET-LIST)

Trivial, for now, because anonymous fixtures are offline.

REPORT-GROUP (GROUP &OPTIONAL (STREAM *NST-OUTPUT-STREAM*) (*NST-VERBOSITY* (MAX *DEFAULT-REPORT-VERBOSITY* *NST-VERBOSITY*)))

Top-level function for reporting the results of the tests in a group.

REPORT-MULTIPLE (PACKAGES GROUPS TESTS &KEY (STREAM *NST-OUTPUT-STREAM*) (VERBOSITY *DEFAULT-REPORT-VERBOSITY*) (SYSTEM NIL SYSTEM-SUPP-P))

Top-level function for reporting the results of several tests.

REPORT-PACKAGE (&OPTIONAL (PACKAGE *PACKAGE*) (STREAM *NST-OUTPUT-STREAM*) (*NST-VERBOSITY* (MAX *DEFAULT-REPORT-VERBOSITY* *NST-VERBOSITY*)))

Top-level function for reporting the results of the tests in a package.

REPORT-TEST (GROUP TEST &OPTIONAL (STREAM *NST-OUTPUT-STREAM*) (*NST-VERBOSITY* (MAX *DEFAULT-REPORT-VERBOSITY* *NST-VERBOSITY*)))

Top-level function for reporting the results of a test.

RESULT-STATS-ELAPSED-TIME (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RESULT-STATS-ERRING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RESULT-STATS-FAILING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RESULT-STATS-PASSING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RESULT-STATS-TESTS (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RESULT-STATS-TIMESTAMP (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RESULT-STATS-WARNING (INSTANCE)

@arg[extid]{A @class{extid}} @return[sytemid]{puri:uri or nil} Returns the System ID part of this External ID.

RUN-GROUP (GROUP-CLASS)

Run a group by its user-given name. Note that this is /not/ an interactive function --- certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.

RUN-GROUP-TESTS (GROUP-OBJ TEST-OBJS)

Programmatic entry point for running all tests in a group.

RUN-PACKAGE (&OPTIONAL (PACKAGE-OR-NAME *PACKAGE*))

Run all groups in a package. Note that this is /not/ an interactive function --- certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.

RUN-TEST (GROUP TEST)

Run a test standalone by its user-given name (and its group's name). Note that this is /not/ an interactive function --- certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.

SIG-PLACE (N VALUE)

Returns the n-th significant place of value

SYMBOL-OR-CAR (NAME-OR-NAME-AND-ARGS)

Return the first element given a list, or return a symbol.

TEST-REPORT (GROUP TEST)

Top-level function for reporting the results of a test.

TRACE-RESULTS

Internal debugging function: dump the results hash.

Undocumented

%MAKE-CHECK-RESULT (&KEY ((TESTS DUM781) 1) ((PASSING DUM782) 0) ((ERRING DUM783) 0) ((FAILING DUM784) 0) ((WARNING DUM785) 0) ((ELAPSED-TIME DUM786) 0) ((TIMESTAMP DUM787) (MULTIPLE-VALUE-LIST (GET-DECODED-TIME))) ((GROUP-NAME DUM788) *NST-GROUP-NAME*) ((CHECK-NAME DUM789) *NST-CHECK-USER-NAME*) ((WARNINGS DUM790) NIL) ((FAILURES DUM791) NIL) ((ERRORS DUM792) NIL) ((INFO DUM793) NIL))

ADD-TEST-CONFIG-ERROR (TEST-OBJ FORMAT &REST ARGS)

ADD-THROWN-ERROR (RESULT E &REST FORMAT-ARGS)

ALL-GROUPS-REPORT

ALL-PACKAGE-REPORT

ALL-TESTS-REPORT

APPLY-CHECK-NOTE-FORMATTER (STREAM CHECK-NOTE)

APPLY-FORMATTER (STREAM FORMATTER ARGS)

ARBITRARY-GENERABLE-TYPES

ASSEMBLE-PROTECTED-OPTION-VALUES (OTHER-VARS)

ASSERT-CRITERION-FN (CRITERION-EXPR VALUES-FORM &KEY MSG-FORMAT MSG-ARGS FATAL FAIL-ON-WARNING)

BUILD-CONTINUE-CHECK-EXPR (CRITERION FORM)

CALIBRATE-CHECK-RESULT (R)

SETFCHECK-NOTE-ARGS (NEW-VALUE INSTANCE)

SETFCHECK-NOTE-CONTEXT (NEW-VALUE INSTANCE)

SETFCHECK-NOTE-FORMAT (NEW-VALUE INSTANCE)

CHECK-NOTE-P (OBJECT)

SETFCHECK-NOTE-STACK (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-CHECK-NAME (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-ELAPSED-TIME (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-ERRING (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-ERRORS (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-FAILING (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-FAILURES (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-GROUP-NAME (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-INFO (NEW-VALUE INSTANCE)

CHECK-RESULT-P (OBJECT)

SETFCHECK-RESULT-PASSING (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-TESTS (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-TIMESTAMP (NEW-VALUE INSTANCE)

CHECK-RESULT-UNION (&REST REPORTS)

SETFCHECK-RESULT-WARNING (NEW-VALUE INSTANCE)

SETFCHECK-RESULT-WARNINGS (NEW-VALUE INSTANCE)

COLLECT-TEST-GENERICS (OBJ)

CONTINUE-CHECK (CRITERION FORM)

COPY-CHECK-NOTE (INSTANCE)

COPY-CHECK-RESULT (INSTANCE)

COPY-ERROR-CHECK-NOTE (INSTANCE)

COPY-GROUP-RESULT (INSTANCE)

COPY-MULTI-RESULTS (INSTANCE)

COPY-NAME-USE (INSTANCE)

COPY-PACKAGE-RESULT (INSTANCE)

COPY-RESULT-STATS (INSTANCE)

DECODE-DEF-TEST-GENERIC-BODY (FORMS)

DECOMPOSE-ARG-VALUES-LAMBDA-LIST (ARGS-FORMALS)

DUMP-PICKER

ELAPSED-TIME-TO-STRING (ELAPSED-TIME)

ENSURE-GROUP-INSTANCE (GROUP)

ENSURE-TEST-INSTANCE (GROUP TEST)

SETFERROR-CHECK-NOTE-ARGS (NEW-VALUE INSTANCE)

SETFERROR-CHECK-NOTE-CONTEXT (NEW-VALUE INSTANCE)

SETFERROR-CHECK-NOTE-ERROR (NEW-VALUE INSTANCE)

SETFERROR-CHECK-NOTE-FORMAT (NEW-VALUE INSTANCE)

ERROR-CHECK-NOTE-P (OBJECT)

SETFERROR-CHECK-NOTE-STACK (NEW-VALUE INSTANCE)

EXTRACT-PARAMETERS (X)

FINISH-MULTIPLE-REPORT (RESULT)

FIXTURE-BINDING-ERROR-NOTE (FIXTURE-NAME VARIABLE-NAME ERROR)

FLAG-FILTER (X)

GENERATE-SAMPLE (DOMAINS)

GET-DISPLAY-CONTEXT-LAYERS (LAYERS)

GET-LOCAL-CRITERION-CONTEXT (CONTEXT-LAYER-LIST)

GET-NAME-USE (NAME)

GET-REPORT-FROM-NAMES (GROUP-OR-PACKAGE GP-SUPP-P TEST TEST-SUPP-P)

GET-UNASSIGNED (P)

SETFGROUP-RESULT-CHECK-RESULTS (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-ELAPSED-TIME (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-ERRING (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-FAILING (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-GROUP-NAME (NEW-VALUE INSTANCE)

GROUP-RESULT-P (OBJECT)

SETFGROUP-RESULT-PASSING (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-TESTS (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-TIMESTAMP (NEW-VALUE INSTANCE)

SETFGROUP-RESULT-WARNING (NEW-VALUE INSTANCE)

INTERESTING-RESULT-P (RESULT)

INVOKE-TEST-METHODS (OBJ)

JUNIT-HEADER (STREAM)

LOW-BIASED-NATNUM (&OPTIONAL (BIAS 1))

MAKE-AND-CALIBRATE-CHECK-RESULT (&REST ARGS)

MAKE-CHECK-NOTE (&KEY ((CONTEXT DUM1246) NIL) ((STACK DUM1247) NIL) ((FORMAT DUM1248) NIL) ((ARGS DUM1249) NIL))

MAKE-CONFIG-ERROR (ERROR TEST-OBJ MSG)

MAKE-ERROR-CHECK-NOTE (&KEY ((CONTEXT DUM1399) NIL) ((STACK DUM1400) NIL) ((FORMAT DUM1401) NIL) ((ARGS DUM1402) NIL) ((ERROR DUM1403) NIL))

MAKE-ERROR-NOTE (E &REST FORMAT-ARGS &AUX FORMAT ARGS)

MAKE-GROUP-RESULT (&KEY ((TESTS DUM574) 0) ((PASSING DUM575) 0) ((ERRING DUM576) 0) ((FAILING DUM577) 0) ((WARNING DUM578) 0) ((ELAPSED-TIME DUM579) 0) ((TIMESTAMP DUM580) (MULTIPLE-VALUE-LIST (GET-DECODED-TIME))) ((GROUP-NAME DUM581) NIL) ((CHECK-RESULTS DUM582) (MAKE-HASH-TABLE TEST 'EQ)))

MAKE-MULTI-RESULTS (&KEY ((TESTS DUM130) 0) ((PASSING DUM131) 0) ((ERRING DUM132) 0) ((FAILING DUM133) 0) ((WARNING DUM134) 0) ((ELAPSED-TIME DUM135) 0) ((TIMESTAMP DUM136) (MULTIPLE-VALUE-LIST (GET-DECODED-TIME))) ((PACKAGE-REPORTS DUM137) NIL) ((GROUP-REPORTS DUM138) NIL) ((TEST-REPORTS DUM139) NIL) ((SYSTEM DUM140) NIL) ((STATS-SOURCE DUM141) NIL))

MAKE-NAME-USE (&KEY ((FIXTURE DUM289) NIL) ((GROUP DUM290) NIL) ((TESTS DUM291) (MAKE-HASH-TABLE TEST 'EQ)))

MAKE-PACKAGE-RESULT (&KEY ((TESTS DUM341) 0) ((PASSING DUM342) 0) ((ERRING DUM343) 0) ((FAILING DUM344) 0) ((WARNING DUM345) 0) ((ELAPSED-TIME DUM346) 0) ((TIMESTAMP DUM347) (MULTIPLE-VALUE-LIST (GET-DECODED-TIME))) ((PACKAGE-NAME DUM348) NIL) ((GROUP-RESULTS DUM349) (MAKE-HASH-TABLE TEST 'EQ)))

MAKE-RESULT-STATS (&KEY ((TESTS DUM181) 0) ((PASSING DUM182) 0) ((ERRING DUM183) 0) ((FAILING DUM184) 0) ((WARNING DUM185) 0) ((ELAPSED-TIME DUM186) 0) ((TIMESTAMP DUM187) (MULTIPLE-VALUE-LIST (GET-DECODED-TIME))))

MID-BIASED-NATNUM (MID &OPTIONAL (BIAS 1))

SETFMULTI-RESULTS-ELAPSED-TIME (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-ERRING (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-FAILING (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-GROUP-REPORTS (NEW-VALUE INSTANCE)

MULTI-RESULTS-P (OBJECT)

SETFMULTI-RESULTS-PACKAGE-REPORTS (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-PASSING (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-STATS-SOURCE (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-SYSTEM (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-TEST-REPORTS (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-TESTS (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-TIMESTAMP (NEW-VALUE INSTANCE)

SETFMULTI-RESULTS-WARNING (NEW-VALUE INSTANCE)

MULTIPLE-REPORT (PACKAGES GROUPS TESTS &KEY SYSTEM)

SETFNAME-USE-FIXTURE (NEW-VALUE INSTANCE)

SETFNAME-USE-GROUP (NEW-VALUE INSTANCE)

NAME-USE-P (OBJECT)

SETFNAME-USE-TESTS (NEW-VALUE INSTANCE)

NEXT-PERMUTATION (P)

NO-EFFECT

NOTE-ARTIFACT-CHOICE (NAME ITEM)

NOTE-EXECUTABLE (NAME ARTIFACT)

NST-XML-DUMP (STREAM)

SETFPACKAGE-RESULT-ELAPSED-TIME (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-ERRING (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-FAILING (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-GROUP-RESULTS (NEW-VALUE INSTANCE)

PACKAGE-RESULT-P (OBJECT)

SETFPACKAGE-RESULT-PACKAGE-NAME (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-PASSING (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-TESTS (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-TIMESTAMP (NEW-VALUE INSTANCE)

SETFPACKAGE-RESULT-WARNING (NEW-VALUE INSTANCE)

PICK-FROM-SEQUENCE (SEQ)

PICKER-DEBUG (FORMATTER &REST VALS)

PPRINT-CDATA-STRING (STREAM STRING)

PPRINT-XML (OBJECT &OPTIONAL (STREAM *STANDARD-OUTPUT*))

PRINC-FILLED-TEXT (STRING &OPTIONAL (STREAM *STANDARD-OUTPUT*))

PRUNE-SUPERCEDED-TEST-ARTIFACTS (NAME TEST)

PULL-TEST-NAME-LIST (FORM)

REFINE-PACKAGE-SYMBOL-DESIGS (PACKAGE-DESIG SYMBOL-DESIG)

RELAX-STACK (P)

REPORT-DETAILS (GROUP-OR-PACKAGE GP-SUPP-P TEST TEST-SUPP-P)

REPORT-INTERESTING

REPORT-SUMMARY (GROUP-OR-PACKAGE GP-SUPP-P TEST TEST-SUPP-P)

RESTORE-PROTECTED-OPTION-VALUES (STORED-VALUES)

SETFRESULT-STATS-ELAPSED-TIME (NEW-VALUE INSTANCE)

SETFRESULT-STATS-ERRING (NEW-VALUE INSTANCE)

SETFRESULT-STATS-FAILING (NEW-VALUE INSTANCE)

RESULT-STATS-P (OBJECT)

SETFRESULT-STATS-PASSING (NEW-VALUE INSTANCE)

SETFRESULT-STATS-TESTS (NEW-VALUE INSTANCE)

SETFRESULT-STATS-TIMESTAMP (NEW-VALUE INSTANCE)

SETFRESULT-STATS-WARNING (NEW-VALUE INSTANCE)

RUN-DEBUG-OPTIONS (OPTIONS-FORM)

RUN-GROUP-INST (GROUP-INST)

RUN-NST-COMMAND (&REST ARGS)

RUN-TEST-INST (TEST-INST)

SEPARATE-GROUP-SUBFORMS (FORMS)

SOFT-DEP-WARNING (PREFIX CND STREAM)

STRING-ESCAPED (STRING)

SYMBOL-TO-JUNIT-NAME (SYMBOL)

TIGHTEN-STACK (P SLOTTABLE)

TIMESTAMP-TO-STRING (TIMESTAMP)

USE-STATS-FROM (STATS-SOURCE STATS-DEST)

WHITTLABLE (PF)

WHITTLE-RELAXED-STACK (P)

WRAP-FIXTURE-NAME-SPECIALS (FIXTURE-NAMES-SPECIAL FORM)

MACRO

Public

ASSERT-CRITERION (KEY-ARGS CRITERION-EXPR &REST VALUE-EXPRS)

The ASSERT-CRITERION macro asserts that an NST criterion should pass. (ASSERT-CRITERION ( [ :MSG-FORMAT FORMAT-STRING ] [ :MSG-ARGS FORMAT-ARGUMENTS ] [ :FATAL FLAG ] [ :FAIL-ON-WARNING FLAG ] ) CRITERION FORM ... FORM) MSG-FORMAT Format string used to build the label of the restart point. MSG-ARGS Format arguments used to build the label of the restart point. FATAL If non-nil, a failure of this assertion indicates that execution of the test forms should be aborted. FAIL-ON-WARNING If non-nil, then an NST result which includes a warning indicates failure of this assertion.

COMPOUND-STRUCTURE (&BODY FORMS)

The COMPOUND-STRUCTURE macro wraps substructure which should be considered compound for the limits set by *MAX-COMPOUND-STRUCTURE-DEPTH* .

DEF-ARBITRARY-INSTANCE-TYPE (TYPE-EXPR &BODY TECHNIQUE)

New type specifications for invariant-testing. are defined with the DEF-ARBITRARY-INSTANCE-TYPE macro. (DEF-ARBITRARY-INSTANCE-TYPE ( SPEC-NAME [ :PARAMS FORMALS ] [ :SCALAR BOOL ] [ :KEY KEY ] ) FORM ... FORM) FORMALS Formal parameter definition used to pass subcomponent types. SCALAR When a non-null value is provided for the SCALAR argument, the new specifier is taken to be generable by the SCALAR specification. (def-arbitrary-instance-type (ratio :scalar t) (/ (arbitrary 'integer) (let ((raw (arbitrary (find-class 'integer)))) (cond ((< raw 0) raw) (t (+ 1 raw)))))) KEY The KEY argument gives a list of keyword arguments which may accompany the new specification. For the CONS type, keyword arguments allow specifications for the left and right components: (def-arbitrary-instance-type (cons :key ((car t car-supp-p) (cdr t cdr-supp-p))) (compound-structure (when (and (not car-supp-p) (>= *current-compound-structure-depth* *max-compound-structure-depth*)) (setf car 'scalar)) (when (and (not cdr-supp-p) (>= *current-compound-structure-depth* *max-compound-structure-depth*)) (setf cdr 'scalar)) (cons (arbitrary car) (arbitrary cdr)))) FORM Construct and return (as if through PROGN ) the arbtrary instance.

DEF-BINARY-NEGATED-PREDICATE-ASSERT (ASSERT-FN PREDICATE DEFAULT-MESSAGE &REST KEYARGS &KEY MESSAGE-DEFVAR)

Macro DEF-BINARY-NEGATED-PREDICATE-ASSERT uses the negated result of a binary predicate as the basis for an assertion function just as DEF-UNARY-NEGATED-PREDICATE-ASSERT uses the negated result of a unary predicate. This macro's arguments are just as for DEF-UNARY-PREDICATE-ASSERT .

DEF-BINARY-PREDICATE-ASSERT (ASSERT-FN PREDICATE DEFAULT-MESSAGE &KEY (MESSAGE-DEFVAR NIL DEFVAR-SUPP-P) (DOC-STATE-FLAG T) (PRED-NAME PREDICATE))

Macro DEF-BINARY-PREDICATE-ASSERT uses a binary predicate as the basis for an assertion function just as DEF-UNARY-PREDICATE-ASSERT uses a unary predicate. This macro's arguments are just as for DEF-UNARY-PREDICATE-ASSERT .

DEF-CRITERION ((NAME ARGS-FORMALS VALUES-FORMALS) &BODY FORMS)

The def-criterion macro defines a new criterion for use in NST tests. These criteria definitions are like generic function method definitions with two sets of formal parameters: the forms provided as the actual parameters of the criterion itself, and the values arising from the evaluation of the forms under test.} (DEF-CRITERION ( NAME CRITERION-LAMBDA-LIST VALUES-LAMBDA-LIST ) [ DOCUMENTATION ] FORM ... FORM) NAME Name of the criterion. CRITERION-LAMBDA-LIST Lambda list for the arguments to the criterion. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: :values for call-by-value, or :forms for call-by-name (the default). The list may include the keywords &key, &optional, &body and &rest but may not use &whole or &environment. Apart from this restriction, in the former case the list may be any ordinary lambda list as for defun, and in the latter case the list may be any macro lambda list as for defmacro. VALUES-LAMBDA-LIST Lambda list for the forms under test. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: :values for call-by-value (the default), or :form for call-by-name. In the former case, the list may include the keywords &key, &optional, &body and &rest, but not &whole or &environment; apart from that restriction, list may be any ordinary lambda list as for defun. In the latter case, the remainder of the list must contain exactly one symbol, to which a form which would evaluate to the values under test will be bound. If the criterion ignores the values, then instead of a lambda list, this argument may be the symbol :ignore. On many platforms, listing a dummy parameter which is then declared ignore or ignorable will produce a style warning: the body of a def-criterion should not be assumed to correspond directly to the body of a defmethod; in general there will be surrounding destructuring-binds. DOCUMENTATION An optional documentation string for the criterion. FORM The body of the criterion definition should return a test result report contructed with the make-success-report, etc. functions. Examples: (def-criterion (:true () (bool)) (if bool (make-success-report) (make-failure-report :format "Expected non-null, got: ~s" :args (list bool)))) (def-criterion (:eql (target) (actual)) (if (eql (eval target) actual) (make-success-report) (make-failure-report :format "Not eql to value of ~s" :args (list target))))

DEF-CRITERION-ALIAS ((NAME . ARGS-FORMALS) DOCSTRING-OR-FORM &OPTIONAL (FORM NIL FORM-SUPP-P))

The simplest mechanism for defining a new criterion involves simply defining one criterion to rewrite as another using def-criterion-alias: (DEF-CRITERION-ALIAS ( NAME ARG ... ARG ) [ DOCUMENTATION ] EXPANSION) The body of the expansion should be a Lisp form which, when evaluated, returns an S-expression quoting the new criterion which the rewrite should produce. The args are passed as for Lisp macros: they are not evaluated and are most typically comma-inserted into a backquoted result. For example: (def-criterion-alias (:forms-eq) `(:predicate eq)) (def-criterion-alias (:symbol name) `(:eq ',name))

DEF-CRITERION-UNEVALUATED ((NAME ARGS-FORMALS FORMS-FORMAL &KEY (IGNORE-FORMS NIL)) &BODY FORMS)

The def-criterion-unevaluated macro is deprecated as of NST 2.1.2. It was consolidated into the def-criterion macro. Replace: (def-criterion-unevaluated name (pattern ... pattern) name BODY) with: (def-criterion name (:forms pattern ... pattern) (:form name) BODY)

DEF-EVAL-TEST (NAME-OR-NAME-AND-ARGS &REST FORMS)

The DEF-EVAL-TEST macro abbreviates a call to DEF-TEST with a single EVAL criterion. Its arguments are just as for DEF-TEST and EVAL . (DEF-EVAL-TEST ( NAME [ :GROUP GROUP-NAME ] [ :SETUP FORM ] [ :CLEANUP FORM ] [ :STARTUP FORM ] [ :FINISH FORM ] [ :FIXTURES ( FIXTURE ... FIXTURE ) ] [ :DOCUMENTATION STRING ] [ :CHECK-WARNINGS FLAG ] [ :MUFFLE-WARNINGS FLAG ] [ :ATTEMPT-CONTINUE FLAG ] [ :FORCE-CONTINUE FLAG ] ) FORM ... FORM) (DEF-EVAL-TEST NAME FORM ... FORM)

DEF-FIXTURES (NAME (&KEY (USES NIL USES-SUPP-P) (ASSUMES NIL ASSUMES-SUPP-P) SPECIAL OUTER INNER DOCUMENTATION CACHE (SETUP NIL SETUP-SUPP-P) (CLEANUP NIL CLEANUP-SUPP-P) (STARTUP NIL STARTUP-SUPP-P) (FINISH NIL FINISH-SUPP-P) EXPORT-NAMES (EXPORT-BOUND-NAMES NIL EXPORT-BOUND-NAMES-SUPP-P) (EXPORT-FIXTURE-NAME NIL EXPORT-FIXTURE-NAME-SUPP-P)) &BODY BINDINGS)

Fixtures are data structures and values which may be referred to by name during testing. NST provides the ability to use fixtures across multiple tests and test groups, and to inject fixtures into the runtime namespace for debugging. A set of fixtures is defined using the def-fixtures macro: (DEF-FIXTURES FIXTURE-NAME ( [ :SPECIAL ( NAME ... NAME (:FIXTURE NAME ... NAME) ) ] [ :OUTER FORM ] [ :INNER FORM ] [ :SETUP FORM ] [ :CLEANUP FORM ] [ :STARTUP FORM ] [ :FINISH FORM ] [ :DOCUMENTATION STRING ] [ :CACHE FLAG ] [ :EXPORT-NAMES FLAG ] [ :EXPORT-FIXTURE-NAME FLAG ] [ :EXPORT-BOUND-NAMES FLAG ] ) ( [ ( [ :CACHE FLAG ] ) ] NAME FORM ) ... ( [ ( [ :CACHE FLAG ] ) ] NAME FORM )) FIXTURE-NAME The name to be associated with this set of fixtures. INNER List of declarations to be made inside the let-binding of names of any use of this fixture. Do not include the "declare" keyword here; NST adds these declarations to others, including a special declaration of all bound names. OUTER List of declarations to be made outside the let-binding of names of any use of this fixture. DOCUMENTATION A documentation string for the fixture set. SPECIAL Specifies a list of names which should be declared special in the scope within which this set's fixtures are evaluated. The individual names are taken to be single variable names. Each (:fixture NAME) specifies all of the names of the given fixture set. This declaration is generally optional under most platforms, but can help supress spurious warnings. Note that multiple (:fixture NAME)s may be listed, and these lists and the bare names may be intermixed. If only one name or fixture is specified, it need not be placed in a list EXPORT-FIXTURE-NAME When non-nil, the fixture name will be added to the list of symbols exported by the current package. EXPORT-BOUND-NAMES When non-nil, the names bound by this fixture will be added to the list of symbols exported by the current package. EXPORT-NAMES When non-nil, sets the default value to t for the two options above. CACHE If specified with the group options, when non-nil, the fixture values are cached at their first use, and re-applied at subsequent fixture application rather than being recalculated. When a fixture is attached to a test or test group, each NAME defined in that fixture becomes available in the body of that test or group as if let*-bound to the corresponding FORM. A fixture in one set may refer back to other fixtures in the same set (again `a la let*) but forward references are not allowed. The four arguments :startup, :finish, :setup and :cleanup specify forms which are run everytime the fixture is applied to a group or test. The :startup (respectively :finish) form is run before fixtures are bound (after their bindings are released). These forms are useful, for example, to initialize a database connection from which the fixture values are drawn. The :setup form is run after inclusion of names from fixture sets, but before any tests from the group. The :cleanup form is normally run after the test completes, but while the fixtures are still in scope. Normally, the :cleanup form will not be run if the :setup form raises an error, and the :finish form will not be run if the :startup form raises an error; although the user is able to select (perhaps unwisely) a restart which disregards the error. The names of a fixture and the names it binds can be exported from the package where the fixture is defined using the export-bound-names and export-fixture-name arguments. The default value of both is the value of export-names, whose default value is nil. The cache option, if non-nil, directs NST to evaluate a fixture's form one single time, and re-use the resulting value on subsequent applications of the fixture. Note that if this value is mutated by the test cases, test behavior may become unpredictable! However this option can considerably improve performance when constant-valued fixtures are applied repeatedly. Caching may be set on or off (the default is off) for the entire fixture set, and the setting may vary for individual fixtures. Examples of fixture definitions: (def-fixtures f1 () (c 3) (d 'asdfg)) (def-fixtures f2 (:special ((:fixture f1))) (d 4) (e 'asdfg) (f c)) (def-fixtures f3 () ((:cache t) g (ackermann 1 2)) ((:cache nil) h (factorial 5))) To cause a side-effect among the evaluation of a fixture's name definitions, nil can be provided as a fixture name. In uses of the fixture, NST will replace nil with a non-interned symbol; in documentation such as form :whatis, any nils are omitted.

DEF-FORM-CRITERION ((NAME ARGS-FORMALS FORM-FORMAL) &REST FORMS)

The def-form-criterion macro was deprecated as of NST 1.3.0. Code using def-form-criterion in any but the simplest ways is very likely to fail. Use def-criterion instead.

DEF-TEST (NAME-OR-NAME-AND-ARGS CRITERION &REST FORMS)

Individual unit tests are encoded with the def-test form: (DEF-TEST ( NAME [ :GROUP GROUP-NAME ] [ :SETUP FORM ] [ :CLEANUP FORM ] [ :STARTUP FORM ] [ :FINISH FORM ] [ :FIXTURES ( FIXTURE ... FIXTURE ) ] [ :DOCUMENTATION STRING ] ) CRITERION FORM ... FORM) (DEF-TEST NAME CRITERION FORM ... FORM) The SETUP, CLEANUP, STARTUP, FINISH and FIXTURES are just as for fixtures and test groups, but apply only to the one test. The CRITERION is a list or symbol specifying the properties which should hold for the FORMs. When a test is not enclosed within a group body, a group name must be provided by the GROUP option. When a test is enclosed within a group body, the GROUP option is not required, but if provided it must agree with the group name. When there are no SETUP, CLEANUP, STARTUP, FINISH or FIXTURES arguments, the NAME may be given without parentheses. Likewise, any criterion consisting of a single symbol, e.g. (:pass), may be abbreviated as just the symbol without the parentheses, e.g. :pass. The :documentation form provides a documentation string in the standard Lisp sense. Since documentation strings are stored against names, and since the same name can be used for several tests (so long as they are all in different packages), documentation strings on tests may not be particularly useful. The def-check form is a deprecated synonym for def-test.

DEF-TEST-GENERIC (FUNCTION-NAME &BODY FORMS)

The def-test-generic declares a generic test function. (DEF-TEST-GENERIC FUNCTION-NAME) For example, (nst:def-test-generic for-clses)

DEF-TEST-GROUP (GROUP-NAME GIVEN-FIXTURES &BODY FORMS)

The def-test-group form} defines a group of the given name, providing one instantiation of the bindings of the given fixtures to each test. Groups can be associated with fixture sets, stateful initiatization, and stateful cleanup. (DEF-TEST-GROUP NAME ( FIXTURE ... FIXTURE ) (:SETUP FORM ... FORM) (:CLEANUP FORM ... FORM) (:STARTUP FORM ... FORM) (:FINISH FORM ... FORM) (:EACH-SETUP FORM ... FORM) (:EACH-CLEANUP FORM ... FORM) (:INCLUDE-GROUPS GROUP ... GROUP) (:DOCUMENTATION STRING) TEST ... TEST) GROUP-NAME Name of the test group being defined GIVEN-FIXTURES List of the names of fixtures and anonymous fixtures to be used with the tests in this group. FORMS Zero or more test forms, given by def-check. SETUP These forms are run once, before any of the individual tests, but after the fixture names are bound. CLEANUP These forms are run once, after all of the individual tests, but while the fixture names are still bound. STARTUP These forms are run once, before any of the individual tests and before the fixture names are bound. FINISH These forms are run once, after all of the individual tests, and after the scope of the bindings to fixture names. EACH-SETUP These forms are run before each individual test. EACH-CLEANUP These forms are run after each individual test. INCLUDE-GROUP The test groups named in this form will be run (respectively reported) anytime this group is run (reported). DOCUMENTATION Docstring for the class.

DEF-TEST-METHOD (FUNCTION-NAME (ARG CLASS) &BODY BODY)

The def-test-method defines a general method for a generic test function. (DEF-TEST-METHOD FUNCTION-NAME ( TEST-VALUE CLASS-NAME ) FORM ... FORM) FUNCTION-NAME The name of the test function for which we are defining a method. TEST-VALUE Formal parameter to which the value under test will be bound. CLASS-NAME The class for which we are defining a method. The method body should return a test result report, constructed with make-success-result, etc. For example: (nst:def-test-method for-clses (o mid-cls) (with-slots (mc1 mc2) o (cond ((< mc1 mc2) (make-success-report)) (t (make-failure-report :format "~d not < ~d" :args (list mc1 mc2)))))) (nst:def-test-method for-clses (o side-cls) (with-slots (sc1 sc2) o (cond ((eql sc1 sc2) (make-success-report)) (t (make-failure-report :format "~d not eql ~d" :args (list sc1 sc2))))))

DEF-TEST-METHOD-CRITERION (FUNCTION-NAME CLASS DOCUMENTATION &OPTIONAL (CRITERION NIL CRIT-SUPP-P))

The def-test-method-criterion macro provides a simple facility for defining a generic test function method in terms of an NST criterion. (DEF-TEST-METHOD-CRITERION FUNCTION-NAME CLASS-NAME CRITERION) FUNCTION-NAME The name of the test function for which we are defining a method. CLASS-NAME The class for which we are defining a method. CRITERION The criterion to be applied to members of the class. For example: (nst:def-test-method-criterion for-clses top-cls (:predicate (lambda (tc) (< (tc1 tc) (tc2 tc)))))

DEF-UNARY-NEGATED-PREDICATE-ASSERT (ASSERT-FN PREDICATE DEFAULT-MESSAGE &REST KEYARGS &KEY MESSAGE-DEFVAR)

Macro DEF-UNARY-NEGATED-PREDICATE-ASSERT uses the negated result of a unary predicate as the basis of an assertion function. This macro's arguments are just as for DEF-UNARY-PREDICATE-ASSERT .

DEF-UNARY-PREDICATE-ASSERT (ASSERT-FN PREDICATE DEFAULT-MESSAGE &KEY (MESSAGE-DEFVAR NIL DEFVAR-SUPP-P) (DOC-STATE-FLAG T) (PRED-NAME PREDICATE) &ALLOW-OTHER-KEYS)

Macro DEF-UNARY-PREDICATE-ASSERT creates an assertion function using the result of a call to a unary predicate. A non-nil result from the predicate corresponds to a successful assertion. (DEF-UNARY-PREDICATE-ASSERT ASSERT-FN PREDICATE DEFAULT-MESSAGE [ :MESSAGE-DEFVAR NAME ] [ :PRED-NAME NAME ] [ :DOC-STATE-FLAG BOOL ]) ASSERT-FN The name of the assertion function being defined. PREDICATE The predicate used to define the assertion function. It should take a single argument. DEFAULT-MESSAGE Format string used by default for reporting failures of this assertion. It should expect to be used in a call to FORMAT with one additional argument, the value being tested. MESSAGE-DEFVAR The name of a global variable into which the default message will be stored. If this argument is omitted, the result of a call to GENSYM is used. PRED-NAME This argument is used only for documenting the underlying predicate in the assertion function's docstring. By default, it is the same as the predicate.

DEF-VALUES-CRITERION ((NAME ARGS-FORMALS FORMS-FORMALS &KEY (DECLARE NIL DECL-SUPP-P)) &BODY FORMS)

The def-values-criterion macro was deprecated as of NST 1.3.0. For new criteria, use def-criterion instead. In the short term, code using def-values-criterion should continue to work as before.

EMIT-FAILURE (&REST ARGS)

The emit-failure function is deprecated; use make-failure-report instead.

EMIT-SUCCESS (&REST ARGS)

The emit-success function is deprecated; use make-success-report instead.

EMIT-WARNING (&REST ARGS)

The emit-warning function is deprecated; use make-warning-report instead.

NST-CMD (&REST ARGS)

User-level NST operations are accessible from the REPL via the~nst-cmd macro. (NST-CMD NST-COMMAND ARG ... ARG) Where a particular system supports the facility,Currently Allegro, and SBCL under ACL-REPL. the top-level alias :nst} provides a shorthand to this function.

WITH-FIXTURES ((&REST FIXTURES) &BODY FORMS)

The with-fixtures macro faciliates debugging and other non-NST uses of fixtures sets: (WITH-FIXTURES ( FIXTURE ... FIXTURE ) FORM ... FORM) This macro evaluates the forms in a namespace expanded with the bindings provided by the fixtures.

Private

DEBUG-CHECK (DEFCHECK)

Debugging aid for def-check forms. Provides all-caps dummy values for dynamic variables normally provided by def-test-group.

DEF-CHECK-ALIAS (&REST ARGS)

Deprecated: use def-criterion-alias instead

DEF-CONTROL-CHECK (&REST ARGS)

Deprecated: use def-criterion instead

EMIT-ERROR (&REST ARGS)

Deprecated; use make-success-report.

HANDLER-BIND-INTERRUPTABLE (HANDLERS &BODY FORMS)

Like handler-bind (and the arguments are similar), but cancel any error- handling if the error is related to a keyboard interrupt.

RETURNING-CRITERION-CONFIG-ERROR ((MSG) &BODY FORMS)

For use within criteria definitions only --- catch errors and process them as errors in the "glue" among criteria and forms.

RETURNING-TEST-ERROR (&BODY FORMS)

For use within criteria definitions only --- catch errors and process them as errors arising from within the .

Undocumented

ADD-CLASS-NAME-INSTANTIATOR-METHOD (FN)

ADD-CLASS-NAME-STATIC-METHOD (FN)

APPLY-DEBUG-OPTIONS (FORMS-SPEC PROTECT-VARS &BODY FORMS)

APPLY-DEFAULT-DEBUG-OPTIONS (&BODY FORMS)

AT-VERBOSITY (LV &BODY FORMS)

CHECK-RESULT (&REST ARGS)

COIN-FLIP (&REST OUTCOMES)

COUNT-NONNULLS (&REST BOOLS)

DEF-BUNDLE-PACKAGE (NAME REPACKAGING &BODY FORMS)

DEF-CHECK (&REST ARGS)

DEF-NST-INTERACTIVE-COMMAND ((NAME &KEY SHORT-HELP (LONG-HELP NIL LONG-HELP-SUPP-P) (LONG-HELP-SPECIAL NIL LONG-HELP-SPECIAL-SUPP-P) (ARGS NIL ARGS-SUPP-P) (REPEATABLE NIL)) &BODY FORMS)

DEF-NST-PROPERTY (NAME VARIABLE &KEY (DOC ) (FILTER '(LAMBDA (X) X)) (UNFILTER '(LAMBDA (X) X)))

DEF-VALUE-CHECK (&REST ARGS)

DEFINE-NST-ERROR (NAME FIELDS (STREAM EXP) &BODY PRINTER)

FORMAT-AT-VERBOSITY (LV FORMAT &REST ARGS)

LOG10 (V)

NAMED-FUNCTION (NAME LAMBDA-EXPRESSION)

PROTECT-NST-CONFIG (&BODY FORMS)

WITH-CONTEXT-LAYER (LAYER-EXPR &BODY BODY)

WITH-CRITERION-CONTEXT-LAYER ((&REST LAYER-ARGS) &BODY BODY)

WITH-METHOD-CONTEXT-LAYER ((METHOD-NAME CLASS-NAME OBJECT) &BODY BODY)

WITH-NST-CONTROL-HANDLERS (((FORMAL FLAG &KEY (FOR-FAIL NIL) (FOR-ERROR T) (CERROR-LABEL-VAR NIL CERROR-LABEL-VAR-SUPP-P) (CERROR-LABEL NIL CERROR-LABEL-SUPP-P) (WITH-RETRY NIL WITH-RETRY-SUPP-P) (CERROR-BY-FLAG T) (POST-CERROR NIL POST-CERROR-SUPP-P) (HANDLER-RETURN-TO NIL HANDLER-RETURN-TO-SUPP-P) (HANDLER-RETURN NIL)) &BODY HANDLER) &BODY BODY)

WITH-OUTPUT-FOR-VERBOSITY ((LV STREAM-NAME) &BODY FORMS)

WITH-PPRINT-CDATA ((STREAM-NAME) &BODY FORMS)

WITH-RETRY ((CONTINUATION-LABEL) &BODY FORMS)

WITH-XML-TAGGED-PPRINT-LOGICAL-BLOCK ((STREAM TAG &KEY PROPERTIES) &BODY FORMS)

GENERIC-FUNCTION

Public

ADD-WARNING (RESULT W &REST ARGS)

The add-warning function adds an warning to a result record. The item can be any of a Lisp warning, an NST check-note or a format string; in the first two cases, no additional arguments should be provided. (ADD-WARNING RESULT-REPORT [ :FORMAT ITEM ] [ :ARGS ARGUMENT-LIST ])

ARBITRARY (TYP)

This function takes a single argument, which determines the type of the value to be generated. For simple types, the name of the type (or the class object, such as returned by FIND-CLASS ) by itself is a complete specification. For more complicated types, ARBITRARY can also take a list argument, where the first element gives the type and the remaining elements are keyword argument providing additional requirements for the generated value.

NST-JUNIT-DUMP (STREAM)

Function nst-junit-dump pushes the entire NST state to a JUnit XML file whose stream is specified by its argument.

Private

APPLY-CRITERION (TOP ARGS FORM)

Internal generic function whose methods are the translations of criteria.

ARBITRARY-BY-SPEC (NAME &KEY (LENGTH (MID-BIASED-NATNUM 4)) (ELEM T ELEM-SUPP-P) (DIMENS NIL DIMENS-SUPP-P) (RANK NIL RANK-SUPP-P) (PACKAGE (PICK-FROM-SEQUENCE (LIST-ALL-PACKAGES))) (NONNULL T) (EXPORTED NIL) (GENSYM NIL) (EXISTING T) (VAL NIL VAL-SUPP-P) (KEY NIL KEY-SUPP-P) (SIZE NIL SIZE-SUPP-P) (TEST NIL TEST-SUPP-P) (CAR T CAR-SUPP-P) (CDR T CDR-SUPP-P) (NONCONTROL T) (RANGE *DEFAULT-CHARACTER-RANGE*) &ALLOW-OTHER-KEYS)

Return an arbitrary value according to a spec

ARBITRARY-GROUNDED-TYPE (BOUND)

Return an arbitrary type bounded by the given type.

BLURB-CONTEXT-LINE (STREAM ID ARGS FORMS)

Give a short description of a context.

BOUND-NAMES (FIXTURE-OR-GROUP)

The names defined by each fixture. Will be given a methods by the macro which expands fixtures.

DETAIL-CONTEXT-LINE (STREAM ID ARGS FORMS)

Give a longer blurb of a context.

DO-GROUP-FIXTURE-ASSIGNMENT (GROUP-OBJ TEST-OBJS)

Fixture declarations translate to an :around method making let* bindings for the group application class.

DO-TEST-FIXTURE-ASSIGNMENT (TEST-OBJ)

Fixture declarations translate to an :around method making let* bindings for the test application class.

FORMAT-FOR-WARNING (STREAM ITEM COLON AT-SIGN &REST PARAMS)

Hook allowing us to sometimes do better than the usual pretty-printer for warnings.

GET-FIXTURE-BINDINGS (FIXTURE)

Internal function: pull the symbolic fixture bindings

GROUP-CLASS-NAME (GROUP-NAME)

Map from groups to the private name with which NST associates the class of group-specific activities.

GROUP-FIXTURE-CLASSES (GROUP-NAME)

Map from groups to the private names of the group's fixtures.

GROUP-RECORD-P (OBJ)

Return non-nil if an item is a group record.

NST-ARG-NAMES (COMMAND)

Return a string giving help for the argument names for an ~ NST REPL command.

NST-LONG-HELP (COMMAND)

Return the long help message for an NST REPL command.

NST-REPL-PROPERTY-DISPLAY (N)

Return the display value of an NST property's internal value.

NST-REPL-PROPERTY-DOC (N)

Return the documentation string of an NST property.

NST-REPL-PROPERTY-ENCODE (PROP VAL)

Encode an NST property's display value as an internal value.

NST-SHORT-HELP (COMMAND)

Return the short help message for an NST REPL command.

OPEN-FIXTURE (FIXTURE-NAME &OPTIONAL PACKAGE)

Inject the names defined by the named fixture into the given package, by default the current package.

PREP-ARG-NAMES-HELP (ARG-LIST)

This function formats the lambda list of an NST REPL command for display in the online-help system. The macro def-nst-interactive-command that expands a REPL command definition into the underlying Lisp defmethods uses this function to generate the bodies of the help-related methods, so this function must be available from compile-time forward.

RESULT-SUMMARY (REPORT &OPTIONAL CODE TOTAL PASSED ERRED FAILED WARNED OTHER)

Receives a reporting structure (or list of them); returns a six-value summary of the results: - A symbol, one of: :error :fail :warn :info :clear - The total number of named checks. - The number passed. - The number raising an error. - The number failing. - The number giving a warning.

RUN-COMMAND-ACTUAL (COMMAND &REST ARGS)

Top-level command interpreter for the NST tester

SHOW-CONTEXT-LAYER (LAYER)

Used to determine whether a context-layer should be displayed. Methods may safely refer and mutate the hashtable stored in the special variable -context-display-state- .

SHOW-NST-PROPERTY (NAME)

Display a property value (presumably) to the REPL.

STACK-TRANSFORMER (ID)

Check form-specific stack transformation.

TEST-NAMES (FIXTURE-OR-GROUP)

The names of tests in a group. Will be given an eql-method by the macros which expand tests and groups.

TEST-RECORD-P (OBJ)

Return non-nil if an item is a group record.

TRACE-FIXTURE (FX)

Provide debugging information about a fixture.

TRACE-GROUP (GR)

Provide debugging information about a group.

TRACE-TEST (GR TS)

Provide debugging information about a test.

Undocumented

ACTUAL-VALUE (CONDITION)

ASSERTION-FAILURE-ARGS (CONDITION)

ASSERTION-FAILURE-FATAL (CONDITION)

ASSERTION-FAILURE-FORMATTER (CONDITION)

CANONICALIZE-LOOKUP-NAME (O)

CHECK-NOTE-TYPE-STRING (TYPE CONTEXT)

CONSIDER-REPL-CALL-SAVE (NAME ARGS)

CRITERION-NAME (CONDITION)

DOMAIN (CONDITION)

EXPECTED-FORM (CONDITION)

FLUSH-FIXTURE-CACHE (F)

FMT-PERMUTER (STREAM P)

GROUP (CONDITION)

JUNIT-GROUP-RESULT (GROUP &REST ARGS &KEY DIR FILE VERBOSE STREAM IF-DIR-DOES-NOT-EXIST IF-FILE-EXISTS)

KEYWORD-ARGS (CONDITION)

NST-ASSERTION-RESULT (CONDITION)

OLD-NAME (CONDITION)

PACKAGE-GROUPS (PACKAGE-OR-SYMBOL)

PACKAGE-OF (CONDITION)

RECORD-NAME-USE (CATEGORY NAME ITEM)

REPLACEMENT (CONDITION)

REQUIRED-KEYARG (CONDITION)

SET-NST-PROPERTY (NAME VALUE)

TEST (CONDITION)

TEST-FIXTURE-CLASSES (NAME)

SLOT-ACCESSOR

Private

CHECK-GROUP-NAME (CHECK-INSTANCE)

The Map from a check instance back to its internal name.

GROUP-FIXTURE-CLASS-NAMES (GROUP-INSTANCE)

The Map from a group instance to the list of fixture classes associated with it.

GROUP-GIVEN-FIXTURES (TEST-GROUP)

The For documentation purposes --- returns a list of names of fixtures that are used by TEST-GROUP.

GROUP-NAME (GROUP-INSTANCE)

The Map from a group or test instance back to its symbolic name.

SPECIAL-FIXTURE-NAMES (TEST-RECORD)

The The unevaluated forms to be used as input to a test.

TEST-CRITERION (TEST-RECORD)

The The unevaluated forms to be used as input to a test.

TEST-FORMS (TEST-RECORD)

The The unevaluated forms to be used as input to a test.

Undocumented

ANON-FIXTURE-FORMS (GROUP-RECORD)

CRITERION (OBJECT)

SETFCRITERION (NEW-VALUE OBJECT)

CRITERION-ARGS (OBJECT)

SETFCRITERION-ARGS (NEW-VALUE OBJECT)

EQL-SPECIALIZER-OBJECT (EQL-SPECIALIZER)

GENERIC-FUNCTION-METHODS (STANDARD-GENERIC-FUNCTION)

SETFGENERIC-FUNCTION-METHODS (NEW-VALUE STANDARD-GENERIC-FUNCTION)

GIVEN-STACK (OBJECT)

SETFGIVEN-STACK (NEW-VALUE OBJECT)

GROUP-EACHTEST-CLEANUP-THUNK (RECORD)

GROUP-EACHTEST-SETUP-THUNK (RECORD)

GROUP-FIXTURES-CLEANUP-THUNK (RECORD)

GROUP-FIXTURES-SETUP-THUNK (RECORD)

GROUP-INCLUDE-GROUPS (GROUP-RECORD)

GROUP-WITHFIXTURES-CLEANUP-THUNK (RECORD)

GROUP-WITHFIXTURES-SETUP-THUNK (RECORD)

HAS-NEXT (OBJECT)

METHOD-NAME (OBJECT)

SETFMETHOD-NAME (NEW-VALUE OBJECT)

METHOD-SPECIALIZERS (M)

OBJECT (OBJECT)

SETFOBJECT (NEW-VALUE OBJECT)

SINGLETON (OBJECT)

SETFSINGLETON (NEW-VALUE OBJECT)

TEST-CLEANUP-FORM (OBJECT)

TEST-FINISH-FORM (OBJECT)

TEST-LIST (TEST-GROUP)

SETFTEST-LIST (NEW-VALUE OBJECT)

TEST-NAME-LOOKUP (GROUP)

TEST-SETUP-FORM (OBJECT)

TEST-STARTUP-FORM (OBJECT)

VARIABLE

Public

*DEBUG-ON-ERROR*

User variable *debug-on-error*: if non-nil, will break into the Lisp REPL debugger upon encountering an unexpected error. If nil, will record the error and continue with other tests.

*DEBUG-ON-FAIL*

User variable *debug-on-fail*: if non-nil, will break into the Lisp REPL debugger upon encountering a test which fails. If nil, will record the failure and continue with other tests. This variable is useful inspecting the dynamic environment under which a test was evaluated.

*DEFAULT-REPORT-VERBOSITY*

User variable *default-report-verbosity* determines the default value for *nst-verbosity* when printing reports (2 by default).

*MAX-COMPOUND-STRUCTURE-DEPTH*

The *MAX-COMPOUND-STRUCTURE-DEPTH* variable sets the maximum nesting depth of compound data structures: beyond that depth, SCALAR rather than T is the default element generator. This restriction does not apply to explicitly specified element types, only to the use of defaults.

*NST-OUTPUT-STREAM*

User variable *nst-output-stream* determines the output stream to which NST should print its output (*standard-output* by default).

Undocumented

*ASSERT-EQ-FORMAT-STRING*

*ASSERT-EQL-FORMAT-STRING*

*ASSERT-EQUAL-FORMAT-STRING*

*ASSERT-EQUALP-FORMAT-STRING*

*ASSERT-NONNIL-FORMAT-STRING*

*ASSERT-NOT-EQ-FORMAT-STRING*

*ASSERT-NOT-EQL-FORMAT-STRING*

*ASSERT-NOT-EQUAL-FORMAT-STRING*

*ASSERT-NOT-EQUALP-FORMAT-STRING*

*ASSERT-NULL-FORMAT-STRING*

*ASSERT-ZERO-FORMAT-STRING*

Private

*ARTIFACT-LOOKUP*

Hash table for looking up name usage. Symbols in :nst-artifact-lookup-package are mapped to a set of denoted items stored in a hash-table mapping items to t.

*BINDING-VARIABLE*

Variable tracking the binding of names by fixtures, checked when binding fails.

*DEBUG-NST-PICKER*

Debugging flag for this experimental feature.

*DEFAULT-DEBUG-CONFIG*

User variable: the default setting applied by default. Should be a list of alternating keyword/forms matching: - nst-set - list of lists, each with arguments to :nst :set - progn - list of forms to be evaluated

*GENERATE-BACKTRACES*

User variable: if non-null, will attempt to capture the Lisp backtrace of errors in tests.

*NST-CHECK-INTERNAL-NAME*

Dynamic variable used to set the name of a test in its result report.

*NST-CHECK-USER-NAME*

Dynamic variable used to set the name of a test in its result report.

*NST-CONTEXT*

Dynamic-scoped variable recording the values under test - a list of context-layer instances.

*NST-CONTEXT-EVALUABLE*

Dynamic-scoped variable tracking whether the values under test should be asusmed evaluated. Used in preparing context expressions.

*NST-DEBUG*

User variable: apply customizable debugging settings.

*NST-GROUP-NAME*

Dynamic variable used to set the name of the group in a test result report.

*NST-GROUP-SHOWN*

Dynamic-scoped variable tracking whether the name of a group has been printed, so that tests need not repeat it.

*NST-INFO-SHOWS-EXPECTED*

Debugging-oriented user flag: when tracing NST structures, print expected values as hardcoded by the macros, rather than recalled via the generic functions whose methods the macros define.

*NST-REPORT-DRIVER*

Dynamic-scoped variable - one of :multiple, :package, :group or :test to determine the top-level form of a report. Used as a control parameter for printing reports.

*NST-STACK*

Dynamic-scoped variable - the stack of values under test by the current criterion.

*NST-VERBOSITY*

User variable determining how verbose NST's output to the REPL should be. Internally, this variable takes an integer value: 0 and below are silent, 1 is the default, 2 and 3 are more verbose, and 4 is for full tracing. The command-line interpreter assigns keywords to these values, from most terse to most verbose: :silent, nil, :quiet, :default, t, :verbose, :vverbose (:quiet and :default are the same, and are the initial setting).

*TEST-METHODS*

Global variable for programmer-defined NST test methods.

+NAME-PACKAGES+

Known occurances of names in different packages.

+NAME-USE+

Known uses of names of a particular package.

+PACKAGE-GROUPS+

Map from packages to the test groups declared in each package.

+RESULTS-RECORD+

Results of test runs.

Undocumented

*CHAR-ESCAPES*

*CURRENT-COMPOUND-STRUCTURE-DEPTH*

*DEFAULT-CHARACTER-RANGE*

*DEFAULT-DEBUG-PROTECT*

*DEFAULT-XML-PPRINT-DISPATCH*

*ENTITIES*

*IMPLICIT-GROUP-CHOICE*

*LAST-NAMED-ARTIFACT-USE*

*LAST-REPL-CALL*

*MAX-AUTO-ARRAY-RANK*

+ARBITRARY-GENERABLE-TYPES+

+NST-REPL-COMMANDS+

+NST-REPL-PROPERTIES+

+SCALAR-GENERABLE-TYPES+

QW

QWERTY

CLASS

Private

CHECK-NOTE

A single note issued in criteria checking. context - the surrounding criteria structure, a list of context-layer structs stack - the stack of values at the note point format, args - further details; this string and list may e.g. be provided to cl:format ** IMPORTANT ** Do not use check-note-format or check-note-args! Do not assume that the format slot is a string! The API for these slots is apply-check-note-formatter, which checks for function values (or other types allowed as a formatter since the time when this docstring was written).

CHECK-RESULT (&REST ARGS)

Overall check result structure, containing notes of four distinct types. A note is an instance of the check-note structure below. The four note types are: warnings - generated warnings failures - criteria which are not met, but not a Lisp error errors - Lisp errors info - supplimentary information Each of these fields is a list; warnings, failures and errors are check-note instances, and the info field is of any value.

CONTEXT-LAYER

Superclass of records of test criterion.

CRITERION-CONTEXT-LAYER

A record of test criterion criterion - the criterion symbol itself criterion-args - arguments to the criterion given-stack - the stack of values assessed by the criterion

ERROR-CHECK-NOTE

A note issued in regards to a thrown error.

GROUP-RESULT

Overall group result structure, mapping checks to results by name.

METHOD-CONTEXT-LAYER

A record of method-based test invocation.

MULTI-RESULTS

Multiple results structure.

NAME-USE

Record for tracking the artifacts which NST associates with a name.

NST-GROUP-RECORD

Superclass of NST group definitions.

NST-TEST-RECORD

Common superclass of NST test records.

PACKAGE-RESULT

Overall package result structure, mapping groups to results by name.

RESULT-STATS

Statistics common to the different result summaries.

STANDARD-FIXTURE

Common superclass of all fixtures.

Undocumented

PERMUTER

PERMUTER-FRAME

SINGLETON-CLASS

CONDITION

Private

DEBUG-FOR-FAIL

Condition raised

NST-DEPRECATION-WARNING-MIXIN

Mixin of field used in deprecation warnings

Undocumented

CRITERION-MISSING-MANDATORY-ARGUMENT

NO-NST-GROUPS-IN-PACKAGE

NO-SUCH-NST-GROUP

NO-SUCH-NST-TEST

NOT-EXPECTED-FORM

NST-ASSERTION-CONDITION

NST-ASSERTION-FAILURE

NST-ASSERTION-RESULT-CHECK

NST-ERROR

NST-HARD-DEPRECATION

NST-SOFT-DEPRECATION

NST-SOFT-KEYARG-DEPRECATION

UNKNOWN-ARBITRARY-DOMAIN

CONSTANT

Private

+KEYBOARD-INTERRUPT-CLASS+

Name of the class of error thrown when a keyboard interrupt is received, or nil if this implementation of Lisp has no such class.