Common Lisp Package: DSO-LEX

Allows the definition of lexers. See DEFLEXER.

README:

FUNCTION

Public

MAKE-LEXER (DEFS &KEY PRIORITY-ONLY)

Returns a lexer function. The DEFS consists of token-class definitions, each being a list of a regular expression, the name of the class, and an optional filter. The returned function takes as arguments an input sequence and an optional start position, and returning the matched token-class, image, and image-length as values. Unless PRIORITY-ONLY is true, the longest match will win, and rule-priority will only be used to break ties. Otherwise, the first match wins. Example: (let ((lexer (make-lexer '(("[0-9]+" number parse-integer) ("[a-zA-Z]" letter))))) (funcall lexer "2pi" 1))

Undocumented

LEX-ALL (LEXER INPUT)

Private

Undocumented

ANCHOR-AND-MODE (REGEX)

BREAK-DEFS (DEFS)

COMBINE (REGEX-LIST)

GREEDY-LEXER-FORM (INPUT-VAR START-VAR DEFS)

LEXER-FORM (INPUT-VAR START-VAR DEFS)

WRAP (REGEX)

MACRO

Public

DEFLEXER (NAME (&KEY PRIORITY-ONLY) &BODY DEFS)

Defines a lexer, called as a function of the given NAME, and returning the matched token-class, image, and image-length as values. The body consists of token-class definitions, each being a list of a regular expression, the name of the class, and an optional filter. Unless PRIORITY-ONLY is true, the longest match will win, and rule-priority will only be used to break ties. Otherwise, the first match wins. Example: (deflexer lexer () ("[0-9]+" number parse-integer) ("[a-zA-Z]" letter)) (lexer "2pi" 1)