Common Lisp Package: TRIVIAL-LDAP

README:

Introduction

One-two, one-two... is this thing working?

This is Kevin Montuori's trivial-ldap, with a few modifications.

A few years back, I needed some mechanism for querying AD servers, as part of a custom content connector for the FAST ESP search engine. I found trivial-ldap, and was soon using it to good effect.

After having used trivial-ldap for a while, I made some modifications, and asked Kevin to review them, and integrate them if he felt that they added value. Unfortunately, Kevin is too busy to spend time on trivial-ldap, so he graciously let me publish whatever changes I had.

Changes

LDAP Filter Parser

The LDAP filter parser has been rewritten from scratch, using cl-yacc. This makes filter parsing somewhat faster, and should also make it easier to maintain. The downside is one more dependency.

Attribute Naming

The original code used symbols in the current package to name LDAP attributes. This has changed, and all attribute names are now interned in the keyword package. So, something like

(ldap:attr-value *entry* 'cname) 

should now be

(ldap:attr-value *entry* :cname) 

and so on. Note: this is probably only important when working with an LDAP entry, as that is the only place where we use symbol identity for matching.

Binary Attributes

There was a tacit assumption in the trivial-ldap code that all attributes are UTF-8 encoded strings, while in reality they can be 7-bit ASCII (USASCII), UTF-8 or even binary. There is now a mechanism in place for giving hints to trivial-ldap that certain attributes should be treated as binary values --- such attributes will be returned as lists of (unsigned-byte 8), instead of as unicode strings.

The interface to this mechanism is

(ldap:attribute-binary-p <attribute-name>) =><generalized-boolean>

and

(setf (ldap:attribute-binary-p <attribute-name>) <generalized-boolean>) 

Note: Elias MÃ¥rtenson has supplied some handy restarts that can be used when it turns out that an attribute cannot be converted to UTF-8 (which, in turn, probably means that it should be treated as binary). See handle-as-binary and handle-as-binary-and-add-known in trivial-ldap.lisp .

List Equivalents

Search filters and values can be specified as lists instead of as strings. This has two advantages:

  • Binary values can be specified (lists of octet values will not be treated as UTF-8 sequences).

  • It is not necessary to build string representations of a filter just to have the filter parser deconstruct it back to the representation that should be natural for Lisp.

  • Values can be specified as octet lists, strings or symbols -- when a symbol is specified, the actual value used is whatever (symbol-name &lt;symbol&gt;) returns.

  • The function #'listify-filter can be used to turn a string filter into an equivalent list representation; this should be useful for experimenting with the list format.

Examples:

(ldap:search *ldap* '(and (= objectclass person) (= cname "rayw")))  
 
(let ((name "rayw"))  
    (ldap:search *ldap* `(and (= objectclass person) (= cname ,name)))) 

Paging Through Results

Support for the LDAP Control Extension "Simple Paged Results" (rfc2696) has been added. It is invoked by setting the :size-limit search parameter to 0 (zero), and setting :paging-size to a positive integer. Note that the server imposes its own restrictions here, so the actual number of results in a batch may be lower than specified.

Apart from setting these two required parameters, the operation of the paging mechanism is wholly transparent: batches are fetched automatically whenever the #'next-search-result method has exhausted all entries in the current batch (assuming that the appropriate parameters have been specified, and that there are actually more results to be fetched.)

Examples:

(and (ldap:search *ldap* '(& (substring samaccountname "ra*") (= objectclass person))  
                  :attributes '("1.1") :size-limit 0 :paging-size 500)  
     (loop for entry = (ldap:next-search-result *ldap*)  
           while entry  
           count entry)) 

FUNCTION

Public

NEW-ENTRY (DN &KEY (ATTRS NIL) (INFER-RDN T))

Instantiate a new entry object.

NEW-ENTRY-FROM-LIST (LIST)

Create an entry object from the list return by search.

NEW-LDAP (&KEY (HOST localhost) (SSLFLAG NIL) (PORT (IF SSLFLAG +LDAP-PORT-SSL+ +LDAP-PORT-NO-SSL+)) (USER ) (PASS ) (BASE NIL) (DEBUG NIL) (REUSE-CONNECTION NIL))

Instantiate a new ldap object.

REPLACE (SEQUENCE1 SEQUENCE2 &REST ARGS &KEY (START1 0) (END1 NIL) (START2 0) (END2 NIL))

Destructively modifies SEQUENCE1 by copying successive elements into it from the SEQUENCE2. Elements are copied to the subseqeuence bounded by START1 and END1, from the subsequence bounded by START2 and END2. If these subsequences are not of the same length, then the shorter length determines how many elements are copied.

Undocumented

ATTRIBUTE-BINARY-P (ATTRIBUTE-NAME)

SETFATTRIBUTE-BINARY-P (VALUE ATTRIBUTE-NAME)

ESCAPE-STRING (STRING)

FIND (ITEM SEQUENCE &REST ARGS &KEY FROM-END (START 0) END KEY TEST TEST-NOT)

LISTIFY-FILTER (FILTER)

UNESCAPE-STRING (STRING)

Private

BASE10->BASE256 (INT)

Return representation of an integer as a list of base 256 'digits'.

BASE256->BASE10 (LIST)

Given a list of base 256 'digits' return an integer.

BER-CLASS-ID (CLASS)

Return the bits to construct a BER tag of type class.

BER-LENGTH (IT)

Given a sequence or integer, return a BER length.

BER-MSG (TAG DATA)

Given a BER tag and a sequence of data, return a message

BER-P/C-BIT (P/C)

Return the bit to construct a BER tag of class primitive or constructed.

BER-TAG (CLASS P/C NUMBER-OR-COMMAND)

Construct the list of bytes that constitute a BER tag number 0-127. CLASS should be the symbol universal, applicaiton, context, or private. P/C should be the symbol primitive or constructed. NUMBER should be either an integer or LDAP application name as symbol.

BER-TAG-TYPE (CLASS P/C)

Construct the bits that kicks off a BER tag byte.

CHAR-CODE-LIST->STRING (CHAR-CODE-LIST)

Convert a list of bytes into a string.

CHECK-MESSAGE-NUMBER (RESPONSE EXPECTED-MESG-NUMBER)

Determine if the message number of a BER response is correct. Returns BER response with message number bytes consumed if it is correct or NIL otherwise.

INT->OCTET-LIST (INT)

Return 2s comp. representation of INT.

LDAP-COMMAND (COMMAND)

Given a symbol naming an ldap command, return the command number.

LDAP-COMMAND-SYM (NUMBER)

Given an application number, return the command name as symbol.

LDAP-DEREF (&OPTIONAL (DEREF 'NEVER))

Given a deref symbol return the enumeration int.

LDAP-FILTER-COMPARISON-CHAR (COMPARISON-CHAR-AS-SYMBOL)

Given a comparison character, return its integer enum value.

LDAP-MODIFY-TYPE (TYPE)

Given a modify type, return the enumeration int.

LDAP-SCOPE (&OPTIONAL (SCOPE 'SUB))

Given a scope symbol return the enumeration int.

LDAP-SUBSTRING (TYPE)

Given a substring type, return its integer choice value.

MSG-COMPARE (DN-OR-ENTRY ATTRIBUTE VALUE)

Return the sequence of bytes representing a compare message.

MSG-DELETE (DN-OR-ENTRY)

Return the sequence of bytes representing a delete message.

MSG-MODDN (DN-OR-ENTRY NEW-RDN DELETE-OLD NEW-SUP)

Return the sequence of bytes representing a moddn message.

MSG-MODIFY (DN-OR-ENTRY MOD-LIST)

Return the sequence of bytes representing a modify message.

NEW-REFERRER (URL)

Instantiate a new referrer object.

OCTET-LIST->INT (OCTET-LIST)

Convert sequence of twos-complement octets into an integer.

RDN-FROM-DN (DN)

Given a DN, return its RDN and a cons of (att . val)

READ-APP-NUMBER (TAG)

Given an application tag, return which ldap app number it represents.

READ-DECODER (RESPONSE)

Decode a BER encoded response (minus initial byte & length) from LDAP.

READ-INTEGER (MESSAGE)

Read an int from the message, return int and number of bytes consumed.

READ-LENGTH (MESSAGE)

Given message starting with length marker, return length and bytes consumed

READ-MESSAGE-NUMBER (RESPONSE EXPECTED-MESG-NUMBER)

Read message number from the seq, return t or nil and bytes consumed.

READ-OCTETS (MESSAGE)

Read an octet vector from the message, return vector and bytes consumed..

READ-STRING (MESSAGE)

Read a string from the message, return string and bytes consumed..

SEQ-ATT-AND-VALUES (ATT VALS)

BER encode an attribute and set of values (for use in modify).

SEQ-ATTRIBUTE-ALIST (ATTS)

BER encode an entry object's attribute alist (for use in add).

SEQ-ATTRIBUTE-ASSERTION (ATT VAL)

BER encode an ldap attribute assertion (for use in compare).

SEQ-ATTRIBUTE-LIST (ATT-LIST)

BER encode a list of attributes (for use in search).

SEQ-ATTRIBUTE-VALUE-ASSERTION (ATT VAL)

BER encode an ldap attribute value assertion (for use in filters).

SEQ-BOOLEAN (T/F)

BER encode a boolean value.

SEQ-CONSTRUCTED-CHOICE (INT &OPTIONAL DATA)

BER encode a context-specific, constructed choice.

SEQ-ENUMERATED (INT)

BER encode an enumeration value.

SEQ-INTEGER (INT)

BER encode an integer value.

SEQ-NULL

BER encode a NULL

SEQ-OCTET-STRING (STRING)

BER encode an octet string value.

SEQ-PRIMITIVE-CHOICE (INT &OPTIONAL DATA)

BER encode a context-specific choice.

SEQ-PRIMITIVE-STRING (STRING)

BER encode a string/symbol for use in a primitive context.

SEQ-SEQUENCE (TLV-SEQ)

BER encode a sequence of TLVs.

SEQ-SET (TLV-SET)

BER encode a set of TLVs.

SEQ-SUBSTRINGS (VALUE)

Given a search value with *s in it, return a BER encoded list.

SPLIT-SUBSTRING (STRING &OPTIONAL LIST)

Split a substring filter value into a list, retaining the * separators.

STRING->CHAR-CODE-LIST (STRING)

Convert a string into a list of bytes.

VALID-LDAP-RESPONSE-P (TAG-BYTE)

Return T if this is the valid initial tag byte for an LDAP response.

Undocumented

ATTRS-FROM-LIST (X)

LDAP-FILTER-LEXER (STRING)

LDAP-RESULT-CODE-STRING (CODE)

LDAP-RESULT-CODE-SYMBOL (CODE)

LIST-ENTRIES-TO-STRING (KEY LIST)

PROCESS-RESPONSE-CONTROLS (LDAP CONTROLS)

READ-CONTROLS (MESSAGE)

READ-GENERIC (MESSAGE &OPTIONAL (RES NIL))

SEQ-FILTER (FILTER)

MACRO

Public

Undocumented

DOSEARCH ((VAR SEARCH-FORM) &BODY BODY)

Private

Undocumented

DEFINE-CONSTANT (NAME VALUE &OPTIONAL DOC)

GENERIC-FUNCTION

Public

Undocumented

ABANDON (LDAP)

ADD (LDAP ENTRY)

ADD-ATTR (ENTRY ATTR VALS)

ATTR-LIST (ENTRY)

ATTR-VALUE (ENTRY ATTR)

BIND (LDAP)

CHANGE-RDN (ENTRY NEW-RDN)

COMPARE (LDAP DN-OR-ENTRY ATTRIBUTE VALUE)

DEL-ATTR (ENTRY ATTR &OPTIONAL VALS)

DELETE (LDAP DN-OR-ENTRY)

LDIF (ENTRY)

MODDN (LDAP DN-OR-ENTRY NEW-RDN &KEY NEW-SUP DELETE-OLD)

MODIFY (LDAP DN-OR-ENTRY LIST-OF-MODS)

NEXT-SEARCH-RESULT (LDAP)

REPLACE-ATTR (ENTRY ATTR VALS)

UNBIND (LDAP)

Private

Undocumented

CLOSE-STREAM (LDAP)

CODE (CONDITION)

CODE-SYM (CONDITION)

DEBUG-MESG (LDAP MESSAGE)

FILTER (CONDITION)

GET-STREAM (LDAP)

HANDLE-EXTENDED-RESPONSE (LDAP CONTENT)

MESG-INCF (LDAP)

MSG (CONDITION)

MSG-ABANDON (LDAP)

MSG-ADD (ENTRY)

MSG-BIND (LDAP)

MSG-UNBIND

PARSE-LDAP-MESSAGE (LDAP &OPTIONAL (RETURN-ENTRY))

POSSIBLY-REOPEN-AND-REBIND (LDAP &OPTIONAL (ABSOLUTELY-NO-BIND))

PROBABLY-BINARY-FIELD-ERROR-KEY (CONDITION)

PROCESS-MESSAGE (LDAP MESSAGE &KEY (SUCCESS 'SUCCESS))

RECEIVE-LENGTH (LDAP)

RECEIVE-MESSAGE (LDAP)

SEND-MESSAGE (LDAP MESSAGE &OPTIONAL (RESPONSE-EXPECTED))

SLOT-ACCESSOR

Public

REUSE-CONNECTION (OBJECT)

nil, t, or bind

SETFREUSE-CONNECTION (NEW-VALUE OBJECT)

nil, t, or bind

Undocumented

ATTRS (OBJECT)

SETFATTRS (NEW-VALUE OBJECT)

BASE (OBJECT)

SETFBASE (NEW-VALUE OBJECT)

DEBUGFLAG (OBJECT)

SETFDEBUGFLAG (NEW-VALUE OBJECT)

DN (CONDITION)

SETFDN (NEW-VALUE OBJECT)

HOST (CONDITION)

SETFHOST (NEW-VALUE OBJECT)

PASS (OBJECT)

SETFPASS (NEW-VALUE OBJECT)

PORT (CONDITION)

SETFPORT (NEW-VALUE OBJECT)

RDN (OBJECT)

SETFRDN (NEW-VALUE OBJECT)

RESPONSE (OBJECT)

SETFRESPONSE (NEW-VALUE OBJECT)

RESULTS-PENDING-P (OBJECT)

SETFRESULTS-PENDING-P (NEW-VALUE OBJECT)

SSLFLAG (OBJECT)

SETFSSLFLAG (NEW-VALUE OBJECT)

USER (OBJECT)

SETFUSER (NEW-VALUE OBJECT)

Private

Undocumented

ENTRY-BUFFER (OBJECT)

SETFENTRY-BUFFER (NEW-VALUE OBJECT)

LDAPSOCK (OBJECT)

SETFLDAPSOCK (NEW-VALUE OBJECT)

LDAPSTREAM (OBJECT)

SETFLDAPSTREAM (NEW-VALUE OBJECT)

MESG (CONDITION)

SETFMESG (NEW-VALUE OBJECT)

SEARCH-FN (OBJECT)

SETFSEARCH-FN (NEW-VALUE OBJECT)

URL (OBJECT)

SETFURL (NEW-VALUE OBJECT)

VARIABLE

Private

*HEX-PRINT*

Format directive to print a list of line wrapped hex numbers.

Undocumented

*BINARY-ATTRIBUTES*

*LDAP-FILTER-PARSER*

CLASS

Public

Undocumented

ENTRY

LDAP

Private

Undocumented

REFERRER

CONDITION

Public

PROBABLY-BINARY-FIELD-ERROR

Condition that is signalled when a binary field is being parsed as a string

Private

Undocumented

LDAP-BIND-ERROR

LDAP-CONNECTION-ERROR

LDAP-ERROR

LDAP-FILTER-ERROR

LDAP-RESPONSE-ERROR

CONSTANT

Private

+BER-LONG-LENGTH-MARKER+

Flag indicating more tag number bytes follow

+BER-MULTIBYTE-TAG-NUMBER+

Flag indicating tag number requires > 1 byte

+LDAP-CONTROL-EXTENSION-PAGING+

OID of the paging control.

+LDAP-DISCONNECTION-RESPONSE+

OID of the unsolicited disconnection reponse.

+LDAP-PORT-NO-SSL+

Default LDAP Port.

+LDAP-PORT-SSL+

Default LDAPS Port.

+LDAP-VERSION+

LDAP version 3.

+MAX-INT+

As defined by the LDAP RFC.

Undocumented

+BER-ABANDON-TAG+

+BER-ADD-TAG+

+BER-APP-CONST-BASE+

+BER-BIND-TAG+

+BER-CLASS-ID+

+BER-COMP-TAG+

+BER-CONTROLS-TAG+

+BER-DEL-TAG+

+BER-MODDN-TAG+

+BER-MODIFY-TAG+

+BER-P/C-BIT+

+BER-SEARCH-TAG+

+BER-TAG-BOOL+

+BER-TAG-CONTROLS+

+BER-TAG-ENUM+

+BER-TAG-EXT-NAME+

+BER-TAG-EXT-VAL+

+BER-TAG-EXTENDEDRESPONSE+

+BER-TAG-INT+

+BER-TAG-REFERRAL+

+BER-TAG-SEQ+

+BER-TAG-SET+

+BER-TAG-STR+

+BER-UNBIND-TAG+

+LDAP-APPLICATION-NAMES+

+LDAP-DEREF+

+LDAP-FILTER-COMPARISON-CHAR+

+LDAP-MODIFY-TYPE+

+LDAP-RESULT-CODES+

+LDAP-SCOPE+

+LDAP-SUBSTRING+