Common Lisp Package: COM.INFORMATIMAGO.COMMON-LISP.DATA-ENCODING.BENCODE

Bencoding is a way to encode integers, strings, lists, and hash-tables as strings (serialization), and bdecoding does the reverse operation. It is part of the torrent metafile specification at http://bittorrent.org/beps/bep_0003.html Notice that torrent files are binary files, since they contain strings with octets instead of characters. Data encoded and decoded ------------------------ +----------------------+----------------------+----------------------+ | Lisp type | BEncoded type | Lisp type | +----------------------+----------------------+----------------------+ | integer | integer (i) | integer | +----------------------+----------------------+----------------------+ | octets | string (:) | string or octets¹ | +----------------------+----------------------+----------------------+ | string | string (:) | string or octets¹ | +----------------------+----------------------+----------------------+ | symbol | string (:) | string² | +----------------------+----------------------+----------------------+ | character | string (:) | string | +----------------------+----------------------+----------------------+ | hash-table³ | dict (d) | hash-table | +----------------------+----------------------+----------------------+ | list | list (l) | list | +----------------------+----------------------+----------------------+ ¹: OCTETS are a vector of (unsigned-byte 8), if a string contains only the standard characters, then it's read as a string, otherwise as an OCTETS. ²: in a dict, key strings can be converted into symbols. ³: all keys must be string designators. Key map ------- Encoded dictionaries use strings as key. Sometimes, the string contains spaces, sometimes it contains dashes. It could contain any characters. By default, we map keywords to strings by merely downcasing them (and vice-versa, by interning the upcased string in the KEYWORD package). To map keywords to a different string, we use key-map-exceptions. Since dictionaries may contain other dictionaries as values, an entry in the key-map-exceptions may specify another key-map-exceptions to be used when reading the value of the dictionary entry. A key-map-exceptions is an a-list mapping keywords to either: - a string, or - a p-list containing the keys :string and/or :key-map. The string substitutes the keyword in the serialized dictionary; the key-map is another key-map exceptions a-list to be used when reading the value of that dictionary entry. *TORRENT-KEY-MAP-EXCEPTIONS* is the key-map-exceptions that can be used to read and write torrent files, binding it to *KEY-MAP-EXCEPTIONS*. Decoding BEncoded strings or streams ------------------------------------ :: (with-open-stream (torrent "example.torrent" :element-type '(unsigned-byte 8)) ;; torrent files contain binary 'strings'... (let ((*key-map-exceptions* *torrent-key-map-exceptions*)) (bdecode-from-binary-stream torrent))) (with-open-stream (stream "file.bencoded") (bdecode-from-character-stream torrent)) (bdecode-from-string "li43e4:spaml4:spam4:eggsed3:cow3:moo4:spam4:eggsed4:spaml1:a1:beee") Encoding BEncoded strings or streams ------------------------------------ :: (with-open-stream (torrent "example.torrent" :element-type '(unsigned-byte 8) :direction :output :if-does-not-exist :create :if-exists :supersede) ;; torrent files contain binary 'strings'... (let ((*key-map-exceptions* *torrent-key-map-exceptions*)) (bencode-to-binary-stream object torrent))) (with-open-stream (stream "file.bencoded" :direction :output :if-does-not-exist :create :if-exists :supersede) (bencode-to-character-stream object stream)) (bencode-to-string (list 42 '("aa" bb #\c :dd) (hashtable :elements '((one 1) ("two" "2") (:three-and-one-third (x x x)))) (hashtable :elements '((:one 1) (:two "2") (:three-and-one-third (x x x)))))) --> "li42el2:aa2:BB1:c2:DDed3:onei1e19:three-and-one-thirdl1:X1:X1:Xe3:two1:2ed3:onei1e19:three-and-one-thirdl1:X1:X1:Xe3:two1:2ee" License: AGPL3 Copyright Pascal J. Bourguignon 2010 - 2012 This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see <http://www.gnu.org/licenses/>

README:

FUNCTION

Public

BDECODE-FROM-BINARY-STREAM (INPUT)

Reads a BEncoded object from the INPUT binary stream.

BDECODE-FROM-CHARACTER-STREAM (INPUT)

Reads a BEncoded object from the INPUT character stream.

BDECODE-FROM-STRING (BENCODED-STRING)

Returns the object BEncoded in the BENCODED-STRING.

BENCODE-TO-BINARY-STREAM (OBJECT OUTPUT)

Writes a BEncoded OBJECT to the OUTPUT binary stream.

BENCODE-TO-CHARACTER-STREAM (OBJECT OUTPUT)

Writes a BEncoded OBJECT to the OUTPUT character stream.

BENCODE-TO-STRING (OBJECT)

Returns a string containing the BEncoded OBJECT.

Private

DECODE (STREAM)

Reads a BEncoded object from the STREAM stream-wrapper.

DECODE-KEY-MAP-EXCEPTION (KEY-STRING)

Searches in *KEY-MAP-EXCEPTIONS* an exceptional mapping of the string KEY-STRING.

DECODE-RAW-CARDINAL (STREAM)

Read a sequence of digits, converting them into a positive integer. There must be at least one digit.

DECODE-RAW-INTEGER (STREAM)

Reads a integer composed of an optional minus sign, and a sequence of at least one digit.

ENCODE-KEY-MAP-EXCEPTION (KEYWORD)

Searches in *KEY-MAP-EXCEPTIONS* an exceptional mapping of the keyword KEYWORD.

Undocumented

CHARACTER-CODE-P (CODE)

DECODE-DICTIONARY (STREAM)

DECODE-INTEGER (STREAM)

DECODE-KEY (KEY-STRING)

DECODE-LIST (STREAM)

DECODE-OBJECT (STREAM)

DECODE-OCTETS (STREAM)

DECODE-STRING (STREAM)

EAT-END (STREAM CONTEXT &REST ARGUMENTS)

ENCODE-KEY (KEYWORD)

ENCODE-OCTETS (OCTETS BINARY-STREAM)

KEY-MAP-ENTRY-FOR-KEY (KEYWORD)

KEY-MAP-ENTRY-KEY-MAP (ENTRY)

KEY-MAP-ENTRY-KEYWORD (ENTRY)

KEY-MAP-ENTRY-STRING (ENTRY)

KEY-MAP-ENTRY-WITH-STRING (KEY-STRING)

KEY-MAP-EXCEPTION (KEYWORD)

LOOKING-AT-END (STREAM)

TEST/STRING (&KEY (VERBOSE NIL))

GENERIC-FUNCTION

Private

Undocumented

ENCODE-OBJECT (VALUE STREAM)

STREAM-PEEK-CHAR (STREAM-WRAPPER &OPTIONAL EOF-ERROR-P EOF-VALUE)

STREAM-READ-CHAR (STREAM-WRAPPER &OPTIONAL EOF-ERROR-P EOF-VALUE)

STREAM-READ-OCTETS (OCTETS STREAM-WRAPPER)

STREAM-READ-STRING (STRING STREAM-WRAPPER)

STREAM-WRITE-OCTETS (OCTETS STREAM-WRAPPER)

STREAM-WRITE-STRING (STRING STREAM-WRAPPER)

SLOT-ACCESSOR

Private

Undocumented

GET-STREAM (OBJECT)

VARIABLE

Public

*KEY-MAP-EXCEPTIONS*

Current key-map exceptions. A key-map-exceptions is an a-list mapping keywords to either: - a string, or - a p-list containing the keys :string and/or :key-map. The string substitutes the keyword in the serialized dictionary; the key-map is another key-map exceptions a-list to be used when reading the value of that dictionary entry.

*TORRENT-KEY-MAP-EXCEPTIONS*

Key-map exceptions for torrent files.

Private

Undocumented

*CHARACTER-CODES*

CLASS

Private

Undocumented

BINARY-STREAM

CHARACTER-STREAM

STREAM-WRAPPER