Common Lisp Package: S-HTTP-CLIENT

A Basic HTTP Client

README:

S-HTTP-CLIENT

A Common Lisp HTTP Client

S-HTTP-CLIENT is a basic implementation of a Common Lisp HTTP Client. This simple package is used as a building block in a number of other open source projects.

Contents

Features

S-HTTP-CLIENT can:

  • access any URL over HTTP/HTTPS using any HTTP verb
  • support connection reuse
  • support basic authentication

Status

S-HTTP-CLIENT is considered stable code.

News

Platforms

S-HTTP-CLIENT is written in ANSI standard Common Lisp and should be portable across any CL implementation, provided parts of S-SYSDEPS are ported.

Installation

The S-HTTP-CLIENT package is loaded using ASDF. There is an excellent tutorial on ASDF to get you started.

CL-USER 1 > (asdf:oos 'asdf:load-op :s-http-client) 

Example of setting up and using ASDF to compile and load the package

Usage

Just call DO-HTTP-REQUEST with the necessary arguments. For example:

CL-USER 1 > (in-package :s-http-client)  
#<The S-HTTP-CLIENT package, 59/128 internal, 5/16 external>  
 
S-HTTP-CLIENT 2 > (s-http-client:do-http-request "http://homepage.mac.com/svc/s-http-client/foo.html")  
"<html><head><title>Foo</title></head><body><h1>Foo</h1><p>Just a test page.</p></body></html>  
"  
200  
((:DATE . "Sat, 19 Nov 2005 22:05:38 GMT")  
 (:CONTENT-LENGTH . "100")  
 (:CONTENT-TYPE . "text/html")  
 (:CACHE-CONTROL . "public")  
 (:X-RESPONDING-SERVER . "webdav11")  
 (:SERVER . "AppleDotMacServer")  
 (:ETAG . "14djc8on-8nlp-j19yhvvj5-c37cvzcpi0")  
 (:LAST-MODIFIED . "Sat, 19 Nov 2005 22:02:31 GMT")  
 (:VIA . "1.1 netcache02 (NetCache NetApp/5.5R6)"))  
#<URI http://homepage.mac.com:80/svc/s-http-client/foo.html>  
:NEW 

In principle, any HTTP verb is allowed. It is the easiest to specify them as keywords. In the case of :POST or :PUT, content as well as a content-type should be specified (content length will be calculated). Content can be specified as a string or byte sequence as far as write-sequence can be called on your platforms TCP/IP stream. Basic authorization is to be specified as (username . password) strings.

Connections are kept alive and reused as long as the same host and port are accessed using the same scheme. Both HTTP/1.0 with Keep-Alive headers as well as HTTP/1.1 is supported. Chunked transfer encoding is also implemented.

API Reference

There is automatically generated in documentation available for the S-HTTP-CLIENT package in doc/API.html

Mailinglist

There is no mailing list for this project.

Changelog

Release Notes:

  • release 1: moved S-HTTP-CLIENT into a seperate project under a new structure

TODO

  • Proxy support is currently not implemented, although it is not really hard.

FAQ

Nothing appropriate.

Bugs

  • HTTPS in only implemented on LispWorks (where it was really easy).

Authors

S-HTTP-CLIENT was written by Sven Van Caekenberghe.

Maintainers

S-HTTP-CLIENT is being maintained by Sven Van Caekenberghe.

License

You are granted the rights to distribute and use this software as governed by the terms of the Lisp Lesser General Public License (http://opensource.franz.com/preamble.html), also known as the LLGPL.

History

This is a new project.

References

There are no references.

Copyright © 2005 Sven Van Caekenberghe, Beta Nine BVBA. All Right Reserved.

FUNCTION

Public

CLOSE-ALL-CONNECTIONS (&OPTIONAL (HTTP-CLIENT-STATE *DEFAULT-HTTP-CLIENT-STATE*) &KEY ABORT)

Close all open connections in http-client-state, optionaly aborting them

DO-HTTP-REQUEST (URI &KEY (METHOD GET) CONTENT CONTENT-TYPE CONTENT-LENGTH BODY BASIC-AUTHORIZATION HEADERS PROXY (STATE *DEFAULT-HTTP-CLIENT-STATE*) CONNECT-TIMEOUT READ-TIMEOUT WRITE-TIMEOUT)

Execute an HTTP request, returns (VALUES body code headers uri kept-alive-state)

MAKE-HTTP-CLIENT-STATE

Make a new HTTP client state object to hold open (keepalive) connections

URI-DECODE-FOR-QUERY (STRING)

URI decode string from a query parameter value

URI-ENCODE-FOR-QUERY (STRING &KEY (SIGNAL-ERRORS T))

URI encode string for use as a query parameter value

Private

CLOSE-CONNECTION (CONNECTION &KEY ABORT)

Close a socket connection stream, optionally aborting

DO-ONE-REQUEST-RESPONSE (CONNECTION URI METHOD BUFFER &KEY CONTENT CONTENT-TYPE CONTENT-LENGTH BODY BASIC-AUTHORIZATION HEADERS)

Do one HTTP request and response on stream

GET-OPEN-CONNECTION (SCHEME HOST PORT STATE &KEY FORCE-NEW CONNECT-TIMEOUT READ-TIMEOUT WRITE-TIMEOUT)

Get an open connection to scheme://host:port reusing one from state unless force-new is t. Release after use

READ-CRLF-LINE (BUFFER STREAM &OPTIONAL (EOF-ERROR-P T) EOF-VALUE)

Read a CRLF termintated line from a character input stream into buffer. Return length excluding CRLF.

READ-RESPONSE (STREAM BUFFER &OPTIONAL BODY (BODY-EXPECTED T))

Read an HTTP response, headers and content, from stream

RELEASE-CONNECTION (SCHEME HOST PORT STATE CONNECTION)

Release a connection obtained through get-open-connection (either stored in state for reuse or not)

WRITE-REQUEST (STREAM BUFFER URI METHOD &KEY CONTENT CONTENT-TYPE CONTENT-LENGTH BASIC-AUTHORIZATION HEADERS)

Write an HTTP request, full header and body, to stream

Undocumented

CONCATENATE-CHUNKS (CHUNKS TOTAL-SIZE)

DO-HTTP-REQUEST-INTERNAL (SCHEME HOST PORT STATE METHOD URI CONTENT CONTENT-TYPE CONTENT-LENGTH BODY BASIC-AUTHORIZATION HEADERS CONNECT-TIMEOUT READ-TIMEOUT WRITE-TIMEOUT)

ENCODE-BASIC-AUTHORIZATION (USERNAME-PASSWORD-PAIR)

FORMAT-HTTP-REQUEST-LINE (STREAM FORMAT-STRING &REST ARGS)

GUNZIP (STRING)

HEADER-FIELD-NAME->KEYWORD (STRING &OPTIONAL (START 0) END)

HEADER-FIELD-VALUE->STRING (STRING &OPTIONAL (START 0) END)

OPEN-SOCKET-STREAM (SCHEME HOST PORT &KEY CONNECT-TIMEOUT READ-TIMEOUT WRITE-TIMEOUT)

RESPONSE-READ-BODY (STREAM LENGTH BODY BUFFER)

RESPONSE-READ-CHUNKED-BODY (STREAM BODY BUFFER)

RESPONSE-READ-CODE (STREAM BUFFER)

RESPONSE-READ-HEADERS (STREAM BUFFER)

RESPONSE-READ-UNDEFINED-BODY (STREAM BODY BUFFER)

WRITE-HTTP-REQUEST-LINE (STRING STREAM)

GENERIC-FUNCTION

Private

Undocumented

CLEANUP-OLD-CONNECTIONS (HTTP-CLIENT-STATE)

CLOSE-ALL-CONNECTIONS-INTERNAL (HTTP-CLIENT-STATE &KEY ABORT)

GET-ENDPOINT (CONDITION)

SETFGET-ENDPOINT (NEW-VALUE CONDITION)

GET-HTTP-SERVER-STATE (HTTP-CLIENT-STATE SCHEME-HOST-PORT)

GET-KEPT-ALIVE-AGE (HTTP-SERVER-STATE)

GET-TIMEOUT (CONDITION)

SETFGET-TIMEOUT (NEW-VALUE CONDITION)

SLOT-ACCESSOR

Private

Undocumented

GET-BUFFER (OBJECT)

SETFGET-BUFFER (NEW-VALUE OBJECT)

GET-IN-USE-P (OBJECT)

SETFGET-IN-USE-P (NEW-VALUE OBJECT)

GET-LOCK (OBJECT)

GET-SCHEME-HOST-PORT (OBJECT)

SETFGET-SCHEME-HOST-PORT (NEW-VALUE OBJECT)

GET-SOCKET (OBJECT)

SETFGET-SOCKET (NEW-VALUE OBJECT)

GET-TIMESTAMP (OBJECT)

SETFGET-TIMESTAMP (NEW-VALUE OBJECT)

VARIABLE

Public

*DEFAULT-HTTP-CLIENT-STATE*

The default, globally shared HTTP client state

*HTTP-CLIENT-ACCEPT*

The value to use for the 'Accept' request header

*HTTP-CLIENT-ACCEPT-ENCODING*

The value to use for the 'Accept-Encoding' request header

*HTTP-CLIENT-AGENT*

The value to use for the 'User-Agent' request header

Private

*CONNECTION-KEEP-ALIVE-TIMEOUT*

Keep-Alive connections not used for this amount of time will be force closed, eventually

Undocumented

+COMMON-RESPONSE-HEADERS+

+URI-ENCODE-CHAR-MAP+

CLASS

Public

HTTP-CLIENT-STATE

Object holding all HTTP client state

Private

HTTP-SERVER-STATE

Object holding a reusable open connection to a scheme://host:port

CONDITION

Private

Undocumented

HTTP-CLIENT-CONNECTION-ERROR

HTTP-CLIENT-ERROR

HTTP-CLIENT-READ-ERROR

HTTP-CLIENT-WRITE-ERROR