Previous Chapter Next Section Table of Contents Glossary Index

Chapter 10. Streams

10.1. Stream Extensions

10.1.1. Stream External Format

STREAM-EXTERNAL-FORMAT can be applied to (and may return a non-null result for) open streams that are not FILE-STREAMs.

(SETF STREAM-EXTERNAL-FORMAT) can be used to change the external format of open streams created with OPEN or MAKE-SOCKET.

10.1.2. Additional keywords for OPEN and MAKE-SOCKET

OPEN and MAKE-SOCKET have each been extended to take the additional keyword arguments: :CLASS, :SHARING, and :BASIC.

:CLASS

A symbol that names the desired class of the stream. The specified class must inherit from FILE-STREAM for OPEN.

:SHARING

Specifies how a stream can be used by multiple threads. The possible values are: :PRIVATE, :LOCK and :EXTERNAL. :PRIVATE is the default. NIL is also accepted as a synonym for :EXTERNAL.

:PRIVATE

Specifies that the stream can only be accessed by the thread that first tries to do I/O to it; that thread becomes the "owner" of the stream and is not necessarily the same thread as the one which created the stream. This is the default. (There was some discussion on openmcl-devel about the idea of "transferring ownership" of a stream; this has not yet been implemented.) Attempts to do I/O on a stream with :PRIVATE sharing from a thread other than the stream's owner yield an error.

:LOCK

Specifies that all access to the stream require the calling thread to obtain a lock. There are separate "read" and "write" locks for IO streams. This makes it possible for instance, for one thread to read from such a stream while another thread writes to it. (see also make-read-write-lock with-read-lock with-write-lock)

:EXTERNAL

Specifies that I/O primitives enforce no access protocol. This may be appropriate for some types of application which can control stream access via application-level protocols. Note that since even the act of reading from a stream changes its internal state (and simultaneous access from multiple threads can therefore lead to corruption of that state), some care must be taken in the design of such protocols.

:BASIC

A boolean that indicates whether or not the stream is a Gray stream, i.e. whether or not the stream is an instance of FUNDAMENTAL-STREAM or CCL::BASIC-STREAM(see Section 10.1.3, “Basic Versus Fundamental Streams”). Defaults to T.

10.1.3. Basic Versus Fundamental Streams

Gray streams (see Section 10.2, “Creating Your Own Stream Classes with Gray Streams”) all inherit from FUNDAMENTAL-STREAM whereas basic streams inherit from CCL::BASIC-STREAM. The tradeoff between FUNDAMENTAL and BASIC streams is entirely between flexibility and performance, potential or actual. I/O primitives can recognize BASIC-STREAMs and exploit knowledge of implementation details. FUNDAMENTAL stream classes can be subclassed and extended in a standard way (the Gray streams protocol).

For existing stream classes (FILE-STREAMs, SOCKETs, and the internal CCL::FD-STREAM classes used to implement file streams and sockets), a lot of code can be shared between the FUNDAMENTAL and BASIC implementations. The biggest difference should be that that code can be reached from I/O primitives like READ-CHAR without going through some steps that're there to support generality and extensibility, and skipping those steps when that support isn't needed can improve I/O performance.

The Gray stream method STREAM-READ-CHAR should work on appropriate BASIC-STREAMs. (There may still be cases where such methods are undefined; such cases should be considered bugs.) It is not guaranteed that Gray stream methods would ever be called by I/O primitives to read a character from a BASIC-STREAM, though there are still cases where this happens.

A simple loop reading 2M characters from a text file runs about 10X faster when the file is opened the new defaults (:SHARING :PRIVATE :BASIC T) than it had before these changes were made. That sounds good, until one realizes that the "equivalent" C loop can be about 10X faster still ...

10.1.4. Stream Timeouts and Deadlines

A stream that is associated with a file descriptor has attributes and accessors: STREAM-INPUT-TIMEOUT, STREAM-OUTPUT-TIMEOUT, and STREAM-DEADLINE. All three accessors have corresponding SETF methods. STREAM-INPUT-TIMEOUT and STREAM-OUTPUT-TIMEOUT are specified in seconds and can be any positive real number less than one million. When a timeout is set and the corresponding I/O operation takes longer than the specified interval, an error is signalled. The error is INPUT-TIMEOUT for input and OUTPUT-TIMEOUT for output. STREAM-DEADLINE specifies an absolute time in internal-time-units. If an I/O operation on the stream does not complete before the deadline then a COMMUNICATION-DEADLINE-EXPIRED error is signalled. A deadline takes precedence over any input/output timeouts that may be set.

10.1.5. Open File Streams

Historically, Clozure CL and MCL maintained a list of open file streams in the value of CCL:*OPEN-FILE-STREAMS*. This functionality has been replaced with the thread-safe function: CCL:OPEN-FILE-STREAMS and its two helper functions: CCL:NOTE-OPEN-FILE-STREAM and CCL:REMOVE-OPEN-FILE-STREAM. Maintaining this list helps to ensure that streams get closed in an orderly manner when the lisp exits.

[Function]

open-file-streams => stream-list
Returns the list of file streams that are currently open.

Values:

stream-list---A list of open file streams. This is a copy of an internal list so it may be destructively modified without ill effect.

Description:

Returns a list of open file streams.

[Function]

note-open-file-stream file-stream
Adds a file stream to the internal list of open file streams that is returned by note-open-file-stream.

Arguments:

file-stream---A file stream.

Description:

Adds a file stream to the internal list of open file streams that is returned by open-file-streams. This function is thread-safe. It will usually only be called from custom stream code when a file-stream is created.

[Function]

remove-open-file-stream file-stream
Removes file stream from the internal list of open file streams that is returned by open-file-streams.

Arguments:

file-stream---A file stream.

Description:

Remove file stream from the internal list of open file streams that is returned by open-file-streams. This function is thread-safe. It will usually only be called from custom stream code when a file-stream is closed.


Previous Chapter Next Section Table of Contents Glossary Index