12.2 binary.pack - Packing binary data

Module: binary.pack ¶

This module provides an interface for packing and unpacking (writing and reading) binary data with templates. The functionality was inspired largely by the Perl pack/unpack functions, with comparison of similar features from other languages, however an effort was made to make it more general and more efficient, to be usable for database-like processing. To that end, the most notable differences are that any packable value is unpackable (and vice versa), and the default behavior is to pack and unpack using port I/O, so you can seek in a large file and unpack from it. Also, templates may be stored as dispatch closures to pack, unpack or even skip over values without re-parsing the template.

Function: pack template list :key output to-string? ¶

{binary.pack} Writes the values in list to the current output port, according to the format specified by the string template. The template string is a series of single character codes, optionally followed by a numeric count (which defaults to 1). The format characters can generally be divided into string types, which interpret the count as a string byte size, and object types, which treat the count as a repetition indicator. The count may be specified as the character *, which means to use the full size of the string for string types, and use all remaining values for object types. Counts may also be specified as a template enclosed in brackets, which means the count is the byte size of the enclosed template. For example, x[L] skips a long. The special format character / may be used to indicate a structure where the packed data contains a dynamic count followed by the value itself. The template is written as <count-item>/<value-item>, where <count-item> is any template character to be interpreted as a numeric count, and <value-item> is any other template character to use this count. If a normal count is given after <value-item> it is ignored. The format character @ may be used with a count to pad to an absolute position since the start of the template. Sub-templates may be grouped inside parentheses. If angle-brackets are used, then they also behave as group operators but recursively operate on nested lists. The string types:

a

An arbitrary incomplete string, null padded.

A

A text string, space padded.

Z

A null terminated (ASCIZ) string, null padded.

b

A bit string (ascending bit order inside each byte).

B

A bit string (descending bit order inside each byte).

h

A hex string (low nybble first).

H

A hex string (high nybble first).

The object types:

c

A signed 8bit integer.

C

An unsigned 8bit integer.

s

A signed short (16 bit) value.

S

An unsigned short (16 bit) value.

i

A signed integer (>= 32 bit) value.

I

An unsigned integer (>= 32 bit) value.

l

A signed long (32 bit) value.

L

An unsigned long (32 bit) value.

n, n!

An unsigned and signed short (16 bit) in "network" (big-endian) order.

N, N!

An unsigned and signed long (32 bit) in "network" (big-endian) order.

v, v!

An unsigned and signed short (16 bit) in "VAX" (little-endian) order.

V, V!

An unsigned and signed long (32 bit) in "VAX" (little-endian) order.

q

A signed quad (64 bit) value.

Q

An unsigned quad (64 bit) value.

f

A single-precision float in the native format.

d

A double-precision float in the native format.

w

A BER compressed integer. An unsigned integer in base 128, most significant digit first, where the high bit is set on all but the final (least significant) byte. Thus any size integer can be encoded, but the encoding is efficient and small integers don’t take up any more space than they would in normal char/short/int encodings.

x

A null byte.

o

An sexp, handled with read and write.

If the optional keyword :output is given that port is used instead of the current output port. If :to-string? is given and true, then pack accumulates and returns the output as a string.

Note that the returned string may be an incomplete string if the packed string contains a byte sequence invalid as a character sequence.

(pack "CCCC" '(65 66 67 68) :to-string? #t)
 ⇒ "ABCD"

(pack "C/a*" '("hello") :to-string? #t)
 ⇒ "\x05hello"

Function: unpack template :key :input :from-string ¶

{binary.pack} The complement of pack, unpack reads values from the current input port assuming they’ve been packed according to the string template and returns the values as a list. unpack accepts the same format strings as pack. Further, the following tautology holds:

(equal? x (unpack fmt :from-string (pack fmt x :to-string? #t)))

for any list x and format string fmt. The only exceptions to this are when the template includes a * and when the o template is used, since Scheme numeric literals cannot be reliably delimited (though future versions of pack may circumvent this by registering a new read syntax).

If the optional keyword :input is given that port is used instead of the current input port. If :from-string is given, then pack reads input from that string.

(unpack "CCCC" :from-string "ABCD")
 ⇒ '(65 66 67 68)

(unpack "C/a*" :from-string "\x05hello")
 ⇒ '("hello")

Note: in the current version, @ in unpack template has a bug and does not work as supposed. It will be fixed in the future version.

Function: unpack-skip template :key :input ¶

{binary.pack} unpack-skip is the same as unpack except it does not return the values. In some cases, particularly with fixed-size templates, this can be much more efficient when you just want to skip over a value.

Function: make-packer template ¶

{binary.pack} The low-level interface. This function returns a dispatch closure that can be used to pack, unpack and skip over the same cached template. The dispatch closure accepts symbol methods as follows:

'pack list

pack the items in list to the current output port.

'unpack

unpack items from the current input port.

'skip

skip items from the current input port.

'packer

return the cached ’pack closure

'unpacker

return the cached ’unpack closure.

'skipper

return the cached ’skip closure.

'length

return the known fixed length of the template.

'variable-length?

return #t if the template has variable length elements.