sxml.ssax- Functional XML parser
sxml.* modules are the adaptation of
Oleg Kiselyov’s SXML framework (SSAX),
which is based on S-expression representation of XML structure.
SSAX is a parser part of SXML framework. This is a quote from SSAX webpage:
A SSAX functional XML parsing framework consists of a DOM/SXML parser, a SAX parser, and a supporting library of lexing and parsing procedures. The procedures in the package can be used separately to tokenize or parse various pieces of XML documents. The framework supports XML Namespaces, character, internal and external parsed entities, attribute value normalization, processing instructions and CDATA sections. The package includes a semi-validating SXML parser : a DOM-mode parser that is an instantiation of a SAX parser (called SSAX).
The current version is based on the SSAX CVS version newer than
the last ’official’ release of SXML toolset (4.9), and
SXML-gauche-0.9 package which was based on SXML-4.9.
There is an important change from that release.
Now the API uses lowercase letter suffix
instead of uppercase
SSAX:—the difference matters since
Gauche is case sensitive by default.
Alias names are defined for backward compatibility,
but the use of uppercase suffixed names are deprecated.
I derived the content of this part of the manual from SSAX source code, just by converting its comments into texinfo format. The original text is by Oleg Kiselyov. Shiro Kawai should be responsible for any typographical error or formatting error introduced by conversion.
The manual entries are ordered in "bottom-up" way, beginning from
the lower-level constructs towards the high-level utilities.
If you just want to parse XML document and obtain SXML,
ssax:xml->sxml in SSAX Highest-level parsers - XML to SXML.
|• SSAX data types:|
|• SSAX low-level parsing code:|
|• SSAX higher-level parsers and scanners:|
|• SSAX Highest-level parsers - XML to SXML:|
a symbol ’
ENTITY-REF that identifies a markup token.
a name (called
GI in the XML Recommendation) as given in an xml
document for a markup token: start-tag,
PI target, attribute name.
GI is an
NCName, UNRES-NAME is this
NCName converted into
a Scheme symbol. If a
GI is a
QName, UNRES-NAME is a pair of
symbols: (PREFIX . LOCALPART)
An expanded name, a resolved version of an UNRES-NAME. For an element or an attribute name with a non-empty namespace URI, RES-NAME is a pair of symbols, (URI-SYMB . LOCALPART). Otherwise, it’s a single symbol.
|anything goes, expect an END tag.|
|no content, and no END-tag is coming.|
|no content, expect the END-tag as the next token.|
|expect character data only, and no children elements.|
A symbol representing a namespace URI – or other symbol chosen
by the user to represent URI. In the former case,
URI-SYMB is created by
%-quoting of bad URI characters and
converting the resulting string into a symbol.
A list representing namespaces in effect. An element of the list has one of the following forms:
(prefix uri-symb . uri-symb)
(prefix user-prefix . uri-symb)
user-prefix is a symbol chosen by the user to represent the URI.
(#f user-prefix . uri-symb)
Specification of the user-chosen prefix and a uri-symbol.
(*DEFAULT* user-prefix . uri-symb)
Declaration of the default namespace
(*DEFAULT* #f . #f)
Un-declaration of the default namespace. This notation represents overriding of the previous declaration
A NAMESPACES list may contain several elements for the same PREFIX. The one closest to the beginning of the list takes effect.
An ordered collection of (NAME . VALUE) pairs, where NAME is a RES-NAME or an UNRES-NAME. The collection is an ADT.
A procedure of three arguments:
(string1 string2 seed)
returning a new seed.
The procedure is supposed to handle a chunk of character data
string1 followed by a chunk of character data string2.
string2 is a short string, often "\n" and even ""
An assoc list of pairs:
(named-entity-name . named-entity-body)
where named-entity-name is a symbol under which the entity was
declared, named-entity-body is either a string, or
(for an external entity) a thunk that will return an
input port (from which the entity can be read).
named-entity-body may also be
#f. This is an indication that a
named-entity-name is currently being expanded. A reference to
this named-entity-name will be an error: violation of the
A record with two slots, kind and token. This record represents a markup, which is, according to the XML Recommendation, "takes the form of start-tags, end-tags, empty-element tags, entity references, character references, comments, CDATA section delimiters, document type declarations, and processing instructions."
an UNRES-NAME. For xml-tokens of kinds ’
CDSECT, the head is
<P> => kind='START, head='P </P> => kind='END, head='P <BR/> => kind='EMPTY-EL, head='BR <!DOCTYPE OMF ...> => kind='DECL, head='DOCTYPE <?xml version="1.0"?> => kind='PI, head='xml &my-ent; => kind = 'ENTITY-REF, head='my-ent
Character references are not represented by xml-tokens as these references are transparently resolved into the corresponding characters.
A record with three slots, elems, entities, and notations.
The record represents a datatype of an XML document: the list of declared elements and their attributes, declared notations, list of replacement strings or loading procedures for parsed general entities, etc. Normally an xml-decl record is created from a DTD or an XML Schema, although it can be created and filled in in many other ways (e.g., loaded from a file).
elems: an (assoc) list of decl-elem or
#f. The latter instructs
the parser to do no validation of elements and attributes.
decl-elem: declaration of one element:
(elem-name elem-content decl-attrs);
elem-name is an UNRES-NAME for the element.
elem-content is an ELEM-CONTENT-MODEL.
decl-attrs is an ATTLIST,
(attr-name . value) associations.
This element can declare a user procedure to handle parsing of an
element (e.g., to do a custom validation, or to build a hash of
IDs as they’re encountered).
decl-attr: an element of an ATTLIST, declaration of one attribute
(attr-name content-type use-type default-value):
attr-name is an UNRES-NAME for the declared attribute;
content-type is a symbol:
CDATA, NMTOKEN, NMTOKENS, ...;
or a list of strings for the enumerated type.
use-type is a symbol:
default-value is a string for the default value, or
#f if not given.
Utility procedures to deal with attribute list, which keeps name-value association.
A constructor and a predicate for a XML-TOKEN record.
Accessor macros of a XML-TOKEN record.
They deal with primitive lexical units (Names, whitespaces, tags)
and with pieces of more generic productions. Most of these parsers
must be called in appropriate context. For example,
must be called only when the start-tag has been detected and its
has been read.
Skip the S (whitespace) production as defined by
 S ::= (#x20 | #x9 | #xD | #xA)
The procedure returns the first not-whitespace character it encounters while scanning the port. This character is left on the input stream.
Check to see if a-char may start a
NCName starting from the current position in the port and
return it as a symbol.
Read a (namespace-) Qualified Name,
QName, from the current
position in the port.
 QName ::= (Prefix ':')? LocalPart  Prefix ::= NCName  LocalPart ::= NCName
Return: an UNRES-NAME.
The prefix of the pre-defined XML namespace, i.e. ’
This procedure starts parsing of a markup token. The current position
in the stream must be
#\<. This procedure scans enough of the input stream
to figure out what kind of a markup token it is seeing. The procedure returns
an xml-token structure describing the token. Note, generally reading
of the current markup is not finished! In particular, no attributes of
the start-tag token are scanned.
Here’s a detailed break out of the return values and the position in the port when that particular value is returned:
PI-target is read.
To finish the Processing Instruction and disregard it,
ssax:read-attributes may be useful
as well (for
PIs whose content is attribute-value
The end tag is read completely; the current position
is right after the terminating
is read and skipped completely. The current position
is right after "
-->" that terminates the comment.
The current position is right after "
ssax:read-cdata-body to read the rest.
We have read the keyword (the one that follows "
identifying this declaration markup. The current
position is after the keyword (usually a
We have read the keyword (
GI) of this start tag.
No attributes are scanned yet. We don’t know if this
tag has an empty content either.
ssax:complete-start-tag to finish parsing of
The current position is inside a
PI. Skip till the rest of the
The current position is right after reading the
PITarget. We read the
PI and return it as a string. The port will point to the
character right after ’
?>’ combination that terminates
 PI ::= '<?' PITarget (S (Char* - (Char* '?>' Char*)))? '?>'
The current pos in the port is inside an internal DTD subset
(e.g., after reading
#\[ that begins an internal DTD subset)
Skip until the "
]>" combination that terminates this DTD
This procedure must be called after we have read a string "
that begins a
CDATA section. The current position must be the first
position of the
CDATA body. This function reads lines of the
body and passes them to a STR-HANDLER, a character data consumer.
The str-handler is a STR-HANDLER, a procedure
string1 string2 seed.
The first string1 argument to STR-HANDLER never contains a newline.
The second string2 argument often will. On the first invocation of
the STR-HANDLER, the seed is the one passed to
as the third argument. The result of this first invocation will be
passed as the seed argument to the second invocation of the line
consumer, and so on. The result of the last invocation of the
STR-HANDLER is returned by the
ssax:read-cdata-body. Note a
similarity to the fundamental ’
CDATA section all characters are taken at their face value,
with only three exceptions:
CRLFare treated as line delimiters, and passed as a single
#\newlineto the STR-HANDLER.
]]>" combination is the end of the
>is treated as an embedded
&are not specially recognized (and are not expanded)!
 CharRef ::= '&#' [0-9]+ ';' | '&#x' [0-9a-fA-F]+ ';'
This procedure must be called after we we have read "
that introduces a char reference.
The procedure reads this reference and returns the corresponding char.
The current position in port will be after "
;" that terminates
the char reference.
According to Section "4.1 Character and Entity References" of the XML Recommendation:
"[Definition: A character reference refers to a specific character in the ISO/IEC 10646 character set, for example one not directly accessible from available input devices.]"
Therefore, we use a
ucscode->char function to convert a character
code into the character – regardless of the current character
encoding of the input stream.
Expand and handle a parsed-entity reference
The result is the one returned by content-handler or str-handler.
WFC: XML-Spec.html#wf-entdeclared WFC: XML-Spec.html#norecursion
This procedure reads and parses a production
 Attribute ::= Name Eq AttValue  AttValue ::= '"' ([^<&"] | Reference)* '"' | "'" ([^<&'] | Reference)* "'"  Eq ::= S? '=' S?
The procedure returns an ATTLIST, of Name (as UNRES-NAME), Value (as string) pairs. The current character on the port is a non-whitespace character that is not an ncname-starting character.
Note the following rules to keep in mind when reading an ’AttValue’ "Before the value of an attribute is passed to the application or checked for validity, the XML processor must normalize it as follows:
#x9) is processed by appending
#x20to the normalized value, except that only a single
#x20is appended for a "
#xD#xA" sequence that is part of an external parsed entity or the literal entity value of an internal parsed entity
WFC: XML-Spec.html#CleanAttrVals WFC: XML-Spec.html#uniqattspec
Convert an unres-name to a res-name given the appropriate namespaces declarations. The last parameter apply-default-ns? determines if the default namespace applies (for instance, it does not for attribute names)
REC-xml-names/#nsc-NSDeclared, "xml" prefix is considered pre-declared
and bound to the namespace name "http://www.w3.org/XML/1998/namespace".
This procedure tests for the namespace constraints: http://www.w3.org/TR/REC-xml-names/#nsc-NSDeclared.
Convert a uri-str to an appropriate symbol.
This procedure is to complete parsing of a start-tag markup. The
procedure must be called after the start tag token has been
read. Tag is an UNRES-NAME.
s is an instance of
it can be
#f to tell the function to do no validation of elements
and their attributes.
This procedure returns several values:
element’s attributes, an ATTLIST of
(res-name . string)
pairs. The list does not include
the input list of namespaces amended with namespace (re-)declarations contained within the start-tag under parsing ELEM-CONTENT-MODEL.
On exit, the current position in port will be the first character after
#\> that terminates the start-tag markup.
VC: XML-Spec.html#enum VC: XML-Spec.html#RequiredAttr VC: XML-Spec.html#FixedAttr VC: XML-Spec.html#ValueType WFC: XML-Spec.html#uniqattspec (after namespaces prefixes are resolved) VC: XML-Spec.html#elementvalid WFC: REC-xml-names/#dt-NSName
Note, although XML Recommendation does not explicitly say it, xmlns and xmlns: attributes don’t have to be declared (although they can be declared, to specify their default value).
This procedure parses an
 ExternalID ::= 'SYSTEM' S SystemLiteral | 'PUBLIC' S PubidLiteral S SystemLiteral  SystemLiteral ::= ('"' [^"]* '"') | ("'" [^']* "'")  PubidLiteral ::= '"' PubidChar* '"' | "'" (PubidChar - "'")* "'"  PubidChar ::= #x20 | #xD | #xA | [a-zA-Z0-9] | [-'()+,./:=?;!*#@$_%]
This procedure is supposed to be called when an
ExternalID is expected;
that is, the current character must be either
#\P that start
PUBLIC token. This procedure returns the
SystemLiteral as a string. A
PubidLiteral is disregarded if present.
They parse productions corresponding to the whole (document) entity or its higher-level pieces (prolog, root element, etc).
Misc production in the context:
 document ::= prolog element Misc*  prolog ::= XMLDecl? Misc* (doctypedec l Misc*)?  Misc ::= Comment | PI | S
The following function should be called in the prolog or epilog contexts.
In these contexts, whitespaces are completely ignored.
The return value from
ssax:scan-Misc is either a
START token, or EOF.
Comments are ignored and not reported.
This procedure is to read the character content of an XML document or an XML element.
 content ::= (element | CharData | Reference | CDSect | PI | Comment)*
To be more precise, the procedure reads
and character entities, and skips comments. The procedure stops
at a named reference, EOF, at the beginning of a
PI or a start/end tag.
a port to read
a boolean indicating if EOF is normal, i.e., the character data may be terminated by the EOF. EOF is normal while processing a parsed entity.
an argument passed to the first invocation of STR-HANDLER.
The procedure returns two results: seed and token.
The seed is the result of the last invocation of str-handler, or the original seed if str-handler was never called.
Token can be either an eof-object (this can happen only if
PI. It’s up to an application to read or skip through the rest of this
CDATA sections and character references are expanded inline and
never returned. Comments are silently disregarded.
As the XML Recommendation requires, all whitespace in character data
must be preserved. However, a
CR character (
#xD) must be disregarded
if it appears before a
LF character (
#xA), or replaced by a
otherwise. See Secs. 2.10 and 2.11 of the XML Recommendation. See also
the canonical XML Recommendation.
Make sure that token is of anticipated kind and has anticipated gi. Note gi argument may actually be a pair of two symbols, Namespace URI or the prefix, and of the localname. If the assertion fails, error-cont is evaluated by passing it three arguments: token kind gi. The result of error-cont is returned.
These parsers are a set of syntactic forms to instantiate a SSAX parser.
A user can instantiate the parser to do the full validation, or
no validation, or any particular validation. The user specifies
PI he wants to be notified about. The user tells what to do
with the parsed character and element data. The latter handlers
determine if the parsing follows a SAX or a DOM model.
Create a parser to parse and process one Processing Element (
An assoc list of pairs (PI-TAG . PI-HANDLER)
where PI-TAG is an NCName symbol, the
PI target, and
PI-HANDLER is a procedure port pi-tag seed
where port points to the first symbol after the
The handler should read the rest of the
PI up to and including
the combination ’
?>’ that terminates the
PI. The handler should
return a new seed.
One of the PI-TAGs may be a symbol
*DEFAULT*. The corresponding
handler will handle
PIs that no other handler will. If the
*DEFAULT* PI-TAG is not specified,
ssax:make-pi-parser will make
one, which skips the body of the
The output of the
ssax:make-pi-parser is a procedure
port pi-tag seed,
that will parse the current
PI accoding to user-specified handlers.
Create a parser to parse and process one element, including its character content or children elements. The parser is typically applied to the root element of a document.
procedure elem-gi attributes namespaces expected-content seed
where elem-gi is a RES-NAME of the element about to be processed. This procedure is to generate the seed to be passed to handlers that process the content of the element.
procedure elem-gi attributes namespaces parent-seed seed
This procedure is called when parsing of elem-gi is finished. The seed is the result from the last content parser (or from my-new-level-seed if the element has the empty content). Parent-seed is the same seed as was passed to my-new-level-seed. The procedure is to generate a seed that will be the result of the element parser.
The generated parser is a:
procedure start-tag-head port elems entities
namespaces preserve-ws? seed.
The procedure must be called after the start tag token has been read. Start-tag-head is an UNRES-NAME from the start-element tag. Elems is an instance of
VC: XML-Spec.html#elementvalid WFC: XML-Spec.html#GIMatch
Create an XML parser, an instance of the XML parsing framework. This will be a SAX, a DOM, or a specialized parser depending on the supplied user-handlers.
user-handler-tag is a symbol that identifies a procedural expression that follows the tag. Given below are tags and signatures of the corresponding procedures. Not all tags have to be specified. If some are omitted, reasonable defaults will apply.
handler-procedure: port docname systemid internal-subset? seed
If internal-subset? is
#t, the current position in the port
is right after we have read
#\[ that begins the internal DTD subset.
We must finish reading of this subset before we return
(or must call skip-internal-subset if we aren’t interested in reading it).
The port at exit must be at the first symbol after the whole
The handler-procedure must generate four values:
elems entities namespaces seed
xml-decl::elems for elems.
It may be
#f to switch off the validation.
namespaces will typically contain USER-PREFIXes for selected URI-SYMBs.
The default handler-procedure skips the internal subset,
if any, and returns
(values #f '() '() seed).
handler-procedure: elem-gi seed
where elem-gi is an UNRES-NAME of the root element. This procedure is called when an XML document under parsing contains no
The handler-procedure, as a DOCTYPE handler procedure above,
must generate four values:
elems entities namespaces seed
The default handler-procedure returns
(values #f '() '() seed).
handler-procedure: elem-gi seed
where elem-gi is an UNRES-NAME of the root element. This procedure is called when an XML document under parsing does contains the
The handler-procedure must generate a new
seed (and verify
that the name of the root element matches the doctype, if the handler
The default handler-procedure is the identity function.
The default value is
The generated parser is a
procedure PORT SEED
This procedure parses the document prolog and then exits to an element parser (created by ssax:make-elem-parser) to handle the rest.
 document ::= prolog element Misc*  prolog ::= XMLDecl? Misc* (doctypedec | Misc*)?  Misc ::= Comment | PI | S  doctypedecl ::= '<!DOCTYPE' S Name (S ExternalID)? S? ('[' (markupdecl | PEReference | S)* ']' S?)? '>'  markupdecl ::= elementdecl | AttlistDecl | EntityDecl | NotationDecl | PI | Comment
A few utility procedures that turned out useful.
given the list of fragments (some of which are text strings) reverse the list and concatenate adjacent text strings.
given the list of fragments (some of which are text strings)
reverse the list and concatenate adjacent text strings.
We also drop "unsignificant" whitespace, that is, whitespace
in front, behind and between elements. The whitespace that
is included in character data is not affected.
We use this procedure to "intelligently" drop "insignificant"
whitespace in the parsed SXML. If the strict compliance with
the XML Recommendation regarding the whitespace is desired, please
ssax:reverse-collect-str procedure instead.
This is an instance of a SSAX parser above that returns an SXML
representation of the XML document to be read from port.
Namespace-prefix-assig is a list of
(USER-PREFIX . URI-STRING)
that assigns USER-PREFIXes to certain namespaces identified by
particular URI-STRINGs. It may be an empty list.
The procedure returns an SXML tree. The port points out to the
first character after the root element.