gauche.test- Unit Testing
Defines a set of functions to write test scripts. A test script will look like this:
(use gauche.test) (test-start "my feature") (load "my-feature") ; load your program (import my-feature) ; if your program defines a module. (test-module 'my-feature) ; tests consistency in your module. (test-section "feature group 1") (test "feature 1-1" EXPECT (lambda () TEST-BODY)) (test "feature 1-2" EXPECT (lambda () TEST-BODY)) … (test-section "feature group 2") (define test-data ...) (test "feature 2-1" EXPECT (lambda () TEST-BODY)) (test "feature 2-2" (test-error) (lambda () TEST-THAT-SIGNALS-ERROR)) … (test-end :exit-on-failure #t)
With this convention, you can run test both interactively or in batch. To run a test interactively, just load the file and it reports a result of each test, as well as the summary of failed test at the end. To run a test in batch, it is convenient to redirect the stdout to some file If stdout is redirected to other than tty, all the verbose logs will go there, and only a small amount of messages go to stderr.
It is recommended to have a "check" target always in Makefile of your module/program, so that the user of your program can run a test easily. The rule may look like this:
check : gosh my-feature-test.scm > test.log
Initializes internal state and prints a log header. This should be called before any tests. Module-name is used only for logging purpose.
Marks beginning of the group of tests. This is just for logging.
This is also just for logging. Creates a formatted string
with fmrstr and args just like
then write it to the current output port, with prefix
and newline at the end.
With the typical Makefile settings, where you redirect stdout of test scripts to a log file, the message only goes to the log file.
Using this, you can dump information that can’t be automatically
tested but may be useful for troubleshooting. For example,
you get a mysterious test failure reports you can’t reproduce
on your machine, and suspect
some aspects of the running systems may unpredictably
affect the test result.
You can put
test-log in the test code to dump such parameters,
and ask the reporter to run the test again and analyze the log.
Prints out list of failed tests. If exit-on-failure is
omitted, this procedure returns the number of failed tests.
Otherwise, this function terminates the
exit. If a fixnum is given to
it becomes the process’s exit status; if other true value is given,
the exit status will be 1.
Suppose you have several test scripts. Normally you run them as a group and what you want to know is a concise summary of the whole results, instead of each result of individual test files.
A test record file is an auxiliary file used to gather summary of the result. It holds a one-line summary of tests like this:
Total: 9939 tests, 9939 passed, 0 failed, 0 aborted.
When a test record file exists,
test-start reads and parses it, and remembers the numbers.
test-end adds the count of the results and
writes them back to the same test record file.
If you writes the
check target in your makefile as follows,
you will get the final one-line summary every time you run
make check, assuming that test1.scm, test2.scm,
and test3.scm all has
before a call to
check: @rm -f test.record test.log gosh test1.scm >> test.log gosh test2.scm >> test.log gosh test3.scm >> test.log @cat test.record
Note that to make
test-record-file work, it must be placed
before the call to
Alternatively, you can use the environment variable
GAUCHE_TEST_RECORD_FILE to specify the test record file.
If this environment variable is set when the test script is run, its value is used as the name of the test record file.
If the test script calls
test-record-file, it takes precedence
and this environment variable is ignored.
If the test record file is set (either by
the enviornment variable
GAUCHE_TEST_RECORD_FILE), read it,
and then exit with status 1 if the record has nonzero failure
count and/or nonzero abort count. If the test record file isn’t
set, this procedure does nothing.
This is useful when you have multiple test scripts and you want
make fail if any of tests fails, but not before all
test script is run. If you make every test script use
stops immediately after the script that fails. Instead, you
:exit-on-failure, but use the test record file
and for the last thing you can call this function:
check: rm -f $GAUCHE_TEST_RECORD_FILE test.log gosh test1.scm >> test.log gosh test2.scm >> test.log cat $GAUCHE_TEST_RECORD_FILE /dev/null gosh -ugauche.test -Etest-summary-check -Eexit
make will run all the test script no matter how
many of them fails (since
gosh exits with status 0), but
detect an error since the last line of
gosh call exits
with status 1 if there has been any failure.
A convenience macro that wraps expr by lambda.
(test* name expected expr) ≡ (test name expected (lambda () expr))
Calls thunk, and checks its result fits expected using a procedure check, which is called as follows:
(check expected result-of-thunk)
It should return
#t if the given result agrees with the
expected value, or
The default check procedure is
below. It compares expected and result-of-thunk
equal?, except when expected is some
of special case test objects. (See “testing ambiguous results”
and “testing abnormal cases”
paragraphs below for this special treatment.)
One typical usage of the custom check procedure is to compare inexact numbers tolerating small error.
(test "test 1" (/ 3.141592653589 4) (lambda () (atan 1)) (lambda (expected result) (< (abs (- expected result)) 1.0e-10)))
Name is a name of the test, for the logging purpose.
When thunk signals an uncaptured error, it is caught and
yields a special error object
You can check it with another error object created by
test-error function to see if it is an expected type
of error. See the entry of
test-error below for the details.
The default procedure
to check the result of the test expression conforms the
expected value. By default,
test-check just compares
expected and result with a procedure fallback,
which is defaulted to
differently if expected is one of special test objects
Sometimes the result of test expression depends on various external environment, and you cannot put an exact expected value. This procedure supports to write such tests conveniently.
Returns a special object representing either one of the choices.
The default check procedure,
test-check, recognizes the object
when it is passed in the expected argument, and returns true
if any one of choice … passes the check against the result.
For example, the following test passes if
proc returns either
1 or 2.
(test* "proc returns either 1 or 2" (test-one-of 1 2) (proc))
test-one-of, but creates a special object
representing none of the choices.
The test succees if the test expression evaluates to a value that
don’t match any of choices.
Returns a new
<test-error> object that mathes with
<test-error> object with the given condition-type.
test-check procedure treats
(test-check err-expected err-actual) returns
condition type is the same as or supertype of
For example, if you want to test a call to
<io-error> (or its condition subtype), you can write
as the following example:
(test "see if foo raises <io-error>" (test-error <io-error>) (foo))
(Deprecated) Bounded to an instance of
<error>. This is only provided for the
backward compatibility; new code should use
If this variable is true, the
test routine prints stack trace
to the current error port when it captures an error. It is useful
when you got an unexpected test-error object and want to check out
where the error is occurring.
This variable is initialized by the environment variable
GAUCHE_TEST_REPORT_ERROR when the
is loaded. For example, you can use the environment variable to check out
an unexpected error from your test script as follows (the value
of the environment variable doesn’t matter).
env GAUCHE_TEST_REPORT_ERROR=1 gosh mytest.scm
Scheme is dynamically typed, which is convenient for incremental and experimental development on REPL, but it tends to delay error detection until the code is actually run. It is very annoying that you run your program for a while only to see it barfs on simple typo of variable name.
Gauche addresses this issue by checking certain types of errors at the test phase. It isn’t purely a static check (we need to load a module or a script, which evaluates toplevel expressions), nor exhaustive (we can’t catch inconsistencies that span over multiple modules or about information that can be added at runtime). Nevertheless it can often catch some common mistakes, such as incorrect variable names or calling procedures with wrong number of arguments.
The two procedures,
load the named module and the script files respectively (which compiles
the Scheme code to VM instructions), then scan the compiled VM code
to perform the following tests:
The check is somewhat heuristic and we may miss some errors and/or can have false positives. For false positives, you can enumerate symbols to be excluded from the test.
Loads the module and runs the quasi-static consistency check. Module must be a symbol module name or a module.
Sometimes you have a global variable that may not be
defined depending on compiler options or platforms, and
you check its existence at runtime before using it.
The undefined variable reference check by
doesn’t follow such logic, and reports an error whenever
it finds your code referring to undefined variable. In such case,
you can give a list of symbols to the allow-undefined
keyword argument; the test will excludes them from the check.
The arity check may also raise false positives, if the module
count on a behavior of global procedures that will be modified
after the module is loaded (e.g. a method with different number
of arguments can be added to a generic function after the module
is loaded, which would make the code valid.) If you know
you’re doing right thing and need to suppress the false positives,
pass a list of names of the functions to
Loads the script named by filename into a fresh anonymous module and runs the quasi-static consistency check. Filename must be a string that names the script file.
The meaning of keyword arguments is the same as
Note that the toplevel forms in filename are evaluated, so scripts that
relies on the actions of toplevel forms could cause unwanted side-effects.
This check works best for the scripts written in
that is, calling actions from
main procedure instead of toplevel
forms. R7RS scripts relies on actions in toplevel forms and can’t be
tested with this procedure.
Scripts that relies on being loaded into
user module also
won’t work well with this check, which loads the forms into anonymous