This is bigloo.info, produced by makeinfo version 6.8 from bigloo.texi.

INFO-DIR-SECTION Programming Language
START-INFO-DIR-ENTRY
* bigloo 4.4c: (bigloo).  The Bigloo Scheme compiler
END-INFO-DIR-ENTRY

4.4c

  This file documents Bigloo, an implementation of an extended version
of the Scheme programming language.

  Copyright (C) 1992-99, 2000-17 Manuel Serrano

     This program is free software; you can redistribute it
     and/or modify it under the terms of the GNU General Public
     License as published by the Free Software Foundation; either
     version 2 of the License, or (at your option) any later version.

     This program is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
     GNU General Public License for more details.

     You should have received a copy of the GNU General Public
     License along with this program; if not, write to the Free
     Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
     MA 02111-1307, USA.


File: bigloo.info,  Node: Regular Parsing,  Next: Lalr Parsing,  Prev: Object System,  Up: Top

10 Regular parsing
******************

Programming languages have poor reading libraries since the lexical
information that can be specified is directly tied to the structure of
the language.  For example, in C it's hard to read a rational number
because there is no type rational.  Programs have been written to
circumvent this problem: Lex [Lesk75], for example, is one of them.  We
choose to incorporate in Bigloo a set of new functions to assist in such
parsing.  The syntax for regular grammar (also known as regular
analyser) of Bigloo 2.0 (the one described in this document) is not
compatible with former Bigloo versions.

* Menu:

* A New Way of Reading::
* The Syntax of The Regular Grammar::
* The Semantics Actions::
* Options and user definitions::
* Examples of Regular Grammar::


File: bigloo.info,  Node: A New Way of Reading,  Next: The Syntax of The Regular Grammar,  Up: Regular Parsing

10.1 A new way of reading
=========================

There is only one way in Bigloo to read text, _regular reading_, which
is done by the new form:

 -- bigloo procedure: read/rp regular-grammar input-port

     The first argument is a regular grammar (also known as regular
     analyser) and the second a Scheme port.  This way of reading is
     almost the same as the Lex's one.  The reader tries to match the
     longest input, from the stream pointed to by INPUT-PORT, with one
     of several regular expressions contained in REGULAR-GRAMMAR.  If
     many rules match, the reader takes the first one defined in the
     grammar.  When the regular rule has been found the corresponding
     Scheme expression is evaluated.

     _remark: _ The traditional 'read' Scheme function is implemented
     as:
          (define-inline (read port)
             (read/rp scheme-grammar port))


File: bigloo.info,  Node: The Syntax of The Regular Grammar,  Next: The Semantics Actions,  Prev: A New Way of Reading,  Up: Regular Parsing

10.2 The syntax of the regular grammar
======================================

A regular grammar is built by the means of the form 'regular-grammar':

 -- bigloo syntax: regular-grammar (binding ...) rule ...

     The BINDING and RULE are defined by the following grammar:

          <binding>  ==> (<variable> <re>)
                       | <option>
          <option>   ==> <variable>
          <rule>     ==> <define>
                       | (<cre> <s-expression> <s-expression> ...)
                       | (else <s-expression> <s-expression> ...)
          <define>   ==> (define <s-expression>)
          <cre>      ==> <re>
                       | (context <symbol> <re>)
                       | (when <s-expr> <re>)
                       | (bol <re>)
                       | (eol <re>)
                       | (bof <re>)
                       | (eof <re>)
          <re>       ==> <variable>
                       | <char>
                       | <string>
                       | (: <re> ...)
                       | (or <re> ...)
                       | (* <re>)
                       | (+ <re>)
                       | (? <re>)
                       | (= <integer> <re>)
                       | (>= <integer> <re>)
                       | (** <integer> <integer> <re>)
                       | (... <integer> <re>)
                       | (uncase <re>)
                       | (in <cset> ...)
                       | (out <cset> ...)
                       | (and <cset> <cset>)
                       | (but <cset> <cset>)
                       | (posix <string>)
          <variable> ==> <symbol>
          <cset>     ==> <string>
                       | <char>
                       | (<string>)
                       | (<char> <char>)

     Here is a description of each construction.

     '(context <symbol> <re>)'
          This allows us to _protect_ an expression.  A _protected_
          expression matches (or accepts) a word only if the grammar has
          been set to the corresponding context.  *Note The Semantics
          Actions::, for more details.

     '(when <s-expr> <re>)'
          This allows us to _protect_ an expression.  A _protected_
          expression matches (or accepts) a word only if the evaluation
          of '<s-expr>' is '#t'.  For instance,

               (define *g*
                  (let ((armed #f))
                     (regular-grammar ()
               	 ((when (not armed) (: "#!" (+ (or #\/ alpha))))
               	  (set! armed #t)
               	  (print "start [" (the-string) "]")
               	  (ignore))
               	 ((+ (in #\Space #\Tab))
               	  (ignore))
               	 (else
               	  (the-failure)))))

               (define (main argv)
                  (let ((port (open-input-string "#!/bin/sh #!/bin/zsh")))
                     (print (read/rp *g* port))))

     '(bol <re>)'
          Matches '<re>' at the beginning of line.

     '(eol <re>)'
          Matches '<re>' at the end of line.

     '(bof <re>)'
          Matches '<re>' at the beginning of file.

     '(eof <re>)'
          Matches '<re>' at the end of file.

     '<variable>'
          This is the name of a variable bound by a <binding>
          construction.  In addition to user defined variables, some
          already exist.  These are:

               all    == (out #\Newline)
               lower  == (in ("az"))
               upper  == (in ("AZ"))
               alpha  == (or lower upper)
               digit  == (in ("09"))
               xdigit == (uncase (in ("af09")))
               alnum  == (uncase (in ("az09")))
               punct  == (in ".,;!?")
               blank  == (in #" \t\n")
               space  == #\Space

          It is a error to reference a variable that it is not bound by
          a <binding>.  Defining a variable that already exists is
          acceptable and causes the former variable definition to be
          erased.  Here is an example of a grammar that binds two
          variables, one called 'ident' and one called 'number'.  These
          two variables are used within the grammar to match identifiers
          and numbers.

               (regular-grammar ((ident  (: alpha (* alnum)))
                                 (number (+ digit)))
                  (ident  (cons 'ident (the-string)))
                  (number (cons 'number (the-string)))
                  (else   (cons 'else (the-failure))))

     '<char>'
          The regular language described by one unique character.  Here
          is an example of a grammar that accepts either the character
          '#\a' or the character '#\b':

               (regular-grammar ()
                  (#\a (cons 'a (the-string)))
                  (#\b (cons 'b (the-string)))
                  (else (cons 'else (the-failure))))

     '<string>'
          This simple form of regular expression denotes the language
          represented by the string.  For instance the regular
          expression '"Bigloo"' matches only the string composed of '#\B
          #\i #\g #\l #\o #\o'.  The regular expression '".*["' matches
          the string '#\. #\* #\['.

     '(: <re> ...)'
          This form constructs sequence of regular expression.  That is
          a form '<re1> <re2> ... <ren>' matches the language
          construction by concatenation of the language described by
          '<re1>', '<re2>', '<ren>'.  Thus, '(: "x" all "y")' matches
          all words of three letters, started by character the '#\x' and
          ended with the character '#\y'.

     '(or <re> ...)'
          This construction denotes conditions.  The language described
          by '(or re1 re2)' accepts words accepted by either 're1' or
          're2'.

     '(* <re>)'
          This is the Kleene operator, the language described by '(*
          <re>)' is the language containing, 0 or more occurrences of
          '<re>'.  Thus, the language described by '(* "abc")' accepts
          the empty word and any word composed by a repetition of the
          'abc' ('abc', 'abcabc', 'abcabcabc', ...).

     '(+ <re>)'
          This expression described non empty repetitions.  The form '(+
          re)' is equivalent to '(: re (* re))'.  Thus, '(+ "abc")'
          matches the words 'abc', 'abcabc', etc.

     '(? <re>)'
          This expression described one or zero occurrence.  Thus, '(?
          "abc")' matches the empty word or the words 'abc'.

     '(= <integer> <re>)'
          This expression described a fix number of repetitions.  The
          form '(= num re)' is equivalent to '(: re re ... re)'.  Thus,
          the expression '(= 3 "abc")' matches the only word
          'abcabcabc'.  In order to avoid code size explosion when
          compiling, '<integer>' must be smaller than an arbitrary
          constant.  In the current version that value is '81'.

     '(>= <integer> <re>)'
          The language described by the expression '(>= int re)' accepts
          word that are, at least, 'int' repetitions of 're'.  For
          instance, '(>= 10 #\a)', accepts words compound of, at least,
          10 times the character '#\a'.  In order to avoid code size
          explosion when compiling, '<integer>' must be smaller than an
          arbitrary constant.  In the current version that value is
          '81'.

     '(** <integer> <integer> <re>)'
          The language described by the expression '(** min max re)'
          accepts word that are repetitions of 're'; the number of
          repetition is in the range 'min', 'max'.  For instance, '(**
          10 20 #\a)'.  In order to avoid code size explosion when
          compiling, '<integer>' must be smaller than an arbitrary
          constant.  In the current version that value is '81'.

     '(... <integer> <re>)'
          The subexpression '<re>' has to be a sequence of characters.
          Sequences are build by the operator ':' or by string literals.
          The language described by '(... int re)', denotes, the first
          letter of 're', or the two first letters of 're', or the three
          first letters of 're' or the 'int' first letters of 're'.
          Thus, '(... 3 "begin")' is equivalent to '(or "b" "be"
          "beg")'.

     '(uncase <re>)'
          The subexpression '<re>' has to be a sequence construction.
          The language described by '(uncase re)' is the same as 're'
          where letters may be upper case or lower case.  For instance,
          '(uncase "begin")', accepts the words '"begin"', '"beGin"',
          '"BEGIN"', '"BegiN"', etc.

     '(in <cset> ...)'
          Denotes union of characters.  Characters may be described
          individually such as in '(in #\a #\b #\c #\d)'.  They may be
          described by strings.  The expression '(in "abcd")' is
          equivalent to '(in #\a #\b #\c #\d)'.  Characters may also be
          described using a range notation that is a list of two
          characters.  The expression '(in (#\a #\d))' is equivalent to
          '(in #\a #\b #\c #\d)'.  The Ranges may be expresses using
          lists of string.  The expression '(in ("ad"))' is equivalent
          to '(in #\a #\b #\c #\d)'.

     '(out <cset> ...)'
          The language described by '(out cset ...)' is opposite to the
          one described by '(in cset ...)'.  For instance, '(out
          ("azAZ") (#\0 #\9))' accepts all words of one character that
          are neither letters nor digits.  One should not that if the
          character numbered zero may be used inside regular grammar,
          the 'out' construction never matches it.  Thus to write a rule
          that, for instances, matches every character but '#\Newline'
          including the character zero, one should write:

               (or (out #\Newline) #a000)

     '(and <cset> <cset>)'
          The language described by '(and cset1 cset2)' accepts words
          made of characters that are in both 'cset1' and 'cset2'.

     '(but <cset> <cset>)'
          The language described by '(but cset1 cset2)' accepts words
          made of characters of 'cset1' that are not member of 'cset2'.

     '(posix <string>)'
          The expression '(posix string)' allows one to use Posix string
          notation for regular expressions.  So, for example, the
          following two expressions are equivalent:

               (posix "[az]+|x*|y{3,5}")

               (or (+ (in ("az"))) (* "x") (** 3 5 "y"))

 -- bigloo syntax: string-case string rule ...
     This form dispatches on strings.  it opens an input on 'string' a
     read into it according to the regular grammar defined by the
     'binding' and 'rule'.  Example:

          (define (suffix string)
             (string-case string
                ((: (* all) ".")
                 (ignore))
                ((+ (out #\.))
                 (the-string))
                (else
                 "")))


File: bigloo.info,  Node: The Semantics Actions,  Next: Options and user definitions,  Prev: The Syntax of The Regular Grammar,  Up: Regular Parsing

10.3 The semantics actions
==========================

The semantics actions are regular Scheme expressions.  These expressions
appear in an environment where some "extra procedures" are defined.
These procedures are:

 -- bigloo rgc procedure: the-port
     Returns the input port currently in used.

 -- bigloo rgc procedure: the-length
     Get the length of the biggest matching string.

 -- bigloo rgc procedure: the-string
     Get a copy of the last matching string.  The function 'the-string'
     returns a fresh copy of the matching each time it is called.  In
     consequence,

          (let ((l1 (the-string)) (l2 (the-string)))
             (eq? l1 l2))
             => #f

 -- bigloo rgc procedure: the-substring start len
     Get a copy of a substring of the last matching string.  If the LEN
     is negative, it is subtracted to the whole match length.  Here is
     an example of a rule extracting a part of a match:

          (regular-grammar ()
             ((: #\" (* (out #\")) #\")
              (the-substring 1 (-fx (the-length) 1))))

     Which can also be written:

          (regular-grammar ()
             ((: #\" (* (out #\")) #\")
              (the-substring 1 -1)))

 -- bigloo rgc procedure: the-character
 -- bigloo rgc procedure: the-byte
     Returns the first character of a match (respectively, the first
     byte).

 -- bigloo rgc procedure: the-byte-ref n
     Returns the N-th bytes of the matching string.

 -- bigloo rgc procedure: the-symbol
 -- bigloo rgc procedure: the-downcase-symbol
 -- bigloo rgc procedure: the-upcase-symbol
 -- bigloo rgc procedure: the-subsymbol start length
     Convert the last matching string into a symbol.  The function
     'the-subsymbol' obeys the same rules as 'the-substring'.

 -- bigloo rgc procedure: the-keyword
 -- bigloo rgc procedure: the-downcase-keyword
 -- bigloo rgc procedure: the-upcase-keyword
     Convert the last matching string into a keyword.

 -- bigloo rgc procedure: the-fixnum
     The conversion of the last matching string to fixnum.

 -- bigloo rgc procedure: the-flonum
     The conversion of the last matching string to flonum.

 -- bigloo rgc procedure: the-failure
     Returns the first char that the grammar can't match or the end of
     file object.

 -- bigloo rgc procedure: ignore
     Ignore the parsing, keep reading.  It's better to use '(ignore)'
     rather than an expression like '(read/rp GRAMMAR PORT)' in
     semantics actions since the '(ignore)' call will be done in a tail
     recursive way.  For instance,

          (let ((g (regular-grammar ()
                      (")"
                       '())
                      ("("
                       (let* ((car (ignore))
                              (cdr (ignore)))
                          (cons car cdr)))
                      ((+ (out "()"))
                       (the-string))))
                (p (open-input-string "(foo(bar(gee)))")))
             (read/rp g p))
             => ("foo" ("bar" ("gee")))

 -- bigloo rgc procedure: rgc-context [context]
     If no CONTEXT is provide, this procedure reset the reader context
     state.  That is the reader is in no context.  With one argument,
     'context' set the reader in the context CONTEXT.  For instance,

          (let ((g (regular-grammar ()
                      ((context foo "foo") (print 'foo-bis))
                      ("foo" (rgc-context 'foo) (print 'foo) (ignore))
                      (else 'done)))
                (p (open-input-string "foofoo")))
             (read/rp g p))
             -| foo
                foo-bis

     Note that RGC context are preserved across different uses of
     'read/rp'.

 -- bigloo rgc procedure: the-context
     Returns the value of the current Rgc context.


File: bigloo.info,  Node: Options and user definitions,  Next: Examples of Regular Grammar,  Prev: The Semantics Actions,  Up: Regular Parsing

10.4 Options and user definitions
=================================

Options act as parameters that are transmitted to the parser on the call
to 'read/rp'.  Local defines are user functions inserted in the produced
parser, at the same level as the pre-defined 'ignore' function.

  Here is an example of grammar using both

     (define gram
        (regular-grammar (x y)

           (define (foo s)
     	 (cons* 'foo x s (ignore)))
           (define (bar s)
     	 (cons* 'bar y s (ignore)))

           ((+ #\a) (foo (the-string)))
           ((+ #\b) (bar (the-string)))
           (else '())))

  This grammar uses two options X and Y.  Hence when invokes it takes
two additional values such as:

     (with-input-from-string "aabb"
        (lambda ()
           (read/rp gram (current-input-port) 'option-x 'option-y)))
        => (foo option-x aa bar option-y bb)


File: bigloo.info,  Node: Examples of Regular Grammar,  Prev: Options and user definitions,  Up: Regular Parsing

10.5 Examples of regular grammar
================================

The reader who wants to find a real example should read the code of
Bigloo's reader.  But here are small examples

10.5.1 Word count
-----------------

The first example presents a grammar that simulates the Unix program
'wc'.
     (let ((*char* 0)
           (*word* 0)
           (*line* 0))
        (regular-grammar ()
           ((+ #\Newline)
            (set! *char* (+ *char* (the-length)))
            (set! *line* (+ *line* (the-length)))
            (ignore))
           ((+ (in #\space #\tab))
            (set! *char* (+ *char* (the-length)))
            (ignore))
           ((+ (out #\newline #\space #\tab))
            (set! *char* (+ *char* (the-length)))
            (set! *word* (+ 1 *word*))
            (ignore))))

10.5.2 Roman numbers
--------------------

The second example presents a grammar that reads Arabic and Roman
number.
     (let ((par-open 0))
        (regular-grammar ((arabic (in ("09")))
                          (roman  (uncase (in "ivxlcdm"))))
           ((+ (in #" \t\n"))
            (ignore))
           ((+ arabic)
            (string->integer (the-string)))
           ((+ roman)
            (roman->arabic (the-string)))
           (#\(
            (let ((open-key par-open))
               (set! par-open (+ 1 par-open))
               (context 'pair)
               (let loop-pair ((walk (ignore)))
                  (cond
                     ((= open-key par-open)
                      '())
                     (else
                      (cons walk (loop-pair (ignore))))))))
           (#\)
            (set! par-open (- par-open 1))
            (if (< par-open 0)
                (begin
                   (set! par-open 0)
                   (ignore))
                #f))
           ((in "+-*\\")
            (string->symbol (the-string)))
           (else
            (let ((char (the-failure)))
               (if (eof-object? char)
                   char
                   (error "grammar-roman" "Illegal char" char))))))


File: bigloo.info,  Node: Lalr Parsing,  Next: Posix Regular Expressions,  Prev: Regular Parsing,  Up: Top

11 Lalr(1) parsing
******************

Regular grammar generators, like Lex, are often coupled with tools, such
as Yacc and Bison, that can generate parsers for more powerful
languages, namely (a subset of) context-free languages.  These tools
take as input a description of the language to be recognized and
generate a parser for that language, written in some other language (for
example, Yacc and Bison generate parsers written in C). The user must
always be aware of the generated parser and that is a nuisance.  Bigloo
provides such a tool that overcomes this annoyance.  It generates
parsers for the class of Lalr(1) grammars in a more opaque way.

* Menu:

* Grammar Definition::
* Precedence and Associativity::
* The Parsing Function::
* The Regular Grammar::
* Debugging Lalr Grammars::
* A Simple Example::


File: bigloo.info,  Node: Grammar Definition,  Next: Precedence and Associativity,  Up: Lalr Parsing

11.1 Grammar definition
=======================

An lalr(1) grammar is defined by the form:

 -- bigloo syntax: lalr-grammar term-def non-term-def...

     TERM-DEF is a list of terminal elements of the grammar.  Terminals
     can grouped together to form precedence groups by including the
     related symbols in a sub-lists of the TERM-DEF list.  Each
     precedence group must start with one of the keywords 'left:',
     'right:' or 'none:'- this indicates the associativity of the
     terminal symbol.  Here is a sample TERM-DEF which declares eight
     terminals:
          (terminal-1 terminal-2
           (left: terminal-3 terminal-4)
           terminal-5
           (right: terminal-6)
           (none: terminal-7)
           terminal-8)

     In this case, 'terminal-3' and 'terminal-4' both have the same
     precedence, which is greater than the precedence assigned to
     'terminal-6'.  No precedence was assigned to symbols 'terminal-1',
     'terminal-2', 'terminal-5' or 'terminal-8'.

     Each NON-TERM-DEF is a list whose first element is the non-terminal
     being defined, i.e.  a symbol.  The remaining elements are the
     production rules associated with this non-terminal.  Each rule is a
     list whose first element is the rule itself (a list of symbols) and
     the other elements are the semantic actions associated with that
     particular rule.

     For example, consider the following grammar:
          E ==> E1 + ID {E.val := E1.val + ID.val}
                | ID {E.val := ID.val}

     With Bigloo, it would be written:
          (lalr-grammar
            (plus id)
            (e
             ((e plus id)   (+ e id))
             ((id)          id)))

     The semantic value of a symbol in a rule can be accessed by simply
     using the name of the symbol in the semantic action associated with
     the rule.  Because a rule can contain multiple occurrences of the
     same symbol, Bigloo provides a way to access these occurrences
     separately.  To do so, the name of each occurrence must be suffixed
     by '@'VAR where VAR is the name of a variable that will be bound to
     the semantic value of the occurrence.  For example, if the rule is

             ifstmt ==> if E then Stmt else Stmt

     then, in Bigloo, it would look like
          (if-stmt
           ((if e then stmt@conseq else stmt@altern)
            (if (eval e)
                (eval conseq)
                (eval altern))))


File: bigloo.info,  Node: Precedence and Associativity,  Next: The Parsing Function,  Prev: Grammar Definition,  Up: Lalr Parsing

11.2 Precedence and associativity
=================================

The bigloo lalr(1) parser generator supports operator precedence and
associativity.  The method for specifying the precedence for terminal
symbols is described in *note Grammar Definition::.  Precedence is
assigned to each non-terminal production from the precedence of the last
terminal symbol appearing in that production.

  Typically, when the parser generator encounters a shift/reduce
conflict, it produces a warning message, then chooses to reduce.  When a
parser generator has precedence and associativity information, it can
make a much more sophisticated decision.

  Let's use this simple calculator grammar as an example:
     (lalr-grammar
      ((left: op-mult op-div)
       (left: op-add op-sub)
       op-lparen op-rparen
       op-semicolon
       number)

      (file
        (())
        ((file stmt)))
      (stmt
        ((expr op-semicolon) (print expr)))
      (expr
        ((number) number)
        ((expr@a op-add expr@b) (+ a b))
        ((expr@a op-sub expr@b) (- a b))
        ((expr@a op-mult expr@b) (* a b))
        ((expr@a op-div expr@b) (/ a b))
        ((op-lparen expr op-rparen) expr))))

  Let's start with this input:
     1 + 2 * 3;

  At the point where the parser has read '1 + 2' and the lookahead
symbol is '*', the parser encounters a shift/reduce conflict.  Should it
first reduce by the '(expr op-add expr)' production or shift the '*' in
the hopes of reducing the latter expression first?

  The '(expr op-add expr)' production has gotten its precedence from the
'op-add' terminal symbol.  This is the precedence of the reduce.  The
precedence of the shift comes from the precedence assigned to the
lookahead terminal symbol, which is 'op-mult'.  Since 'op-mult' has
higher precedence, the parser generator in this state chooses to shift
and does not produce a warning.

  Here's an example which we can use to demonstrate associativity:
     1 + 2 - 3;

  The parser generator encounters a similar shift/reduce conflict this
time, except that when it tries to determine whether to shift or reduce,
it finds that both actions have the same precedence.  In this case, the
parser generator looks at the associativity of the precedence group
containing the 'op-add' and 'op-sub'.  Since these are declared to be
left-associative, the parser generator chooses to reduce from this
state, effectively calculating the '1 + 2'.  Had these symbols been
right-associative, the parser would have chosen to shift, effectively
calculating '2 - 3' first.  If these symbols had been declared
non-associative with the 'none:' keyword, the parser would generate an
error if it ever encountered this state.


File: bigloo.info,  Node: The Parsing Function,  Next: The Regular Grammar,  Prev: Precedence and Associativity,  Up: Lalr Parsing

11.3 The parsing function
=========================

Once a grammar has been defined, it can be used to parse some input
using the following function:

 -- bigloo procedure: read/lalrp lg rg port [emptyp]
     This function takes three, possibly four, arguments.  The first,
     LG, is the Lalr(1) grammar.  The second, RG, is the lexical
     analyzer that feeds the grammar with tokens.  The third argument,
     PORT, is the port that contains the input to be parsed.  The last
     argument, EMPTYP, if provided, should be a function of one
     argument.  It is called with each new token read from the port and
     should return '#t' if the token denotes the end of input.  The
     result of the call is the value computed by the semantic actions of
     the production rules.


File: bigloo.info,  Node: The Regular Grammar,  Next: Debugging Lalr Grammars,  Prev: The Parsing Function,  Up: Lalr Parsing

11.4 The regular grammar
========================

In order to work properly, the regular grammar used with an Lalr(1)
grammar should follow some conventions:

   * If a semantic value is to be associated with the token just parsed,
     the regular grammar should return a pair whose 'car' is the token
     name (a symbol) and the 'cdr' is the semantic value.
   * If there is no value associated with the token, the regular grammar
     can return just the token name.  When used in conjunction with an
     Lalr grammar, regular grammar should never return '#f' as a token
     value.  This is specially true when the regular grammar detects the
     end of parsing.  In that case, the regular grammar _must not_
     return the '#f' value.  A good way to handle end-of-file is
     illustrated in the following example:

          (let ((g (regular-grammar ()
                       ...
                       (else
                        (let ((c (the-failure)))
                           (if (eof-object? c)
                               c
                               (error 'rgc "Illegal character" c))))))
                (l (lalr-grammar ...)))
             (read/lalrp l g (current-input-port)))

     This way, the Lalr grammar will automatically handles the
     end-of-file.


File: bigloo.info,  Node: Debugging Lalr Grammars,  Next: A Simple Example,  Prev: The Regular Grammar,  Up: Lalr Parsing

11.5 Debugging Lalr Grammars
============================

Currently the debugging facility for debugging Lalr grammars is very
limited.  When the parameter 'bigloo-debug' is set to a value greater or
equal to 100, the Lalr engine outputs all of the state changes the
parser is going through.


File: bigloo.info,  Node: A Simple Example,  Prev: Debugging Lalr Grammars,  Up: Lalr Parsing

11.6 A simple example
=====================

Here is the code for a simple calculator implemented by an Lalr(1)
grammar:

     (begin
       (read/lalrp
        (lalr-grammar
         (nl plus mult minus div const lpar rpar)
         (lines
          (())
          ((lines expression nl)    (display "--> ")
                                    (display expression)
                                    (newline))
          ((lines nl)))
         (expression
          ((expression plus term)   (+ expression term))
          ((expression minus term)  (- expression term))
          ((term)                   term))
         (term
          ((term mult factor)       (* term factor))
          ((term div factor)        (/ term factor))
          ((factor)                 factor))
         (factor
          ((lpar expression rpar)   expression)
          ((const)                  const)))

        (regular-grammar ()
         ((+ (or #\tab #\space)) (ignore))
         (#\newline              'nl)
         ((+ digit)              (cons 'const (string->number (the-string))))
         (#\+                    'plus)
         (#\-                    'minus)
         (#\*                    'mult)
         (#\/                    'div)
         (#\(                    'lpar)
         (#\)                    'rpar))

        (current-input-port))
       (reset-eof (current-input-port)))


File: bigloo.info,  Node: Posix Regular Expressions,  Next: Command Line Parsing,  Prev: Lalr Parsing,  Up: Top

12 Posix Regular Expressions
****************************

This whole section has been written by Dorai Sitaram.  It consists in
the documentation of the 'pregexp' package that may be found at
<http://www.ccs.neu.edu/~dorai/pregexp/pregexp.html>.


  The regexp notation supported is modeled on Perl's, and includes such
powerful directives as numeric and nongreedy quantifiers, capturing and
non-capturing clustering, POSIX character classes, selective case- and
space-insensitivity, backreferences, alternation, backtrack pruning,
positive and negative lookahead and lookbehind, in addition to the more
basic directives familiar to all regexp users.  A _regexp_ is a string
that describes a pattern.  A regexp matcher tries to _match_ this
pattern against (a portion of) another string, which we will call the
_text string_.  The text string is treated as raw text and not as a
pattern.

  Most of the characters in a regexp pattern are meant to match
occurrences of themselves in the text string.  Thus, the pattern '"abc"'
matches a string that contains the characters 'a', 'b', 'c' in
succession.

  In the regexp pattern, some characters act as _metacharacters_, and
some character sequences act as _metasequences_.  That is, they specify
something other than their literal selves.  For example, in the pattern
'"a.c"', the characters 'a' and 'c' do stand for themselves but the
_metacharacter_ '.' can match _any_ character (other than newline).
Therefore, the pattern '"a.c"' matches an 'a', followed by _any_
character, followed by a 'c'.

  If we needed to match the character '.' itself, we _escape_ it, ie,
precede it with a backslash ('\').  The character sequence '\.' is thus
a _metasequence_, since it doesn't match itself but rather just '.'.
So, to match 'a' followed by a literal '.' followed by 'c', we use the
regexp pattern '"a\\.c"'.(1)  (*note Posix Regular
Expressions-Footnote-1::) Another example of a metasequence is '\t',
which is a readable way to represent the tab character.

  We will call the string representation of a regexp the _U-regexp_,
where _U_ can be taken to mean _Unix-style_ or _universal_, because this
notation for regexps is universally familiar.  Our implementation uses
an intermediate tree-like representation called the _S-regexp_, where
_S_ can stand for _Scheme_, _symbolic_, or _s-expression_.  S-regexps
are more verbose and less readable than U-regexps, but they are much
easier for Scheme's recursive procedures to navigate.

* Menu:

* Regular Expressions Procedures::
* The Regular Expressions Pattern Language::
* An Extended Example::


File: bigloo.info,  Node: Posix Regular Expressions-Footnotes,  Up: Posix Regular Expressions

   (1) The double backslash is an artifact of Scheme strings, not the
regexp pattern itself.  When we want a literal backslash inside a Scheme
string, we must escape it so that it shows up in the string at all.
Scheme strings use backslash as the escape character, so we end up with
two backslashes -- one Scheme-string backslash to escape the regexp
backslash, which then escapes the dot.  Another character that would
need escaping inside a Scheme string is '"'.


File: bigloo.info,  Node: Regular Expressions Procedures,  Next: The Regular Expressions Pattern Language,  Up: Posix Regular Expressions

12.1 Regular Expressions Procedures
===================================

Four procedures 'pregexp', 'pregexp-match-positions', 'pregexp-match',
'pregexp-replace', and 'pregexp-replace*' enable compilation and
matching of regular expressions.

 -- bigloo procedure: pregexp U-regexp . opt-args
     The procedure 'pregexp' takes a U-regexp, which is a string, and
     returns an S-regexp, which is a tree.

          (pregexp "c.r") => (:sub (:or (:seq #\c :any #\r)))

     There is rarely any need to look at the S-regexps returned by
     'pregexp'.

     The OPT-ARGS specifies how the regular expression is to be matched.
     Until documented the argument should be the empty list.

 -- bigloo procedure: pregexp-match-positions regexp string [beg 0] [end
          -1]

     The procedure 'pregexp-match-positions' takes a regexp pattern and
     a text string, and returns a _match_ if the pattern _matches_ the
     text string.  The pattern may be either a U- or an S-regexp.
     ('pregexp-match-positions' will internally compile a U-regexp to an
     S-regexp before proceeding with the matching.  If you find yourself
     calling 'pregexp-match-positions' repeatedly with the same
     U-regexp, it may be advisable to explicitly convert the latter into
     an S-regexp once beforehand, using 'pregexp', to save needless
     recompilation.)

     'pregexp-match-positions' returns '#f' if the pattern did not match
     the string; and a list of _index pairs_ if it did match.  Eg,

          (pregexp-match-positions "brain" "bird")
           => #f
          (pregexp-match-positions "needle" "hay needle stack")
           => ((4 . 10))

     In the second example, the integers 4 and 10 identify the substring
     that was matched.  1 is the starting (inclusive) index and 2 the
     ending (exclusive) index of the matching substring.

          (substring "hay needle stack" 4 10)
           => "needle"

     Here, 'pregexp-match-positions''s return list contains only one
     index pair, and that pair represents the entire substring matched
     by the regexp.  When we discuss _subpatterns_ later, we will see
     how a single match operation can yield a list of _submatches_.

     'pregexp-match-positions' takes optional third and fourth arguments
     that specify the indices of the text string within which the
     matching should take place.

          (pregexp-match-positions "needle"
            "his hay needle stack -- my hay needle stack -- her hay needle stack"
            24 43)
           => ((31 . 37))

     Note that the returned indices are still reckoned relative to the
     full text string.

 -- bigloo procedure: pregexp-match regexp string
     The procedure 'pregexp-match' is called like
     'pregexp-match-positions' but instead of returning index pairs it
     returns the matching substrings:

          (pregexp-match "brain" "bird")
           => #f
          (pregexp-match "needle" "hay needle stack")
           => ("needle")

     'pregexp-match' also takes optional third and fourth arguments,
     with the same meaning as does 'pregexp-match-positions'.

 -- bigloo procedure: pregexp-replace regexp string1 string2
     The procedure 'pregexp-replace' replaces the matched portion of the
     text string by another string.  The first argument is the regexp,
     the second the text string, and the third is the _insert string_
     (string to be inserted).

          (pregexp-replace "te" "liberte" "ty")
           => "liberty"

     If the pattern doesn't occur in the text string, the returned
     string is identical ('eq?') to the text string.

 -- bigloo procedure: pregexp-replace* regexp string1 string2
     The procedure 'pregexp-replace*' replaces _all_ matches in the text
     STRING1 by the insert STRING2:

          (pregexp-replace* "te" "liberte egalite fraternite" "ty")
           => "liberty egality fratyrnity"

     As with 'pregexp-replace', if the pattern doesn't occur in the text
     string, the returned string is identical ('eq?') to the text
     string.

 -- bigloo procedure: pregexp-split regexp string
     The procedure 'pregexp-split' takes two arguments, a regexp pattern
     and a text string, and returns a list of substrings of the text
     string, where the pattern identifies the delimiter separating the
     substrings.

          (pregexp-split ":" "/bin:/usr/bin:/usr/bin/X11:/usr/local/bin")
           => ("/bin" "/usr/bin" "/usr/bin/X11" "/usr/local/bin")

          (pregexp-split " " "pea soup")
           => ("pea" "soup")

     If the first argument can match an empty string, then the list of
     all the single-character substrings is returned.

          (pregexp-split "" "smithereens")
           => ("s" "m" "i" "t" "h" "e" "r" "e" "e" "n" "s")

     To identify one-or-more spaces as the delimiter, take care to use
     the regexp '" +"', not '" *"'.

          (pregexp-split " +" "split pea     soup")
           => ("split" "pea" "soup")

          (pregexp-split " *" "split pea     soup")
           => ("s" "p" "l" "i" "t" "p" "e" "a" "s" "o" "u" "p")

 -- bigloo procedure: pregexp-quote string

     The procedure 'pregexp-quote' takes an arbitrary STRING and returns
     a U-regexp (string) that precisely represents it.  In particular,
     characters in the input string that could serve as regexp
     metacharacters are escaped with a backslash, so that they safely
     match only themselves.

          (pregexp-quote "cons")
           => "cons"

          (pregexp-quote "list?")
           => "list\\?"

     'pregexp-quote' is useful when building a composite regexp from a
     mix of regexp strings and verbatim strings.


File: bigloo.info,  Node: The Regular Expressions Pattern Language,  Next: An Extended Example,  Prev: Regular Expressions Procedures,  Up: Posix Regular Expressions

12.2 Regular Expressions Pattern Language
=========================================

* Menu:

* Basic assertions::
* Characters and character classes::
* Quantifiers::
* Clusters::
* Alternation::
* Backtracking::
* Looking ahead and behind::

Here is a complete description of the regexp pattern language recognized
by the 'pregexp' procedures.


File: bigloo.info,  Node: Basic assertions,  Next: Characters and character classes,  Up: The Regular Expressions Pattern Language

12.2.1 Basic assertions
-----------------------

The _assertions_ '^' and '$' identify the beginning and the end of the
text string respectively.  They ensure that their adjoining regexps
match at one or other end of the text string.  Examples:

     (pregexp-match-positions "^contact" "first contact") => #f

The regexp fails to match because 'contact' does not occur at the
beginning of the text string.

     (pregexp-match-positions "laugh$" "laugh laugh laugh laugh") => ((18 . 23))

The regexp matches the _last_ 'laugh'.

  The metasequence '\b' asserts that a _word boundary_ exists.

     (pregexp-match-positions "yack\\b" "yackety yack") => ((8 . 12))

The 'yack' in 'yackety' doesn't end at a word boundary so it isn't
matched.  The second 'yack' does and is.

  The metasequence '\B' has the opposite effect to '\b'.  It asserts
that a word boundary does not exist.

     (pregexp-match-positions "an\\B" "an analysis") => ((3 . 5))

The 'an' that doesn't end in a word boundary is matched.


File: bigloo.info,  Node: Characters and character classes,  Next: Quantifiers,  Prev: Basic assertions,  Up: The Regular Expressions Pattern Language

12.2.2 Characters and character classes
---------------------------------------

Typically a character in the regexp matches the same character in the
text string.  Sometimes it is necessary or convenient to use a regexp
metasequence to refer to a single character.  Thus, metasequences '\n',
'\r', '\t', and '\.' match the newline, return, tab and period
characters respectively.

  The _metacharacter_ period ('.') matches _any_ character other than
newline.

     (pregexp-match "p.t" "pet") => ("pet")

It also matches 'pat', 'pit', 'pot', 'put', and 'p8t' but not 'peat' or
'pfffft'.

  A _character class_ matches any one character from a set of
characters.  A typical format for this is the _bracketed character
class_ '['...']', which matches any one character from the non-empty
sequence of characters enclosed within the brackets.(1)  (*note
Characters and character classes-Footnote-1::) Thus '"p[aeiou]t"'
matches 'pat', 'pet', 'pit', 'pot', 'put' and nothing else.

  Inside the brackets, a hyphen ('-') between two characters specifies
the ascii range between the characters.  Eg, '"ta[b-dgn-p]"' matches
'tab', 'tac', 'tad', _and_ 'tag', _and_ 'tan', 'tao', 'tap'.

  An initial caret ('^') after the left bracket inverts the set
specified by the rest of the contents, ie, it specifies the set of
characters _other than_ those identified in the brackets.  Eg,
'"do[^g]"' matches all three-character sequences starting with 'do'
except 'dog'.

  Note that the metacharacter '^' inside brackets means something quite
different from what it means outside.  Most other metacharacters ('.',
'*', '+', '?', etc) cease to be metacharacters when inside brackets,
although you may still escape them for peace of mind.  '-' is a
metacharacter only when it's inside brackets, and neither the first nor
the last character.

  Bracketed character classes cannot contain other bracketed character
classes (although they contain certain other types of character classes
-- see below).  Thus a left bracket ('[') inside a bracketed character
class doesn't have to be a metacharacter; it can stand for itself.  Eg,
'"[a[b]"' matches 'a', '[', and 'b'.

  Furthermore, since empty bracketed character classes are disallowed, a
right bracket (']') immediately occurring after the opening left bracket
also doesn't need to be a metacharacter.  Eg, '"[]ab]"' matches ']',
'a', and 'b'.

12.2.3 Some frequently used character classes
---------------------------------------------

Some standard character classes can be conveniently represented as
metasequences instead of as explicit bracketed expressions.  '\d'
matches a digit ('[0-9]'); '\s' matches a whitespace character; and '\w'
matches a character that could be part of a "word".(2)  (*note
Characters and character classes-Footnote-2::)

  The upper-case versions of these metasequences stand for the
inversions of the corresponding character classes.  Thus '\D' matches a
non-digit, '\S' a non-whitespace character, and '\W' a non-"word"
character.

  Remember to include a double backslash when putting these
metasequences in a Scheme string:

     (pregexp-match "\\d\\d" "0 dear, 1 have 2 read catch 22 before 9") => ("22")

  These character classes can be used inside a bracketed expression.
Eg, '"[a-z\\d]"' matches a lower-case letter or a digit.

12.2.4 POSIX character classes
------------------------------

A _POSIX character class_ is a special metasequence of the form
'[:'...':]' that can be used only inside a bracketed expression.  The
POSIX classes supported are

     [:alnum:]  letters and digits
     [:alpha:]  letters
     [:algor:]  the letters c, h, a and d
     [:ascii:]  7-bit ascii characters
     [:blank:]  widthful whitespace, ie, space and tab
     [:cntrl:]  ``control'' characters, viz, those with code < 32
     [:digit:]  digits, same as \d
     [:graph:]  characters that use ink
     [:lower:]  lower-case letters
     [:print:]  ink-users plus widthful whitespace
     [:space:]  whitespace, same as \s
     [:upper:]  upper-case letters
     [:word:]   letters, digits, and underscore, same as \w
     [:xdigit:] hex digits

For example, the regexp '"[[:alpha:]_]"' matches a letter or underscore.

     (pregexp-match "[[:alpha:]_]" "--x--") => ("x")
     (pregexp-match "[[:alpha:]_]" "--_--") => ("_")
     (pregexp-match "[[:alpha:]_]" "--:--") => #f

  The POSIX class notation is valid _only_ inside a bracketed
expression.  For instance, '[:alpha:]', when not inside a bracketed
expression, will _not_ be read as the letter class.  Rather it is (from
previous principles) the character class containing the characters ':',
'a', 'l', 'p', 'h'.

     (pregexp-match "[[:alpha:]]" "--a--") => ("a")
     (pregexp-match "[[:alpha:]]" "--_--") => #f

  By placing a caret ('^') immediately after '[:', you get the inversion
of that POSIX character class.  Thus, '[:^alpha]' is the class
containing all characters except the letters.


File: bigloo.info,  Node: Characters and character classes-Footnotes,  Up: Characters and character classes

   (1) Requiring a bracketed character class to be non-empty is not a
limitation, since an empty character class can be more easily
represented by an empty string.

   (2) Following regexp custom, we identify "word" characters as
'[A-Za-z0-9_]', although these are too restrictive for what a Schemer
might consider a "word".


File: bigloo.info,  Node: Quantifiers,  Next: Clusters,  Prev: Characters and character classes,  Up: The Regular Expressions Pattern Language

12.2.5 Quantifiers
------------------

The _quantifiers_ '*', '+', and '?' match respectively: zero or more,
one or more, and zero or one instances of the preceding subpattern.

     (pregexp-match-positions "c[ad]*r" "cadaddadddr") => ((0 . 11))
     (pregexp-match-positions "c[ad]*r" "cr")          => ((0 . 2))

     (pregexp-match-positions "c[ad]+r" "cadaddadddr") => ((0 . 11))
     (pregexp-match-positions "c[ad]+r" "cr")          => #f

     (pregexp-match-positions "c[ad]?r" "cadaddadddr") => #f
     (pregexp-match-positions "c[ad]?r" "cr")          => ((0 . 2))
     (pregexp-match-positions "c[ad]?r" "car")         => ((0 . 3))

12.2.6 Numeric quantifiers
--------------------------

You can use braces to specify much finer-tuned quantification than is
possible with '*', '+', '?'.

  The quantifier '{m}' matches _exactly_ 'm' instances of the preceding
_subpattern_.  'm' must be a nonnegative integer.

  The quantifier '{m,n}' matches at least 'm' and at most 'n' instances.
'm' and 'n' are nonnegative integers with 'm <= n'.  You may omit either
or both numbers, in which case 'm' defaults to 0 and 'n' to infinity.

  It is evident that '+' and '?' are abbreviations for '{1,}' and
'{0,1}' respectively.  '*' abbreviates '{,}', which is the same as
'{0,}'.

     (pregexp-match "[aeiou]{3}" "vacuous")  => ("uou")
     (pregexp-match "[aeiou]{3}" "evolve")   => #f
     (pregexp-match "[aeiou]{2,3}" "evolve") => #f
     (pregexp-match "[aeiou]{2,3}" "zeugma") => ("eu")

12.2.7 Non-greedy quantifiers
-----------------------------

The quantifiers described above are _greedy_, ie, they match the maximal
number of instances that would still lead to an overall match for the
full pattern.

     (pregexp-match "<.*>" "<tag1> <tag2> <tag3>")
      => ("<tag1> <tag2> <tag3>")

  To make these quantifiers _non-greedy_, append a '?' to them.
Non-greedy quantifiers match the minimal number of instances needed to
ensure an overall match.

     (pregexp-match "<.*?>" "<tag1> <tag2> <tag3>") => ("<tag1>")

  The non-greedy quantifiers are respectively: '*?', '+?', '??', '{m}?',
'{m,n}?'.  Note the two uses of the metacharacter '?'.


File: bigloo.info,  Node: Clusters,  Next: Alternation,  Prev: Quantifiers,  Up: The Regular Expressions Pattern Language

12.2.8 Clusters
---------------

_Clustering_, ie, enclosure within parens '('...')', identifies the
enclosed _subpattern_ as a single entity.  It causes the matcher to
_capture_ the _submatch_, or the portion of the string matching the
subpattern, in addition to the overall match.

     (pregexp-match "([a-z]+) ([0-9]+), ([0-9]+)" "jan 1, 1970")
      => ("jan 1, 1970" "jan" "1" "1970")

  Clustering also causes a following quantifier to treat the entire
enclosed subpattern as an entity.

     (pregexp-match "(poo )*" "poo poo platter") => ("poo poo " "poo ")

  The number of submatches returned is always equal to the number of
subpatterns specified in the regexp, even if a particular subpattern
happens to match more than one substring or no substring at all.

     (pregexp-match "([a-z ]+;)*" "lather; rinse; repeat;")
      => ("lather; rinse; repeat;" " repeat;")

Here the '*'-quantified subpattern matches three times, but it is the
last submatch that is returned.

  It is also possible for a quantified subpattern to fail to match, even
if the overall pattern matches.  In such cases, the failing submatch is
represented by '#f'.

     (define date-re
       ;match `month year' or `month day, year'.
       ;subpattern matches day, if present
       (pregexp "([a-z]+) +([0-9]+,)? *([0-9]+)"))

     (pregexp-match date-re "jan 1, 1970")
      => ("jan 1, 1970" "jan" "1," "1970")

     (pregexp-match date-re "jan 1970")
      => ("jan 1970" "jan" #f "1970")

12.2.9 Backreferences
---------------------

Submatches can be used in the insert string argument of the procedures
'pregexp-replace' and 'pregexp-replace*'.  The insert string can use
'\n' as a _backreference_ to refer back to the _n_th submatch, ie, the
substring that matched the _n_th subpattern.  '\0' refers to the entire
match, and it can also be specified as '\&'.

     (pregexp-replace "_(.+?)_"
       "the _nina_, the _pinta_, and the _santa maria_"
       "*\\1*")
      => "the *nina*, the _pinta_, and the _santa maria_"

     (pregexp-replace* "_(.+?)_"
       "the _nina_, the _pinta_, and the _santa maria_"
       "*\\1*")
      => "the *nina*, the *pinta*, and the *santa maria*"

     ;recall: \S stands for non-whitespace character

     (pregexp-replace "(\\S+) (\\S+) (\\S+)"
       "eat to live"
       "\\3 \\2 \\1")
      => "live to eat"

  Use '\\' in the insert string to specify a literal backslash.  Also,
'\$' stands for an empty string, and is useful for separating a
backreference '\n' from an immediately following number.

  Backreferences can also be used within the regexp pattern to refer
back to an already matched subpattern in the pattern.  '\n' stands for
an exact repeat of the _n_th submatch.(1)  (*note Clusters-Footnote-1::)

     (pregexp-match "([a-z]+) and \\1"
       "billions and billions")
      => ("billions and billions" "billions")

Note that the backreference is not simply a repeat of the previous
subpattern.  Rather it is a repeat of _the particular substring already
matched by the subpattern_.

  In the above example, the backreference can only match 'billions'.  It
will not match 'millions', even though the subpattern it harks back to
-- '([a-z]+)' -- would have had no problem doing so:

     (pregexp-match "([a-z]+) and \\1"
       "billions and millions")
      => #f

  The following corrects doubled words:

     (pregexp-replace* "(\\S+) \\1"
       "now is the the time for all good men to to come to the aid of of the party"
       "\\1")
      => "now is the time for all good men to come to the aid of the party"

  The following marks all immediately repeating patterns in a number
string:

     (pregexp-replace* "(\\d+)\\1"
       "123340983242432420980980234"
       "{\\1,\\1}")
      => "12{3,3}40983{24,24}3242{098,098}0234"

12.2.10 Non-capturing clusters
------------------------------

It is often required to specify a cluster (typically for quantification)
but without triggering the capture of submatch information.  Such
clusters are called _non-capturing_.  In such cases, use '(?:' instead
of '(' as the cluster opener.  In the following example, the
non-capturing cluster eliminates the "directory" portion of a given
pathname, and the capturing cluster identifies the basename.

     (pregexp-match "^(?:[a-z]*/)*([a-z]+)$"
       "/usr/local/bin/mzscheme")
      => ("/usr/local/bin/mzscheme" "mzscheme")

12.2.11 Cloisters
-----------------

The location between the '?' and the ':' of a non-capturing cluster is
called a _cloister_.(2)  (*note Clusters-Footnote-2::) You can put
_modifiers_ there that will cause the enclustered subpattern to be
treated specially.  The modifier 'i' causes the subpattern to match
_case-insensitively_:

     (pregexp-match "(?i:hearth)" "HeartH") => ("HeartH")

  The modifier 'x' causes the subpattern to match _space-insensitively_,
ie, spaces and comments within the subpattern are ignored.  Comments are
introduced as usual with a semicolon (';') and extend till the end of
the line.  If you need to include a literal space or semicolon in a
space-insensitized subpattern, escape it with a backslash.

     (pregexp-match "(?x: a   lot)" "alot")
      => ("alot")

     (pregexp-match "(?x: a  \\  lot)" "a lot")
      => ("a lot")

     (pregexp-match "(?x:
        a \\ man  \\; \\   # ignore
        a \\ plan \\; \\   # me
        a \\ canal         # completely
        )"
      "a man; a plan; a canal")
      => ("a man; a plan; a canal")

  You can put more than one modifier in the cloister.

     (pregexp-match "(?ix:
        a \\ man  \\; \\   # ignore
        a \\ plan \\; \\   # me
        a \\ canal         # completely
        )"
      "A Man; a Plan; a Canal")
      => ("A Man; a Plan; a Canal")

  A minus sign before a modifier inverts its meaning.  Thus, you can use
'-i' and '-x' in a _subcluster_ to overturn the insensitivities caused
by an enclosing cluster.

     (pregexp-match "(?i:the (?-i:TeX)book)"
       "The TeXbook")
      => ("The TeXbook")

This regexp will allow any casing for 'the' and 'book' but insists that
'TeX' not be differently cased.


File: bigloo.info,  Node: Clusters-Footnotes,  Up: Clusters

   (1) '\0', which is useful in an insert string, makes no sense within
the regexp pattern, because the entire regexp has not matched yet that
you could refer back to it.

   (2) A useful, if terminally cute, coinage from the abbots of Perl.


File: bigloo.info,  Node: Alternation,  Next: Backtracking,  Prev: Clusters,  Up: The Regular Expressions Pattern Language

12.2.12 Alternation
-------------------

You can specify a list of _alternate_ subpatterns by separating them by
'|'.  The '|' separates subpatterns in the nearest enclosing cluster (or
in the entire pattern string if there are no enclosing parens).

     (pregexp-match "f(ee|i|o|um)" "a small, final fee")
      => ("fi" "i")

     (pregexp-replace* "([yi])s(e[sdr]?|ing|ation)"
        "it is energising to analyse an organisation
        pulsing with noisy organisms"
        "\\1z\\2")
      => "it is energizing to analyze an organization
        pulsing with noisy organisms"

  Note again that if you wish to use clustering merely to specify a list
of alternate subpatterns but do not want the submatch, use '(?:' instead
of '('.

     (pregexp-match "f(?:ee|i|o|um)" "fun for all")
      => ("fo")

  An important thing to note about alternation is that the leftmost
matching alternate is picked regardless of its length.  Thus, if one of
the alternates is a prefix of a later alternate, the latter may not have
a chance to match.

     (pregexp-match "call|call-with-current-continuation"
       "call-with-current-continuation")
      => ("call")

  To allow the longer alternate to have a shot at matching, place it
before the shorter one:

     (pregexp-match "call-with-current-continuation|call"
       "call-with-current-continuation")
      => ("call-with-current-continuation")

  In any case, an overall match for the entire regexp is always
preferred to an overall nonmatch.  In the following, the longer
alternate still wins, because its preferred shorter prefix fails to
yield an overall match.

     (pregexp-match "(?:call|call-with-current-continuation) constrained"
       "call-with-current-continuation constrained")
      => ("call-with-current-continuation constrained")


File: bigloo.info,  Node: Backtracking,  Next: Looking ahead and behind,  Prev: Alternation,  Up: The Regular Expressions Pattern Language

12.2.13 Backtracking
--------------------

We've already seen that greedy quantifiers match the maximal number of
times, but the overriding priority is that the overall match succeed.
Consider

     (pregexp-match "a*a" "aaaa")

The regexp consists of two subregexps, 'a*' followed by 'a'.  The
subregexp 'a*' cannot be allowed to match all four 'a''s in the text
string '"aaaa"', even though '*' is a greedy quantifier.  It may match
only the first three, leaving the last one for the second subregexp.
This ensures that the full regexp matches successfully.

  The regexp matcher accomplishes this via a process called
_backtracking_.  The matcher tentatively allows the greedy quantifier to
match all four 'a''s, but then when it becomes clear that the overall
match is in jeopardy, it _backtracks_ to a less greedy match of _three_
'a''s.  If even this fails, as in the call

     (pregexp-match "a*aa" "aaaa")

the matcher backtracks even further.  Overall failure is conceded only
when all possible backtracking has been tried with no success.

  Backtracking is not restricted to greedy quantifiers.  Nongreedy
quantifiers match as few instances as possible, and progressively
backtrack to more and more instances in order to attain an overall
match.  There is backtracking in alternation too, as the more rightward
alternates are tried when locally successful leftward ones fail to yield
an overall match.

12.2.14 Disabling backtracking
------------------------------

Sometimes it is efficient to disable backtracking.  For example, we may
wish to _commit_ to a choice, or we know that trying alternatives is
fruitless.  A nonbacktracking regexp is enclosed in '(?>'...')'.

     (pregexp-match "(?>a+)." "aaaa")
      => #f

  In this call, the subregexp '?>a*' greedily matches all four 'a''s,
and is denied the opportunity to backpedal.  So the overall match is
denied.  The effect of the regexp is therefore to match one or more
'a''s followed by something that is definitely non-'a'.


File: bigloo.info,  Node: Looking ahead and behind,  Prev: Backtracking,  Up: The Regular Expressions Pattern Language

12.2.15 Looking ahead and behind
--------------------------------

You can have assertions in your pattern that look _ahead_ or _behind_ to
ensure that a subpattern does or does not occur.  These "look around"
assertions are specified by putting the subpattern checked for in a
cluster whose leading characters are: '?=' (for positive lookahead),
'?!' (negative lookahead), '?<=' (positive lookbehind), '?<!' (negative
lookbehind).  Note that the subpattern in the assertion does not
generate a match in the final result.  It merely allows or disallows the
rest of the match.

12.2.16 Lookahead
-----------------

Positive lookahead ('?=') peeks ahead to ensure that its subpattern
_could_ match.

     (pregexp-match-positions "grey(?=hound)"
       "i left my grey socks at the greyhound")
      => ((28 . 32))

The regexp '"grey(?=hound)"' matches 'grey', but _only_ if it is
followed by 'hound'.  Thus, the first 'grey' in the text string is not
matched.

  Negative lookahead ('?!') peeks ahead to ensure that its subpattern
could not possibly match.

     (pregexp-match-positions "grey(?!hound)"
       "the gray greyhound ate the grey socks")
      => ((27 . 31))

The regexp '"grey(?!hound)"' matches 'grey', but only if it is _not_
followed by 'hound'.  Thus the 'grey' just before 'socks' is matched.

12.2.17 Lookbehind
------------------

Positive lookbehind ('?<=') checks that its subpattern _could_ match
immediately to the left of the current position in the text string.

     (pregexp-match-positions "(?<=grey)hound"
       "the hound in the picture is not a greyhound")
      => ((38 . 43))

The regexp '(?<=grey)hound' matches 'hound', but only if it is preceded
by 'grey'.

  Negative lookbehind ('?<!') checks that its subpattern could not
possibly match immediately to the left.

     (pregexp-match-positions "(?<!grey)hound"
       "the greyhound in the picture is not a hound")
      => ((38 . 43))

  The regexp '(?<!grey)hound' matches 'hound', but only if it is _not_
preceded by 'grey'.

  Lookaheads and lookbehinds can be convenient when they are not
confusing.


File: bigloo.info,  Node: An Extended Example,  Prev: The Regular Expressions Pattern Language,  Up: Posix Regular Expressions

12.3 An Extended Example
========================

Here's an extended example from Friedl that covers many of the features
described above.  The problem is to fashion a regexp that will match any
and only IP addresses or _dotted quads_, ie, four numbers separated by
three dots, with each number between 0 and 255.  We will use the
commenting mechanism to build the final regexp with clarity.  First, a
subregexp 'n0-255' that matches 0 through 255.

     (define n0-255
       "(?x:
       \\d          ;  0 through   9
       | \\d\\d     ; 00 through  99
       | [01]\\d\\d ;000 through 199
       | 2[0-4]\\d  ;200 through 249
       | 25[0-5]    ;250 through 255
       )")

  The first two alternates simply get all single- and double-digit
numbers.  Since 0-padding is allowed, we need to match both 1 and 01.
We need to be careful when getting 3-digit numbers, since numbers above
255 must be excluded.  So we fashion alternates to get 000 through 199,
then 200 through 249, and finally 250 through 255.(1)  (*note An
Extended Example-Footnote-1::)

  An IP-address is a string that consists of four 'n0-255's with three
dots separating them.

     (define ip-re1
       (string-append
         "^"        ;nothing before
         n0-255     ;the first n0-255,
         "(?x:"     ;then the subpattern of
         "\\."      ;a dot followed by
         n0-255     ;an n0-255,
         ")"        ;which is
         "{3}"      ;repeated exactly 3 times
         "$"        ;with nothing following
         ))

  Let's try it out.

     (pregexp-match ip-re1 "1.2.3.4")        => ("1.2.3.4")
     (pregexp-match ip-re1 "55.155.255.265") => #f

  which is fine, except that we also have

     (pregexp-match ip-re1 "0.00.000.00") => ("0.00.000.00")

  All-zero sequences are not valid IP addresses!  Lookahead to the
rescue.  Before starting to match 'ip-re1', we look ahead to ensure we
don't have all zeros.  We could use positive lookahead to ensure there
_is_ a digit other than zero.

     (define ip-re
       (string-append
         "(?=.*[1-9])" ;ensure there's a non-0 digit
         ip-re1))

  Or we could use negative lookahead to ensure that what's ahead isn't
composed of _only_ zeros and dots.

     (define ip-re
       (string-append
         "(?![0.]*$)" ;not just zeros and dots
                      ;(note: dot is not metachar inside [])
         ip-re1))

  The regexp 'ip-re' will match all and only valid IP addresses.

     (pregexp-match ip-re "1.2.3.4") => ("1.2.3.4")
     (pregexp-match ip-re "0.0.0.0") => #f


File: bigloo.info,  Node: An Extended Example-Footnotes,  Up: An Extended Example

   (1) Note that 'n0-255' lists prefixes as preferred alternates,
something we cautioned against in section *note Alternation::.  However,
since we intend to anchor this subregexp explicitly to force an overall
match, the order of the alternates does not matter.


File: bigloo.info,  Node: Command Line Parsing,  Next: Cryptography,  Prev: Posix Regular Expressions,  Up: Top

13 Command Line Parsing
***********************

Bigloo supports command line argument parsing.  That is, when an
application is spawn from an Unix shell, the 'main' function is called
and its argument is bound to the list of the command line arguments,
*Note Module Declaration: Modules.  The 'args-parse' form may be used to
parse these.

 -- bigloo syntax: args-parse list rules [null-rule] [else-rule] ...

     The argument LIST is a list of strings.  RULES is defined by the
     following grammar:

          <rule>      ==> (section <string>)
                        | ((<option> <help>) <s-expression>)
                        | ((<option>) <s-expression>)
                        | ((<flag> <var> <var> ...) <s-expression>)
                        | ((<flag> <var> <var> ... <help>) <s-expression>)
          <null-rule> ==> (() <s-expression>)
          <else-rule> ==> (else <s-expression>)
          <option>    ==> <flag>
                        | <string><var>
          <flag>      ==> <string>
                        | (<string>+)
          <var>       ==> an identifier leaded by the ? character
          <help>      ==> (help <s-expression>)
                        | (help <string> <s-expression>)

     Each elements of LIST are match against the RULES.  If one of these
     matches, 'args-parse' proceeds as follows:

       1. The matched argument elements of LIST are removed from the
          list.
       2. The '<s-expression>' associated to the matching rule is
          evaluated in an environment where the rule variables are
          bound.
       3. The argument parsing is resumed with the rest of LIST.

  In addition to parsing the command line arguments, 'args-parse'
enables help message printing.

 -- bigloo procedure: args-parse-usage fmt
     This is a procedure of one argument, an boolean.
     'Args-parse-usage' constructs an help message from all the option
     described in a 'args-parse' form.  'Args-parse-usage' is only
     defined in the '<s-expression>' of an 'args-parse' form.

  At last, if no rule matches an argument and if the 'args-parse' form
contains an 'else' rule, this is evaluated.  In the '<s-expression>'
part of that rule, the pseudo-variable 'else' is bound to the first
unmatched argument and the pseudo-variable 'rest' is bound to all the
unmatched arguments.

  Here is an example of argument parsing deploying all the possible
rules:

     (module args-example
        (main main))

     (define (main argv)
        (args-parse (cdr argv)
           (section "Help")
           (("?")
            (args-parse-usage #f))
           ((("-h" "--help") (help "?,-h,--help" "This help message"))
            (args-parse-usage #f))
           (section "Misc")
           ((("-v" "--version") (help "Version number"))
            (print *version*))
           (("-o" ?file (help "The output file"))
            (set! *dest* file))
           (("--input=?file" (help "The input file"))
            (set! *input* file))
           (else
            (print "Illegal argument `" else "'. Usage:")
            (args-parse-usage #f))))

  Invoking the compiled 'args-example' module could produce:

     > bigloo.new args.scm
     args.scm:
     > a.out toto
     Illegal argument 'toto'. Usage:

     Help:
        ?,-h,-help    -  This help message

     Misc:
        -v,-version   -  Version number
        -o <file>      -  The output file
        -input=<file> -  The input file


File: bigloo.info,  Node: Cryptography,  Next: Errors Assertions and Traces,  Prev: Command Line Parsing,  Up: Top

14 Cryptography
***************

Bigloo provides several functions for encrypting and decrypting
documents.  These are described in the chapter.  Unless explicitly
mentioned all functions presented in this document are accessible via
the 'crypto' library.

  _None of the cryptographic functions are protected against timing
attacks.  No effort has been spent on protecting used memory._

  Here is an example of a module that uses this library:

     ;; Encrypt a string using AES.
     (module aes-encrypt
        (library crypto)
        (main main))

     (define (main argv)
        (when (and (pair? (cdr argv)) (pair? (cddr argv)))
           (let ((encrypt? (string=? "-e" (cadr argv)))
                 (passwd (caddr argv))
                 (input (read-string)))
              (if encrypt?
                  (display (encrypt 'aes input passwd))
                  (display (decrypt 'aes input passwd))))))

* Menu:

* Symmetric Block Ciphers::
* Public Key Cryptography::
* OpenPGP::


File: bigloo.info,  Node: Symmetric Block Ciphers,  Next: Public Key Cryptography,  Up: Cryptography

14.1 Symmetric Block Ciphers
============================

* Menu:

* String to Key::

Bigloo supports some common block ciphers.  Block ciphers work on blocks
of fixed size.  A _mode of operation_ defines the way bigger input is
handled.  For instance in ECB (Electronic Codebook mode) the blocks are
all encrypted separately, whereas CBC (Cipher-Block Chaining) chains all
blocks.

  All modes that chain the blocks need an IV (Initial Vector) to
"bootstrap" the chaining.

  Block ciphers by themselves can only work on full blocks.  Some modes
are constructed in a way that even incomplete blocks can be safely
processed.  For the remaining blocks a padding function needs to be
given.

  Most block ciphers only work with keys of specific length.  The
following functions take passwords (strings of arbitrary length) as
input, and preprocess the given password by a _:string->key_ function.
The result must then be of correct length.

 -- bigloo cryptography procedure: encrypt::bstring cipher plain
          password [:string->key] [:mode 'cfb] [:IV #f] [:pad 'none]
          [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: encrypt-string::bstring cipher
          plaintext::bstring password [:string->key] [:mode 'cfb] [:IV
          #f] [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: encrypt-mmap::bstring cipher
          plaintext::mmap password [:string->key] [:mode 'cfb] [:IV #f]
          [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: encrypt-port::bstring cipher
          plaintext::input-port password [:string->key] [:mode 'cfb]
          [:IV #f] [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: encrypt-file::bstring cipher
          filename::bstring password [:string->key] [:mode 'cfb] [:IV
          #f] [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: encrypt-sendchars cipher
          in::input-port out::output-port password [:string->key] [:mode
          'cfb] [:IV #f] [:pad 'none] [:nonce-init!] [:nonce-update!]

     The procedure 'encrypt' encrypts its input using the chosen CIPHER.
     The result is returned as string.  'encrypt' dispatches depending
     on the type of PLAIN.  Strings are processed by 'encrypt-string'
     (and not 'encrypt-file').

     The function 'encrypt-sendchars' reads from an input-port IN and
     encrypts its output directly into an output-port OUT.

     The symbol CIPHER can be one of:
        * 'des': Data Encryption Standard (DES). DES works on blocks of
          64 bits.  DES requires keys of length 64 (bits), but only 56
          of these bits are actually used.  Bigloo's implementation
          therefore accepts both.  _DES is considered to be insecure and
          its usage is discouraged._
        * 'des3': Triple DES, Triple Data Encryption Algorithm (DES3,
          TDEA). DES3 works on blocks of 64 bits.  DES3 requires keys of
          length 128 or 192 (bits), but only 112/168 of these bits are
          actually used.  Bigloo's implementation therefore accepts the
          smaller keys too.

          Bigloo's DES3 implementation has been changed with release
          3.4b.  Earlier versions did not use the full key for
          en/decryption.

        * 'des-np': Same as 'des', but the initial and final
          permutations are not performed.
        * 'des3-np': Same as 'des3', but the initial and final
          permutations are not performed.
        * 'aes': Advanced Encryption Standard (AES). AES works on blocks
          of 128 bits.  AES requires keys of length 128, 192 or 256
          bits.
        * 'cast-128': CAST-128 (CAST5).  CAST-128 works on blocks of 64
          bits.  CAST-128 requires a key-length of 40-128 bits.
        * 'idea': International Data Encryption Algorithm (IDEA). IDEA
          works on blocks of 64 bits.  It requires keys of length 128
          (in bits).  _IDEA is patented in many countries (including the
          USA and most European countries) but it is free for
          non-commercial use._

     The given password must be a string.  An optional parameter
     :STRING->KEY should transform this password so that it has the
     correct length for the cipher.  A small list of possible functions
     are provided in the *note String to Key:: section.

     By default 'string->key-hash' with SHA-1 will be used.  The
     key-length will depend on the chosen cipher:
        * 'des': 56 bits.
        * 'des3': 112 bits.
        * 'des-np': Same as 'des'.
        * 'des3-np': Same as 'des3'.
        * 'aes': 192 bits.
        * 'cast-128': 128 bits.
        * 'idea': 128 bits.

     Bigloo supports the following block cipher modes (:MODE):
        * 'ecb': Electronic codebook.
        * 'cbc': Cipher-block chaining.
        * 'pcbc': Propagating cipher-block chaining.
        * 'cfb': Cipher feedback.
        * 'ofb': Output feedback.
        * 'ctr': Counter.

     By default 'cfb' is chosen.

     Electronic codebook mode en/decodes each block independently and is
     hence the closest to the block cipher.  It is however inherently
     unsafe as blocks with the same content are encrypted to the same
     output.

     With the exception of 'ecb' all other modes can be initialized with
     an IV (Initialization vector).  If :IV is false, then a random one
     will be generated.  During encryption this randomly generated IV
     will be prefixed to the result.  When calling the decryption
     routine without any IV the procedure will use the first block of
     the input as IV.

     In 'ctr' (counter) mode the IV parameter serves as nonce.  Two
     additional key-parameters ':nonce-init' and ':nonce-update' are
     then used to initialize and update the block-sized nonce string.
     Before encrypting the first block 'nonce-init' will be invoked with
     an empty block-sized string and the initial nonce (IV). It must
     initialize the string with the nonce.  For each block
     'nonce-update' will be called with the string, the nonce, and the
     number of already encrypted blocks (hence 0 at the very beginning).
     By default 'nonce-init' takes the IV-_string_ and blits it into the
     given string.  'nonce-update' simply increments the string
     (treating the given string as one big number).

     Note that the initial nonce (passed using IV) may be of any type.
     As long as 'nonce-init' and 'nonce-update' correctly initialize and
     update the passed string.

     The input's length of modes 'ecb', 'cbc' and 'pcbc' must be a
     multiple of the block-size.  Should this not be the case a padding
     algorithm must be specified (':pad').  Currently are implemented
     (examples for hexadecimal string "DD" and cipher block size 4):

        * 'none': No padding.  Raises an error should the input not be a
          multiple.
        * 'bit': Bit padding.  Add a '1' bit and then '0' bits.
          Example: "DD 80 00 00".
        * 'ansi-x.923': Byte padding.  Fill with #x00s followed by the
          number of added bytes (the counter inclusive).  Example: "DD
          00 00 03".
        * 'iso-10126': Fill with random characters followed by the
          number of added bytes (the counter inclusive).  Example: "DD
          42 31 03".
        * 'pkcs7': Fill with the number of added bytes.  Example: "DD 03
          03 03".
        * 'zero': Fill with zeros.  This is only reversible if the input
          is guaranteed not to finish with a zero character.  Example:
          "DD 00 00 00".

     Alternatively users can supply their own (un)pad functions (instead
     of a symbol).  The signature of a padding function is '(pad::bool
     str::bstring valid-chars::long)'.  It receives the last block of
     the input.  Should the input be of correct length then the an empty
     block will be sent to the padding function.  'valid-chars'
     indicates the number of read characters.  It ranges from 0 to
     blocksize-1.  The padding function should fill the block and return
     '#t' if this last block should be encoded.  By returning '#f' the
     last block will be discarded.  This makes only sense if
     'valid-chars' was equal to 0.

     The unpadding procedure has the signature '(unpad::long
     str::bstring)'.  The input string will have the length of the
     block-size.  The unpadding function may modify the string and must
     return the number of characters that are valid.

 -- bigloo cryptography procedure: decrypt::bstring cipher ciphertext
          password [:string->key] [:mode 'cfb] [:IV #f] [:pad 'none]
          [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: decrypt-string::bstring cipher
          ciphertext::bstring password [:string->key] [:mode 'cfb] [:IV
          #f] [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: decrypt-mmap::bstring cipher
          ciphertext::mmap password [:string->key] [:mode 'cfb] [:IV #f]
          [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: decrypt-port::bstring cipher
          ciphertext::input-port password [:string->key] [:mode 'cfb]
          [:IV #f] [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: decrypt-file::bstring cipher
          filename::bstring password [:string->key] [:mode 'cfb] [:IV
          #f] [:pad 'none] [:nonce-init!] [:nonce-update!]
 -- bigloo cryptography procedure: decrypt-sendchars cipher
          in::input-port out::output-port password [:string->key] [:mode
          'cfb] [:IV #f] [:pad 'none] [:nonce-init!] [:nonce-update!]

     Counterpart to the encryption functions.  With the same parameters
     the 'decrypt' function will decrypt the result of an 'encrypt'
     call.  Without :IV (Initial Vector) the 'decrypt' function will use
     the first block as IV.

  For compatibility the following functions remain in Bigloo.  They are
in the default library and not inside the 'crypto' library.

 -- bigloo procedure: aes-ctr-encrypt text password [nbits 128]
 -- bigloo procedure: aes-ctr-encrypt-mmap mmap password [nbits 128]
 -- bigloo procedure: aes-ctr-encrypt-string string password [nbits 128]
 -- bigloo procedure: aes-ctr-encrypt-port iport password [nbits 128]
 -- bigloo procedure: aes-ctr-encrypt-file filename password [nbits 128]
     These functions are equivalent to a call to 'aes-encrypt' with mode
     set to 'ctr' and a special ':string->key' parameter.  The optional
     argument NBITS must either be '128', '192', or '256' and determines
     the size of the key.

 -- bigloo procedure: aes-ctr-decrypt text password [nbits 128]
 -- bigloo procedure: aes-ctr-decrypt-mmap mmap password [nbits 128]
 -- bigloo procedure: aes-ctr-decrypt-string string password [nbits 128]
 -- bigloo procedure: aes-ctr-decrypt-port iport password [nbits 128]
 -- bigloo procedure: aes-ctr-decrypt-file filename password [nbits 128]
     Counterpart to 'aes-ctr-encrypt'.


File: bigloo.info,  Node: String to Key,  Up: Symmetric Block Ciphers

14.1.1 String to Key
--------------------

The following string->key algorithms take a password string and
transform it to a key string of a given length.  In all the functions
the LEN is expressed in bytes.

 -- bigloo cryptography procedure: string->key-zero str len
     If the length of the input string STR is greater or equal to LEN
     bytes then the first STR characters are returned.  Otherwise STR is
     suffixed with '0' ('#a000') characters.

 -- bigloo cryptography procedure: string->key-hash str len hash-fun
     The input string STR is run through the given hash function
     HASH-FUN.  The result is then concatenated multiple times (with
     itself) until a string of the LEN bytes is obtained.

     In the following example we encrypt SOME-MESSAGE using a password
     '"my password"'.  The password will be transformed to 256 bits (32
     bytes) using the 'string->key256' function.

          (define (string->key256 password)
            (string->key-hash password 32
                              (lambda (str) (string-hex-intern (sha1sum str)))))
          (encrypt 'aes some-message "my password" :string->key string->key256)

     Note that the following example yields an identical result:

          (define (string->key256 password)
            (string->key-hash password 32
                              (lambda (str) (string-hex-intern (sha1sum str)))))
          (encrypt 'aes some-message (string->key256 "my password")
                   :string->key (lambda (x) x))

 -- bigloo cryptography procedure: string->key-simple str len hash-fun

     This function implements the simple s2k algorithm of OpenPGP (RFC
     2440).  Basically STR is run through the hash-fun several times
     until the concatenation of the results is long enough.  At each
     iteration the string is prefixed with COUNT '0'-bytes (where COUNT
     is the iteration counter).

 -- bigloo cryptography procedure: string->key-salted str len hash-fun
          salt
     This function implements the salted s2k algorithm of OpenPGP (RFC
     2440).  Similar to 'string->key-simple' but the input string is
     first prefixed with SALT.

 -- bigloo cryptography procedure: string->key-iterated-salted str len
          hash-fun salt count
     This function implements the iterated salted s2k algorithm of
     OpenPGP (RFC 2440).  The variable COUNT must be a long.  This
     algorithm is an extension of 'string->key-salted' where the hash
     function is applied repeatedly.

     This function has changed with release 3.4b.  Earlier versions
     could be incompatible with RFC 2440.


File: bigloo.info,  Node: Public Key Cryptography,  Next: OpenPGP,  Prev: Symmetric Block Ciphers,  Up: Cryptography

14.2 Public Key Cryptography
============================

* Menu:

* RSA::
* DSA::
* ElGamal::
* PEM::


File: bigloo.info,  Node: RSA,  Next: DSA,  Up: Public Key Cryptography

14.2.1 Rivest, Shamir, and Adleman (RSA)
----------------------------------------

Bigloo's implementation of RSA is based on RFC 3447, PKCS #1 v2.1.  It
does not feature multiprime RSA, though.

  Bigloo's implementation is _not_ secure against timing attacks.
Furthermore some error codes might reveal information to attackers.

14.2.1.1 RSA Keys
.................

There are two kinds of RSA keys inside Bigloo: complete and partial
keys.  A complete key contains the information of both the public and
the private key (together with other information that could be
reconstructed out of the private key).  A partial key just contains the
modulus and the private or public exponent.

 -- bigloo cryptography class: RSA-Key
 -- bigloo cryptography class: Complete-RSA-Key
          (class Rsa-Key modulus::bignum exponent::bignum)
          (final-class Complete-Rsa-Key::Rsa-Key
            ;; for the complete-rsa-key "exponent" takes the role of 'd'
            e::bignum p::bignum q::bignum
            exp1::bignum   ;; d mod (p-1)
            exp2::bignum   ;; d mod (q-1)
            coeff::bignum) ;; (inverse of q) mod p

  RSA keys can be read and written using 'read-pem-key' and
'write-pem-key' (*note PEM::).

 -- bigloo cryptography procedure: generate-rsa-key [:key 1024]
          [:show-trace]

     This function generates a new RSA key (with its public and private
     components).

     Do not use this function for critical applications.  No special
     effort has been undertaken to guarantee the randomness of the
     generated prime numbers, nor to weed out insecure keys.

  Complete keys can be accessed using the following functions:
 -- bigloo cryptography procedure: extract-public-rsa-key complete-key
     Returns the public partial key of the given complete key.

     This procedure is implemented as follows:
          (define (extract-public-rsa-key::Rsa-Key key::Complete-Rsa-Key)
             (with-access::Complete-Rsa-Key key (modulus e)
                (make-Rsa-Key modulus e)))

 -- bigloo cryptography procedure: extract-private-rsa-key complete-key
     Returns the private partial key of the given complete key.

 -- bigloo cryptography procedure: rsa-key=? key1 key2
     Returns true if the two keys have the same modulus and public
     exponent.  The exponent of a partial key is considered to be
     public.

 -- bigloo cryptography procedure: rsa-key-length key
     Returns the key length in bytes.

14.2.1.2 RSA basic operations
.............................

RSA only works on bignums (up to the size of the modulus).  The
following procedures implement basic encryption, decryption, signing and
signature verification.

 -- bigloo cryptography procedure: rsa-encrypt key m
     Encrypts the bignum M using the given key.  If the key is a
     complete key then its public exponent is used.  For partial keys
     only one exponent is available (which is assumed to be the public
     'e' of the recipient).  The result is again a bignum.

 -- bigloo cryptography procedure: rsa-decrypt key c
     Decrypts the bignum C using the given key.  If the key is a
     complete key then its private exponent is used.  For partial keys
     only one exponent is available (which is assumed to be the private
     'd').  The result is again a bignum.

 -- bigloo cryptography procedure: rsa-sign k m
     Signs the bignum M using key K.  Uses the private exponent of
     complete keys.  The result is a bignum.

 -- bigloo cryptography procedure: rsa-verify k m s
     Verifies the signature S.  Returns true if S is the signature of M.
     The key K should be the public key of the signer.

14.2.1.3 Examples
.................

In this section we will present an example of using RSA.

  Let's start by generating an RSA key in openssl:
     $ openssl genrsa -out my_rsa_key.pem 1024

  Our key will have 1024 bits (for the public modulus), and therefore
RSA will only be able to work with bignums up to 1024 bits (128 bytes).

  Now some Bigloo code that uses this key.

  Start by loading the library.
     (module rsa-example (library crypto))

  Now read the key:
     (define *key* (read-pem-key "my_rsa_key.pem"))
     (define *public-key* (extract-public-rsa-key *key*))

  The public portion of the key can be distributed:
     ;; publish the *public-key*:
     (write-pem-key-string *public-key*)

  Now let's sign the message "My Important Message".  This message is
sufficiently short to be signed directly, but in general it is better to
get a hash of the message:
     (define msg-hash (sha1sum "my message"))
     (define msg-hash-bignum (octet-string->bignum msg-hash))

  The result of 'sha1sum' returns a human readable representation of the
hash.  It would hence be possible to transform it back to an internal
representation before applying the 'octet-string->bignum' function:
     (define msg-hash-bignum (octet-string->bignum (string-hex-intern msg-hash)))
  In our case both variants are small enough to fit into our keys.  The
latter version is however more often used.

  Now that we have a message hash in bignum form we can sign it.
     (define signature (rsa-sign *key* msg-hash-bignum))
  The signature is again in bignum form.  If needed there are several
ways to transform it into string-form (for instance 'bignum->string' or
'bignum->octet-string').

  The signature can now be distributed.  Anyone wanting to verify the
signature simply has to create the same message-hash and call
'rsa-verify' with our public key:

     (rsa-verify *public-key* msg-hash-bignum signature) => #t

  Encryption and decryption work in a similar way.

  Suppose someone (let's say "Alice") wants to send us the following
secret message "Cryptography".  The encryption and decryption functions
work, similar to the signature functions, on bignums.  We could, as
before, simply transform this short string into a bignum and directly
encrypt the bignum.  This approach would however not work for longer
strings.  In the following we will present the generic version that
works with strings of any size.

  Public key cryptography is relatively slow and Alice thus starts by
encrypting our message a fast block cipher with a "random" password:
     (define encrypted (encrypt 'aes "Cryptography" "my random password"))

  Alice can already send us the encrypted message.  We will just not yet
be able to decrypt it, as we don't have the random password yet.

  Alice now takes her random password string and encrypts it with our
public key:
     (define encrypted-key (rsa-encrypt *public-key* (octet-string->bignum "my random password")))

  Alice simply sends us the 'encrypted-key'.  On our side we can now
decrypt the key:
     (define aes-key (bignum->octet-string (rsa-decrypt *key* encrypted-key)))

  We can now decrypt the previously received message:
     (decrypt 'aes aes-key encrypted) => "Cryptography"

14.2.1.4 RSA RFC 3447
.....................

The following functions have been defined in RFC 3447.

 -- bigloo cryptography procedure: RSAEP k m
 -- bigloo cryptography procedure: RSADP k c
 -- bigloo cryptography procedure: RSASP1 k m
 -- bigloo cryptography procedure: RSAVP1 k s
     These are the RFC 3447 names for encryption, decryption, signature
     and signature verification.  Note that the verification does not
     receive the original message as parameter.

     In fact 'rsa-verify' is implemented as follows:
          (define (rsa-verify k m s)
            (=bx m (RSAVP1 k s)))

 -- bigloo cryptography procedure: PKCS1-v1.5-pad m-str key-len mode
 -- bigloo cryptography procedure: PKCS1-v1.5-unpad em-str mode
     Pads (resp.  unpads) the given string using PKCS1-v1.5
     specifications.  Mode must be 0, 1 or 2.

 -- bigloo cryptography procedure: RSAES-PKCS1-v1.5-encrypt key m-str
 -- bigloo cryptography procedure: RSAES-PKCS1-v1.5-decrypt key c-str
 -- bigloo cryptography procedure: RSASSA-PKCS1-v1.5-sign key msg-str
          [:hash-algo 'sha-1]
 -- bigloo cryptography procedure: RSASSA-PKCS1-v1.5-verify key msg-str
          S-str
 -- bigloo cryptography procedure: RSASSA-PKCS1-v1.5-sign-bignum key
          msg-str [:hash-algo 'sha-1]
 -- bigloo cryptography procedure: RSASSA-PKCS1-v1.5-verify-bignum key
          msg-str S
     RSAES-PKCS1-v1.5 functions work on strings.  However their length
     is limited by the size of the modulus (to be exact: by key-len -
     11).  The '-bignum' functions skip the last step of converting the
     internal bignum to strings.

     The optional ':hash-algo' must be either 'sha-1' or 'md5' (RFC 3447
     allows other hash algorithms, but they are not yet implemented).

 -- bigloo cryptography procedure: RSAES-OAEP-encrypt key m-str [:label
          ""]
 -- bigloo cryptography procedure: RSAES-OAEP-decrypt key cypher-str
          [:label ""]
 -- bigloo cryptography procedure: RSASSA-PSS-sign key msg-str
 -- bigloo cryptography procedure: RSASSA-PSS-verify key msg-str sig-str
     These functions pad, mask, etc the input string before they perform
     their operation on them.  See RFC 3447 for more information.


File: bigloo.info,  Node: DSA,  Next: ElGamal,  Prev: RSA,  Up: Public Key Cryptography

14.2.2 Digital Signature Algorithm (DSA)
----------------------------------------

Bigloo has rudimentary (but usually sufficient) support for DSA. While
it is not possible to generate new DSA keys inside Bigloo one can sign
or verify with Bigloo.

  DSA keys can be read and written using 'read-pem' (*note PEM::).

  For consistency with RSA we have named DSA keys in a similar way as
the RSA keys.  The public part of a DSA key can be found in the class
'DSA-Key' while the private part is added in the 'Complete-DSA-Key'
subclass.

 -- bigloo cryptography class: DSA-Key
 -- bigloo cryptography class: Complete-DSA-Key
          (class Dsa-Key
             p::bignum q::bignum g::bignum y::bignum)
          (final-class Complete-Dsa-Key::Dsa-Key
             x::bignum)) ;; the private key

 -- bigloo cryptography procedure: extract-public-dsa-key complete-key
     Returns a 'DSA-Key' without the private X.

 -- bigloo cryptography procedure: dsa-sign m key
     Signs the bignum M using the private dsa key KEY.  The result are
     two values: R and S.

     A typical call to 'dsa-sign' is hence of the following form
          (receive (r s)
            (dsa-sign secret-key hashed-msg-bignum)
            (process-signature r s))

 -- bigloo cryptography procedure: dsa-verify m key r s
     Verifies a signature (consisting of R and S).

  DSA works very similar to RSA. Have a look at RSA's example section.


File: bigloo.info,  Node: ElGamal,  Next: PEM,  Prev: DSA,  Up: Public Key Cryptography

14.2.3 ElGamal
--------------

Bigloo supports ElGamal encryption (but not signing).  Bigloo's
implementation is minimal.

  For consistency with RSA ElGamal keys are similarly named as their RSA
counterparts.

 -- bigloo cryptography class: ElGamal-Key
 -- bigloo cryptography class: Complete-ElGamal-Key
          (class ElGamal-Key
            p::bignum
            g::bignum
            y::bignum)
          (final-class Complete-ElGamal-Key::ElGamal-Key
            x::bignum)) ;; the private key

 -- bigloo cryptography procedure: extract-public-elgamal-key
          complete-key
     Returns a copy of the public part (as 'ElGamal Key').

 -- bigloo cryptography procedure: elgamal-encrypt key m
     Encrypts the bignum M using the given public key.  The result are
     two values 'c1' and 'c2'.

     Note that ElGamal encryption needs random bytes for every
     encryption.  This means that this function may return different
     results with the same parameters.  It furthermore implies that the
     result is insecure if the operating system provides bad random
     numbers, or if Bigloo's random-number generation is buggy.  For
     critical applications be sure to verify both requirements.

 -- bigloo cryptography procedure: elgamal-decrypt complete-key c1 c2
     Decrypts an ElGamal encrypted message (consisting of the two
     bignums C1 and C2) with the given private key.

 -- bigloo cryptography procedure: elgamal-key-length key
     Returns the key length in bytes.

  ElGamal works very similar to RSA. Have a look at RSA's example
section.


File: bigloo.info,  Node: PEM,  Prev: ElGamal,  Up: Public Key Cryptography

14.2.4 PEM
----------

Bigloo is able to read and write RSA and DSA keys in PEM format.  This
is the default format used by OpenSSL.

  The following example creates a new DSA key pair in OpenSSL and stores
it in PEM format.
     $ openssl dsaparam 1024 -out /tmp/dsaparam
     $ openssl gendsa /tmp/dsaparam

 -- bigloo cryptography procedure: read-pem-key in
 -- bigloo cryptography procedure: read-pem-key-port input-port
 -- bigloo cryptography procedure: read-pem-key-file filename
 -- bigloo cryptography procedure: read-pem-key-string str
     These functions will read a PEM encoded key.  The encoded file may
     contain a private or public RSA key, or a private or public DSA
     key.

     The procedure 'read-pem-key' accepts input-ports and strings.  In
     the case of a string it will invoke 'read-pem-key-file' (and not
     'read-pem-key-string').

 -- bigloo cryptography procedure: write-pem-key key out
          [public-key-only?]
 -- bigloo cryptography procedure: write-pem-key-port key out
          [public-key-only?]
 -- bigloo cryptography procedure: write-pem-key-file key out
          [public-key-only?]
 -- bigloo cryptography procedure: write-pem-key-string key
          [public-key-only?]
     These functions write the given key.  The key may be a
     private/public RSA/DSA key.

     The procedure 'write-pem-key' accepts output-ports and strings as
     OUT parameter.  If OUT is a string it will delegate to
     'write-pem-key-file'.


File: bigloo.info,  Node: OpenPGP,  Prev: Public Key Cryptography,  Up: Cryptography

14.3 OpenPGP
============

* Menu:

* Examples::
* Development::

Bigloo implements parts of OpenPGP (RFC 2440, RFC 4880).  All OpenPGP
functions are accessible via the 'openpgp' library.

  Here is an example of a module that uses this library:

     ;; Encrypt a string using openpgp default encryption.
     (module pgp-encrypt
        (library openpgp)
        (main main))

     (define (main argv)
        (when (and (pair? (cdr argv)) (pair? (cddr argv)))
           (let ((encrypt? (string=? "-e" (cadr argv)))
                 (passwd (caddr argv))
                 (input (read-string)))
              (if encrypt?
                  (display (pgp-write-string (pgp-encrypt input
                                                          '()  ;; no public keys
                                                          (list passwd))))
                  (let ((composition (pgp-read-string input)))
                    (display (pgp-decrypt composition
                                          :passkey-provider (lambda () passwd))))))))

 -- bigloo openpgp procedure: pgp-composition? composition
     Returns 'true' if and only if COMPOSITION is a PGP-Composition.
     Returns 'false' otherwise.

 -- bigloo openpgp procedure: pgp-read-string str
 -- bigloo openpgp procedure: pgp-read-port iport
 -- bigloo openpgp procedure: pgp-read-file file-name
     These functions read and decode PGP data.  OpenPGP allows several
     keys to be stored in the same message.  Therefore 'pgp-read' will
     return keys always in a list (even if the message only contains one
     key).

     The return value is either a list of PGP-compositions (PGP-Keys),
     or a single PGP-composition.

 -- bigloo openpgp procedure: pgp-write-string composition [:format
          'armored]
 -- bigloo openpgp procedure: pgp-write-port oport composition [:format
          'armored]
 -- bigloo openpgp procedure: pgp-write-file file-name composition
          [:format 'armored]
     The counter-part of 'pgp-read'.  These functions encode
     PGP-compositions.  By default the result is armored (i.e.  encoded
     with ASCII characters).  If the optional ':format' parameter is
     different than the symbol 'armored', then the composition is
     encoded in binary.

     Note that there is no means to encode a list of PGP-keys.

 -- bigloo openpgp procedure: pgp-encrypt msg-string keys passwords
          [:hash-algo 'sha-1] [:symmetric-algo 'cast5]
     Encrypts the given string.  The returned composition can be
     decrypted by the owners of the keys, or with one of the passwords.

     In the following example Alice and Bob may use their private key to
     decrypt the secret message.  Users knowing the one of the passwords
     ("foo" and "bar") will also be able to decrypt the message.
          (pgp-write-file "encrypted.pgp"
             (pgp-encrypt "my secret message"
                          (list alice-key bob-key)
                          '("foo" "bar")))
     The given keys should be subkeys of a PGP-key, but if a PGP-key is
     given Bigloo will do its best to pick the correct subkey for
     encryption.
        * If only one subkey exists (the main-key) then this subkey is
          used.
        * If two subkeys exist, and the non-main key is suitable for
          encryption, then the non-main key is used.
        * If only one of many subkeys (including the main-key) is
          suitable for encryption, then this subkey is used.
        * Else Bigloo raises an error.

 -- bigloo openpgp procedure: pgp-password-encrypt msg-string password
          [:hash-algo 'sha-1] [:symmetric-algo 'cast5] [:mdc #t]
     Deprecated.  Encrypts 'msg-string' with the given password.  The
     returned PGP-composition does not contain any information which
     hash-algorithm and symmetric encryption algorithm has been used.
     RFC 4880 specifies that IDEA and MD5 should be used.  However GnuPG
     uses SHA-1 and CAST5.  Therefore Bigloo defaults to the latter
     algorithms.

     Even though the usage of this PGP message is deprecated it yields
     the smallest encrypted data.  It may be of interest when
     compatibility with other tools is not a requirement (but why use
     OpenPGP then).

     The optional 'mdc' flag triggers the usage of a modification
     detection code.  It is more secure against tampering but requires
     more space and might not be recognized by old openpgp
     implementations.

 -- bigloo openpgp procedure: pgp-decrypt encrypted [:passkey-provider
          (lambda () #f)] [:password-provider (lambda (key) #f)]
          [:key-manager (lambda (key-id) '())] [:hash-algo 'sha-1]
          [:symmetric-algo 'cast5] [:ignore-bad-packets #f]
     Decrypts a PGP-composition that has been generated by 'pgp-encrypt'
     or by 'pgp-password-encrypt'.  The function returns the decrypted
     message (a string) or '#f' if decryption was not possible.

     If the message can be decrypted with a private key, then Bigloo
     will call the 'key-manager' and request a list of PGP-subkeys that
     match the given key-id.

     If a subkey (returned by the key-manager) is not yet decrypted,
     Bigloo will invoke the 'password-provider' with the subkey, and
     request a password to decrypt the private part of the subkey.

     If the message can be decrypted with a password Bigloo will then
     request a passkey by invoking the 'passkey-provider'.

     The optional arguments 'hash-algo' and 'symmetric-algo' are only
     used for messages that have been encrypted with
     'pgp-password-encrypt'.

     The option arguments 'ignore-bad-packets' enables decoding partial
     packets but pay attention that a message decoded with bad packets
     may have been tempered.

 -- bigloo openpgp procedure: pgp-sign msg-string key password-provider
          [:detached-signature? #t] [:one-pass? #f] [:hash-algo 'sha-1]
     Signs 'msg-string' with the given key.  Ideally the key should be a
     subkey, but if a complete PGP-Key is given, Bigloo will use the
     main-key instead.  If the main-key is not suitable for signing,
     then an error is raised.

     If the private part of the key has not yet been decrypted then
     Bigloo will call the password-provider (a procedure) with the
     subkey to get a password (a string).

     The function returns a PGP-composition.

     If the optional 'detached-signature?' parameter is set to '#f' then
     the msg-string is not included in the returned composition.

     The 'one-pass?' and 'hash-algo' parameters are usually left at its
     default values.

     Example:
          (let ((my-key (car (pgp-read-file "my-key.pgp"))))
            (pgp-write-file "msg.sig"
               (pgp-sign "my signed message"
                         my-key
                         (lambda (key) "my-password")
                         :detached-signature? #f)))

 -- bigloo openpgp procedure: pgp-verify signature key-manager [:msg #f]
     Verifies a signature.

     The key-manager is a function that takes a substring identifier and
     returns a list of keys matching this id.  Since a signature
     composition may contain several signatures this function may be
     invoked several times.

     The result is a list of subkeys that signed the message.  If the
     key-manager doesn't have any of the signature-keys then the result
     is the empty list.

     A message (string) needs only be given if the signature is
     detached.  Otherwise the original message is encoded in the
     signature-composition.

     Example:
          (let ((sig (pgp-read-file "msg.sig")))
            (let ((signers (pgp-verify sig my-key-manager)))
             (for-each (lambda (subkey)
                         (print (subkey->string subkey) " signed the message"))
                       signers)))

 -- bigloo openpgp procedure: pgp-signature-message signature
     Returns the signature's message, or '#f' if the signature is a
     detached signature.

 -- bigloo openpgp procedure: pgp-key? key
 -- bigloo openpgp procedure: pgp-subkey? key
     Predicates for PGP-Key and PGP-Subkey.

 -- bigloo openpgp procedure: pgp-subkeys key
     Returns a list of PGP-Subkeys of the PGP-Key.  The first key in the
     list is the _main-key_.  The main-key is used as default for
     signatures.

 -- bigloo openpgp procedure: pgp-key->string key
 -- bigloo openpgp procedure: pgp-subkey->string key
     Returns a string representation of the key (resp.  subkey).

     Example outputs:
          (pgp-key->string key)
          => John Doe john.doe@gmail.com
          => bd4df3b2ddef790c RSA (Encrypt or Sign)
          => 424610a65032c42e RSA (Encrypt or Sign)

          (pgp-subkey->string (car (pgp-subkeys key)))
          => John Doe john.doe@gmail.com
          => bd4df3b2ddef790c RSA (Encrypt or Sign)

 -- bigloo openpgp procedure: pgp-key-id subkey
 -- bigloo openpgp procedure: pgp-key-fingerprint subkey
     Returns the id (resp.  fingerprint) of a subkey.

     A subkey-id is a 8-character binary string.

     A fingerprint is a 20-character binary string.

 -- bigloo openpgp procedure: pgp-make-key-db
 -- bigloo openpgp procedure: pgp-key-db?
 -- bigloo openpgp procedure: pgp-add-key-to-db db key
 -- bigloo openpgp procedure: pgp-add-keys-to-db db keys
 -- bigloo openpgp procedure: pgp-resolve-key db id
 -- bigloo openpgp procedure: pgp-db-print-keys db
     A simple key-manager implementation based on lists.


File: bigloo.info,  Node: Examples,  Next: Development,  Up: OpenPGP

14.3.1 Examples
---------------

* Menu:

* Signatures::
* Email Usage::
* Encryption::


File: bigloo.info,  Node: Signatures,  Next: Email Usage,  Up: Examples

14.3.1.1 Signatures
...................

Unless you already have a gpg key create a new PGP key with gpg.  Note
that DSA with a keysize greater than 1024 does not work with SHA-1.
SHA-224,256,384,512 would work, but are not yet implemented in Bigloo.

     $ gpg -gen-key
     ...
     pub   1024D/A2DA694E 2010-08-07 [expires: 2010-08-27]
           Key fingerprint = DFAF 5894 9003 8640 D45B  6199 07CA 0495 A2DA 694E
     uid                  Bigloo Example
     sub   1024g/0B8985E5 2010-08-07 [expires: 2010-08-27]

  We export both the public and the private key.
     $ gpg  -a -o A8453FAB_Bigloo_Example_User.pkey -export A8453FAB
     $ gpg  -a -o A8453FAB_Bigloo_Example_User.skey -export-secret-keys A8453FAB

  This small program will simply read the key and print a human-readable
representation.
     ;; contents of print-key.scm
     (module print-key
        (library openpgp)
        (main my-main))

     (define (my-main args)
        (let ((public-key (car (pgp-read-file "A2DA694E_Bigloo_Example.pkey")))
              (secret-key (car (pgp-read-file "A2DA694E_Bigloo_Example.skey"))))
           (display (pgp-key->string public-key))
           (display (pgp-key->string secret-key))))

  The compilation is straight-forward and does not require any special
flags:
     $ bigloo print-key.scm -o print-key
     $ ./print-key
     Bigloo Example
     07ca0495a2da694e DSA (Digital Signature Standard)
     5fa4e8c90b8985e5 ElGamal (Encrypt-Only)
     Bigloo Example
     07ca0495a2da694e DSA (Digital Signature Standard)
     5fa4e8c90b8985e5 ElGamal (Encrypt-Only)

  As can be seen, the 'pgp-key->string' routine does not differentiate
between public and private keys.

  We can also sign a message:
     (let ((my-key (car (pgp-read-file "A2DA694E_Bigloo_Example.skey"))))
       (pgp-write-file "msg.sig"
          (pgp-sign (read-string)
                    my-key
                    (lambda (key) "<Bigloo Example Password>")
                    :detached-signature? #f)))

  Signatures from Bigloo follow RFC 4880 and can therefore be verified
by 'gpg'.
     $ echo "Gpg can verify Bigloo's signature" | ./sign
     $ gpg -verify msg.sig
     gpg: Signature made Sat 07 Aug 2010 10:12:21 PM CEST using DSA key ID A2DA694E
     gpg: Good signature from "Bigloo Example"

  Inversely Bigloo can verify 'pgp''s signature.  Here we first generate
a signature with 'gpg'.

     $ echo "Bigloo can verify gpg's signatures." | \
      gpg -o msg_gpg.sig -a \
          -default-key "Bigloo Example" \
          -passphrase <Bigloo Example Password> \
          -sign

     You need a passphrase to unlock the secret key for
     user: "Bigloo Example"
     1024-bit DSA key, ID A2DA694E, created 2010-08-07

  The following program reads OpenPGP signatures and verifies them.  For
simplicity the key database will only contain one key, but it could
contain any number of keys.
     (let ((my-key (car (pgp-read-file "A2DA694E_Bigloo_Example.pkey")))
           (sig (pgp-read-file "msg_gpg.sig"))
           (db (pgp-make-key-db)))
        (pgp-add-key-to-db db my-key)
        (print "Signature message: " (pgp-signature-message sig))
        (let ((signers (pgp-verify sig (lambda (id) (pgp-resolve-key db id)))))
           (for-each (lambda (subkey)
                        (display (pgp-subkey->string subkey)))
                     signers)))

  As expected, the program verifies the correct signature.
     $ ./verify
     Signature message: Bigloo can verify gpg's signatures.

     Bigloo Example
     07ca0495a2da694e DSA (Digital Signature Standard)


File: bigloo.info,  Node: Email Usage,  Next: Encryption,  Prev: Signatures,  Up: Examples

14.3.1.2 Email Usage
....................

Usage of OpenPGP within mails is described in RFC 3156.

  Encrypted parts and signatures are encoded with their separate
content-types.  Signatures are done over a canonicalized version of the
message.  They also hash over the content-type headers.

  OpenPGP's recette program has an example for a signature from kmail,
that can be succesfully verified with Bigloo.


File: bigloo.info,  Node: Encryption,  Prev: Email Usage,  Up: Examples

14.3.1.3 Encryption
...................

OpenPGP allows messages to be encrypted with passwords (in this context
"passkey") or public keys.  It is also possible to encrypt a message for
more than one recipient.  In such a case the data will be encrypted by a
session-key which in turn is encrypted separately for each recipient.
Since the session-key is not very big (compared to the data) the size
overhead is usually insignificant.

  Let's start by encrypting a message with a simple passkey.
     (let* ((secret-data "My secret data\n")
            (composition (pgp-encrypt secret-data '() '("My secret passkey"))))
       (pgp-write-file "secret.pgp" composition))

  As usual the pgp message is compatible with 'gpg':
     $ gpg secret.pgp
     gpg: CAST5 encrypted data
     Enter passphrase: <My secret passkey>
     gpg: encrypted with 1 passphrase
     $ cat secret
     My secret data

  As expected, Bigloo can decrypt password protected files that have
been generated by gpg:
     $ echo "A secret message encrypted with gpg." | \
       gpg -o encrypted.pgp -symmetric \
           -passphrase "secret key"

  The Bigloo code to decrypt the message is very simple:
     (print (pgp-decrypt (pgp-read-file "encrypted.pgp")
                         :passkey-provider (lambda () "secret key"))))

  In a similar vein it is possible to use public key encryption.  The
following example tests the encryption and decryption part of Bigloo.
     (let* ((my-key (car (pgp-read-file "A2DA694E_Bigloo_Example.skey")))
            (db (pgp-make-key-db))
            (secret-data "My secret message")
            (encrypted (pgp-encrypt secret-data `(,my-key) '())))
        (pgp-add-key-to-db db my-key)
        (let* ((key-manager (lambda (id) (pgp-resolve-key db id)))
               (password-provider (lambda (key) <Bigloo Example Password>))
               (decrypted (pgp-decrypt encrypted
                                      :key-manager key-manager
                                      :password-provider password-provider)))
          (if (not (string=? decrypted secret-data))
              (error "decrypt-test"
                     "Something went horribly wrong"
                     decrypted))))

  Note that a secret secret key has a part that is encrypted by a
password.  During decryption Bigloo needs access to this encrypted data
and therefore invokes the password-provider so it can decrypt it.  In
many cases this will trigger an interactive callback with the user.
Here, in this toy example, we know that the password that is needed is
for the Bigloo Example key.  In a more general case the
password-provider will have to print the key to give more information to
the user.

  In the following example we show how to encrypt data for 3 passwords
and one key.

     (let* ((my-key (car (pgp-read-file "A2DA694E_Bigloo_Example.skey")))
            (db (pgp-make-key-db))
            (secret-data "My secret message")
            (encrypted (pgp-encrypt secret-data `(,my-key)
                                                '("pass1" "pass2" "pass3"))))
        (pgp-write-file "multi_receiver.pgp" encrypted))

  We believe that 'gpg' has a bug and does not know how to handle such
messages correctly.  Bigloo, however, decrypts the message with any of
the possible options.


File: bigloo.info,  Node: Development,  Prev: Examples,  Up: OpenPGP

14.4 Development
================

Bigloo's OpenPGP implementation only exposes few library functions.  As
a consequence some features are not accessible.  The key-management
system is very rough, and there are no means to inspect messages in more
detail.  It should be possible to expose or implement many of those
missing features with little effort.  The most time-consuming part is
generally designing a clean API and the testing/debugging of new
features: when something goes wrong it can take a huge amount of time to
find the reason.

  Developers interested in improving Bigloo's OpenPGP library can print
a huge amount of debugging information by enabling the debug-macro in
'util.scm'.  Bigloo's OpenPGP implementation is not designed for speed
and takes no shortcuts.  The debugging output can therefore be used to
follow the specification of RFC 4880 (or 2440).


File: bigloo.info,  Node: Errors Assertions and Traces,  Next: Threads,  Prev: Cryptography,  Up: Top

15 Errors, Assertions, and Traces
*********************************

* Menu:

* Errors and Warnings::         Functions to raise errors and warnings
* Exceptions::                  Functions and classes dealing with exceptions
* Try::                         The deprecated try form
* Assertions::                  Assertions
* Tracing::                     Simple debug facility


File: bigloo.info,  Node: Errors and Warnings,  Next: Exceptions,  Up: Errors Assertions and Traces

15.1 Errors and Warnings
========================

Bigloo permits to signal an error via the 'error' function.  Errors are
implemented by the means of exceptions (see 'with-exception-handler',
'with-handler', and 'raise' forms).  Assertions allow the checking of
predicates at certain points in programs.

 -- bigloo procedure: typeof obj
     Returns a string which is the name of the dynamic type of OBJ.

 -- bigloo procedure: error proc msg obj
     This form signals an error by calling the current error handler
     with PROC, MSG and OBJ as arguments.

          (define (foo l)
             (if (not (pair? l))
                 (error "foo" "argument not a pair" l)
                 (car l)))

          (foo 4)
          error-> *** ERROR:bigloo:foo:
                   argument not a pair -- 4

     Switching on the '-g' compilation switch enables stack dumping when
     the 'error' function is invoked.  That is, when a program is
     compiled with '-g' and when, at runtime, the shell variable
     'BIGLOOSTACKDEPTH' is set and contains a number, an execution stack
     of depth 'BIGLOOSTACKDEPTH' is printed when an error is raised.

 -- bigloo procedure: error/location proc msg obj file location
     This form signals an error by calling the current error handler
     with PROC, MSG and OBJ as arguments.  The error is prompted in
     FILE, at character position LOCATION.

          (define (foo l)
             (if (not (pair? l))
                 (error/location
                   "foo" "argument not a pair" l "foo.scm" 115)
                 (car l)))

          (foo 4)
          error-> File "foo.scm", line 4, character 115:
                   #       (car l)))
                   #       ^
                   # *** ERROR:bigloo:foo
                   # argument not a pair -- 4
                       0. FOO
                       1. DYNAMIC-WIND
                       2. INTERP
                       3. ENGINE
                       4. MAIN

 -- bigloo procedure: get-trace-stack size
 -- bigloo procedure: dump-trace-stack output-port size
     Switching on the '-g' compilation switch enables stack dumping
     *note Compiler Description::.  That is, the list of the pending
     calls can be dumped by the runtime-system.  The function
     'get-trace-stack' builds such a trace.  The list built by
     'get-trace-stack' only contains the SIZE top most pending calls.
     The function 'dump-trace-stack' displays a representation of this
     stack on the OUTPUT-PORT.

 -- bigloo procedure: warning/location file location [arg]...
     This form signals a warning.  That is, is ARG are displayed on the
     standard error port.  The warning is prompted in FILE at character
     position LOCATION.

          (define (foo l)
             (if (not (pair? l))
                 (begin
                    (warning/location
                      "foo.scm" 154 "foo:" "argument not a pair -- " l)
                    '())
                 (car l)))

          (foo 4)
          -| File "foo.scm", line 6, character 154:
             #       (car l)))
             #       ^
             # *** WARNING:bigloo:foo:
             argument not a pair -- 4
          => '()

 -- bigloo procedure: exception-notify exc
 -- bigloo procedure: error-notify err
 -- bigloo procedure: warning-notify err
     Display a message describing the error or warning on the default
     error port.


File: bigloo.info,  Node: Exceptions,  Next: Try,  Prev: Errors and Warnings,  Up: Errors Assertions and Traces

15.2 Exceptions
===============

 -- SRFI-18 function: current-exception-handler
     Returns the current exception handler with is a 0-ary procedure.

 -- SRFI-18 function: with-exception-handler handler thunk
     Returns the result(s) of calling THUNK with no arguments.  The
     HANDLER, which must be a procedure, is installed as the current
     exception handler in the dynamic environment in effect during the
     call to THUNK.  When possible, prefer 'with-handler' to
     'with-exception-handler' because the former provides better
     debugging support and because its semantics is more intuitive.

 -- bigloo form: with-handler handler body
     Returns the result(s) of evaluating BODY.  The HANDLER, which must
     be a procedure, is installed as the current exception handler in
     the dynamic environment in effect during the evaluation of BODY.
     Contrarily to 'with-exception-handler', if an exception is raised,
     the HANDLER is invoked and the value of the 'with-handler' form is
     the value produced by invoking the HANDLER.  The handler is
     executed in the continuation of the 'with-handler' form.

     JVM note: When executed within a JVM, the form 'with-handler' also
     catches Java exceptions.

     Important note: Since Bigloo version 3.2c, error handlers are
     executed _after_ the execution stack is unwound.  Hence, error
     handlers are executed _after_ protected blocks.  For instance in
     the following code:

          (with-handler
             (lambda (e) action)
             (unwind-protect
                body
                protect))

     The ACTION is executed _after_ PROTECT.

 -- SRFI-18 function: raise obj

     Calls the current exception handler with obj as the single
     argument.  obj may be any Scheme object.  Note that invoking the
     current handler does not escape from the current computation.  It
     is up the to handler to perform the escape.  It an error, signaled
     by the runtime system, if the current exception handler returns.

          (define (f n)
            (if (< n 0) (raise "negative arg") (sqrt n))))

          (define (g)
            (bind-exit (return)
              (with-exception-handler
                (lambda (exc)
                  (return
                    (if (string? exc)
                        (string-append "error: " exc)
                        "unknown error")))
                (lambda ()
                  (write (f 4.))
                  (write (f -1.))
                  (write (f 9.))))))

          (g)  -| 2. and returns "error: negative arg"

  The standard Bigloo runtime system uses the following classes for
signaling errors and warnings.

 -- class: &exception
          (class &exception
             (fname read-only (default #f))
             (location read-only (default #f)))

 -- class: &error::&exception
          (class &error::&exception
             (proc read-only)
             (msg read-only)
             (obj read-only))

 -- class: &type-error::&error

          (class &type-error::&error
             (type read-only))

 -- class: &io-error::&error

          (class &io-error::&error)

 -- class: &io-port-error::&io-error

          (class &io-port-error::&io-error)

 -- class: &io-read-error::&io-error

          (class &io-read-error::&io-port-error)

 -- class: &io-write-error::&io-port-error

          (class &io-write-error::&io-port-error)

 -- class: &io-closed-error::&io-port-error

          (class &io-closed-error::&io-port-error)

 -- class: &io-file-not-found-error::&io-error

          (class &io-file-not-found-error::&io-error)

 -- class: &io-parse-error::&io-error

          (class &io-parse-error::&io-error)

 -- class: &io-unknown-host-error::&io-error

          (class &io-unknown-host-error::&io-error)

 -- class: &io-malformed-url-error::&io-error

          (class &io-malformed-url-error::&io-error)

 -- class: &http-error::&error

          (class &http-error::&error)

 -- class: &http-redirection-error::&http-error

          (class &http-redirection-error::&http-error)

 -- class: &http-status-error::&http-error

          (class &http-status-error::&http-error)

 -- class: &http-redirection::&http-error

          (class &http-redirection::&exception
            (port::input-port read-only)
            (url::bstring read-only))

 -- class: process-exception::&error

          (class &process-exception::&error)

 -- class: &warning::&exception

          (class &warning::&exception
             (args read-only))

 -- class: &eval-warning::&warning

          (class &warning::&warning)


File: bigloo.info,  Node: Try,  Next: Assertions,  Prev: Exceptions,  Up: Errors Assertions and Traces

15.3 Deprecated try form
========================

 -- bigloo syntax: try exp handler

     This form is deprecated.  As much as possible, it should be
     replaced with true exceptions (i.e., 'with-exception-handler' and
     'raise').  The argument EXP is evaluated.  If an error is raised,
     the HANDLER is called.  The argument HANDLER is a procedure of four
     arguments.  Its first argument is the continuation of TRY.  The
     other arguments are PROC, MES and OBJ.  Invoking the first argument
     will resume after the error.

          (let ((handler (lambda (escape proc mes obj)
                            (print "***ERROR:" proc ":" mes " -- " obj)
                            (escape #f))))
             (try (car 1) handler))
             -| ***ERROR:CAR:not a pair -- 1
             => #f

     The argument HANDLER is not evaluated in the dynamic scope of its
     'try' form.  That is:

          (let ((handler (lambda (escape proc mes obj)
                            (escape (car obj)))))
             (try (car 1) handler))
             error-> *** ERROR:bigloo:CAR
                      Type `PAIR' expected, `BINT' provided -- 1

  Some library functions exist to help in writing handlers:

 -- bigloo procedure: warning [arg]...
     This form signals a warning.  That is, is ARG are displayed on the
     standard error port.

          (define (foo l)
             (if (not (pair? l))
                 (begin
                    (warning "foo:" "argument not a pair -- " l)
                    '())
                 (car l)))

          (foo 4)
          -| *** WARNING:bigloo:foo:
             argument not a pair -- 4
          => '()


File: bigloo.info,  Node: Assertions,  Next: Tracing,  Prev: Try,  Up: Errors Assertions and Traces

15.4 Assertions
===============

 -- bigloo syntax: assert (var...) s-expression

     Assertions can be enabled or disabled using Bigloo's compilation
     flags '-g' flag to enable them).  If the assertions are disabled
     they are not evaluated.  If an assertion is evaluated, if the
     expression EXP does not evaluate to '#t', an error is signaled and
     the interpreter is launched in an environment where VAR... are
     bound to their current values.

     Assertion forms are legal expressions which _always_ evaluate to
     the 'unspecified' object.

     Here is an example of assertion usage:

          (module foo
             (eval (export foo)))

          (define (foo x y)
             [assert (x y) (< x y)]
             (labels ((gee (t)
                           [assert (t) (>= t 0)]
                           (let ((res (+ x t)))
                              [assert (res t) (> res 10)]
                              res)))
                (set! x (gee y))
                [assert (x) (> x 10)]
                x))

          (repl)

     This module is compiled with the '-g' flag to enable assertions,
     then the produced executable is run:

          $ a.out

          1:=> (foo 1 2)

          File "foo.scm", line 9, character 158:
          #                   [assert (res t) (> res 10)]
          #                   ^
          # *** ERROR:bigloo:assert
          # assertion failed - (BEGIN (> RES 10))
              0. GEE
              1. FOO
          ---------------
          Variables' value are :
             RES : 3
             T : 2
          ---------------
          *:=> ^D
          File "foo.scm", line 12, character 228:
          #      [assert (x) (> x 10)]
          #      ^
          # *** ERROR:bigloo:assert
          # assertion failed - (BEGIN (> X 10))
              0. FOO
          ---------------
          Variables' value are :
             X : 3
          ---------------

          *:=> 3
          1:=> (foo 1 2)
          File "foo.scm", line 9, character 158:
          #                   [assert (res t) (> res 10)]
          #                   ^
          # *** ERROR:bigloo:assert
          # assertion failed - (BEGIN (> RES 10))
              0. GEE
              1. FOO
          ---------------
          Variables' value are :
             RES : 3
             T : 2
          ---------------

          *:=>


File: bigloo.info,  Node: Tracing,  Prev: Assertions,  Up: Errors Assertions and Traces

15.5 Tracing
============

Bigloo provides a trace facility whose is intended for simple debugging
tasks.  It is a replacement for user 'display's that clutters the source
code.  Here is a typical example using it:

     (define (foo x)
        (with-trace 1 'foo
           (let loop ((n x))
     	 (with-trace 2 'loop
     	    (trace-item "n=" n)
     	    (when (> n 0)
     	       (let liip ((m n))
     		  (with-trace 2 'liip
     		     (trace-item "m=" m))
     		  (when (> m 0)
     		     (liip (- m 1))))
     	       (loop (- n 1)))))))

     (foo 3)

  which produces the following output:

     + foo
       |--+ loop
       |  |- n=3
       |  |--+ liip
       |  |  |- m=3
       |  |--+ liip
       |  |  |- m=2
       |  |--+ liip
       |  |  |- m=1
       |  |--+ liip
       |  |  |- m=0
       |  |--+ loop
       |  |  |- n=2
       |  |  |--+ liip
       |  |  |  |- m=2
       |  |  |--+ liip
       |  |  |  |- m=1
       |  |  |--+ liip
       |  |  |  |- m=0
       |  |  |--+ loop
       |  |  |  |- n=1
       |  |  |  |--+ liip
       |  |  |  |  |- m=1
       |  |  |  |--+ liip
       |  |  |  |  |- m=0
       |  |  |  |--+ loop
       |  |  |  |  |- n=0

  Traces generation is controlled by a set of functions and parameters
(see *note Parameters::).  The functions are described in this chapter.

 -- bigloo syntax: with-trace level label . body

     The variable LEVEL is the level of a trace.  It is a positive
     integer.  It enables simple filtering for traces.  A trace is
     displayed if and only if the debugging level used to compile or to
     execute the program is greater than the trace level.  The variable
     LABEL is a label, .e.i., an identifier denoting the trace.  This
     identifier will be displayed in debug mode.  The variable BODY is
     the body of the form, that is, the expression to be evaluated.

     Unless a trace is activated '(with-trace lv la body)' (when its
     level LV is greater than the current debug level) is equivalent to
     '(begin body)'.  When traces are activated, before executing BODY.

     The debugging level is controlled by two parameters: 'bigloo-debug'
     and 'bigloo-compiler-debug' (see *note Parameters::).

 -- bigloo function: trace-item . args
     This function displays all its arguments.  It has to be used nested
     in a WITH-TRACE form.

 -- bigloo function: trace-bold s
 -- bigloo function: trace-string s
     These two functions are provided for convenience.  They returns
     strings made of their parameters.

 -- bigloo function: trace-color color . args
     The COLOR argument is a positive integer.  This function returns a
     string which is the representation of ARGS and that appears on the
     terminal in color COLOR.

     Colors can be enable or disabled using the 'bigloo-trace-color'
     parameter (see *note Parameters::).

 -- bigloo function: trace-margin
 -- bigloo function: trace-margin-set!
     The TRACE-MARGIN parameter is used to control the characters that
     are displayed in the margin of a trace.  Usual applications should
     not use this.  However, it may be convenient to set the margin by
     hands in some context.  For instance, it can be used to
     distinguished threads in a multi-threaded application such as:

          (make-thread (lambda ()
                          (trace-margin-set! (trace-color 1 "="))
                          ...))
          (make-thread (lambda ()
                          (trace-margin-set! (trace-color 2 "="))
                          ...))

 -- bigloo function: trace-port
 -- bigloo function: trace-port-set!
     These functions return and set the output port used by traces.


File: bigloo.info,  Node: Threads,  Next: Database,  Prev: Errors Assertions and Traces,  Up: Top

16 Threads
**********

Bigloo supports multithreaded programming.  Two different libraries
programming are available.  The first one, the _Fair Thread_ (see
Section *note Fair Threads::), enables, simple, easy to develop and to
maintain code.  The second one, the Posix Thread (see Section *note
Posix Threads::) enables more easily to take benefit of the actual
parallelism that is now available on stock hardware.  Because it is
easier to program with 'fthread' than with 'pthread', we strongly
recommend to use the former as much as possible and leave the former for
specially demanding applications.  Both libraries are described in this
chapter.

* Menu:

* Thread Common Functions::
* Fair Threads::
* Posix Threads::
* Mixing Thread APIs::


File: bigloo.info,  Node: Thread Common Functions,  Next: Fair Threads,  Up: Threads

16.1 Thread Common Functions
============================

* Menu:

* Thread API::
* Mutexes::
* Condition Variables::

Bigloo implements SRFI-18 (Multithreading support).  This SRFI is
available at <http://srfi.schemers.org/srfi-18/srfi-18.html>.  As
Bigloo's threads are objects (see Section *note Object System::), the
SRFI-18's thread specific functions can be used with either the
'pthread' or the 'fthread' library.

  This section describes the functions that are available independently
of the multi-threading library.


File: bigloo.info,  Node: Thread API,  Next: Mutexes,  Up: Thread Common Functions

16.1.1 Thread API
-----------------

Bigloo uses a set of _primitive_ functions and methods to create, run
and handle thread.  For the sake of standardization the name and
semantic of SRFI-18 has been used.  This section presents only the
mandatory functions to program with threads in Bigloo.

  The most important difference with SRFI-18, is the missing of the
function 'make-thread', which is not available for all libraries, as it
can be hard to predict the type of thread which will be created if
several thread libraries are used simultaneously.  As threads are
regular Bigloo objects, they can be created using the 'instantiate'
syntax.  See the *note Fair Threads:: and *note Posix Threads:: specific
sections for more details about thread creation and examples.

  The examples given in this section use a _generic_ syntax with
'instantiate::thread', to run the examples, you will have to put them in
a function in a module (see Section *note Modules::, and import one of
the libraries using 'library' module declaration.

 -- SRFI-18 function: current-thread
     Returns the current thread.

 -- SRFI-18 function: thread? OBJ
     Returns '#t' if OBJ is a thread, otherwise returns '#f'.

 -- SRFI-18 function: thread-name THREAD
     Returns the name of the THREAD.

 -- SRFI-18 function: thread-specific THREAD
 -- SRFI-18 function: thread-specific-set! THREAD OBJ
     Returns and sets value in the specific field of the THREAD.  If no
     value has been set, 'thread-specific' returns an unspecified value.

          (let ((t (instantiate::thread
                    (body (lambda ()
                             (print (thread-specific (current-thread))))))))
             (thread-specific-set! t 'foo)
             (thread-start! t)) -| foo

 -- Bigloo function: thread-cleanup THREAD
 -- Bigloo function: thread-cleanup-set! THREAD FUN
     Associates a cleanup function to a thread.  The cleanup function is
     called with the thread itself.  The cleanup function is executed in
     a context where 'current-thread' is the thread owning the cleanup
     function.

          (let ((t (instantiate::thread (body (lambda () 'done) 'foo))))
             (thread-cleanup-set! t (lambda (v) (print (thread-name (current-thread))
          					     ", exit value: " v)))
             (thread-start! t)) -| foo, exit value: done

 -- Bigloo function: thread-parameter IDENT
 -- Bigloo function: thread-parameter-set! IDENT VALUE
     Returns the value of the parameter IDENT in the current thread.  If
     no value is bound to this parameter, '#f' is returned.

     A thread parameter is implemented by a chunk of memory specific to
     each thread.  All threads are created with an empty set of
     parameters.

  The next functions have different behaviors depending in the library
used, more details will be given in the specific sections below.

 -- SRFI-18 function: thread-start! THREAD [ARGS]
 -- Bigloo function: thread-start-joinable! THREAD
 -- SRFI-18 function: thread-join! THREAD [TIMEOUT]
 -- SRFI-18 function: thread-terminate! THREAD
 -- SRFI-18 function: thread-yield!
 -- SRFI-18 function: thread-sleep! TIMEOUT


File: bigloo.info,  Node: Mutexes,  Next: Condition Variables,  Prev: Thread API,  Up: Thread Common Functions

16.1.2 Mutexes
--------------

Thread locking mechanism is common to Fair Threads and Posix Threads.

 -- SRFI-18 function: mutex? OBJ
 -- SRFI-18 function: make-mutex [NAME]
 -- SRFI-18 function: make-spinlock [NAME]
 -- SRFI-18 function: mutex-name MUTEX
 -- SRFI-18 function: mutex-specific MUTEX
 -- SRFI-18 function: mutex-specific-set! MUTEX OBJ
 -- SRFI-18 function: mutex-state MUTEX
 -- SRFI-18 function, deprecated: mutex-lock! MUTEX [TIMEOUT [THREAD]]
 -- SRFI-18 function, deprecated: mutex-unlock! MUTEX

     The function 'make-spinlock' creates a spin lock on architectures
     on support it, otherwise it creates a regular mutex as if
     'make-mutex' was called.  The support for spin lock can be checked
     with:

     'bigloo-config 'have-spinlock'

          (let ((m (make-mutex)))
             (thread-start!
              (instantiate::thread
                 (body (lambda ()
                          (let loop ()
                             (if (mutex-lock! m 0)
                                 (begin
                                    (display "locked")
                                    (mutex-unlock! m))
                                 (begin
                                    (thread-yield!)
                                    (loop)))))))))
            -| locked

          (let ((res '()))
             (define (mutex-lock-recursively! mutex)
                (if (eq? (mutex-state mutex) (current-thread))
                    (let ((n (mutex-specific mutex)))
                       (mutex-specific-set! mutex (+ n 1)))
                    (begin
                       (mutex-lock! mutex)
                       (mutex-specific-set! mutex 0))))
             (define (mutex-unlock-recursively! mutex)
                (let ((n (mutex-specific mutex)))
                   (if (= n 0)
                       (mutex-unlock! mutex)
                       (mutex-specific-set! mutex (- n 1)))))
             (thread-start!
              (instantiate::thread
                 (body (lambda ()
                          (let ((m (make-mutex)))
                             (mutex-lock-recursively! m)
                             (mutex-lock-recursively! m)
                             (mutex-lock-recursively! m)
                             (set! res (cons (mutex-specific m) res))
                             (mutex-unlock-recursively! m)
                             (mutex-unlock-recursively! m)
                             (mutex-unlock-recursively! m)
                             (set! res (cons (mutex-specific m) res)))))))
             res)
            => (0 2)

 -- Bigloo form: synchronize mutex exp1 exp2 ...
     The function SYNCHRONIZE evaluates the expressions EXP1, EXP2, etc.
     The mutex MUTEX is acquired and released before EXP1 gets
     evaluated.  Its value is the value of the evaluated expression.
     The form 'synchronize' ensures that however the form returns, the
     mutex MUTEX is always unlocked.

          (synchronize mutex
             (print "before read...")
             (read p))

 -- Bigloo function, deprecated: with-lock mutex thunk
     The form 'with-lock' is similar to 'synchronize' into which it is
     expanded.

     The function WITH-LOCK evaluates the body of the THUNK.  The mutex
     MUTEX is acquired and released before THUNK gets invoked.  The
     function WITH-LOCK might be implemented as:

          (define (with-lock mutex thunk)
             (synchronize mutex
                (thunk)))


File: bigloo.info,  Node: Condition Variables,  Prev: Mutexes,  Up: Thread Common Functions

16.1.3 Condition Variables
--------------------------

 -- SRFI-18 function: condition-variable? OBJ
 -- SRFI-18 function: make-condition-variable [NAME]
 -- SRFI-18 function: condition-variable-name CV
 -- SRFI-18 function: condition-variable-specific CV
 -- SRFI-18 function: condition-variable-specific-set! CV OBJ
 -- Bigloo function: condition-variable-wait! CV MUTEX [timeout]
 -- SRFI-18 function: condition-variable-signal! CV
 -- SRFI-18 function: condition-variable-broadcast! CV
          (let ((res 0))
             (define (make-semaphore n)
                (vector n (make-mutex) (make-condition-variable)))
             (define (semaphore-wait! sema)
                (mutex-lock! (vector-ref sema 1))
                (let ((n (vector-ref sema 0)))
                   (if (> n 0)
                       (begin
                          (vector-set! sema 0 (- n 1))
                          (mutex-unlock! (vector-ref sema 1)))
                       (begin
                          (condition-variable-wait! (vector-ref sema 2) (vector-ref sema 1))
                          (mutex-unlock! (vector-ref sema 1))
                          (semaphore-wait! sema)))))
             (define (semaphore-signal-by! sema increment)
                (mutex-lock! (vector-ref sema 1))
                (let ((n (+ (vector-ref sema 0) increment)))
                   (vector-set! sema 0 n)
                   (if (> n 0)
                       (condition-variable-broadcast! (vector-ref sema 2)))
                   (mutex-unlock! (vector-ref sema 1))))
             (let ((sema (make-semaphore 10)))
                (let ((t1 (thread-start!
                           (instantiate::thread
                              (body (lambda ()
                                       (semaphore-wait! sema)
                                       (set! res (current-time)))))))
                      (t2 (thread-start!
                           (instantiate::thread
                              (body (lambda ()
                                       (let loop ((n 10))
                                          (if (> n 0)
                                              (begin
                                                 (semaphore-signal-by! sema 1)
                                                 (thread-yield!)
                                                 (loop (- n 1)))))))))))
                   (scheduler-start!)
                   res)))
            => 2


File: bigloo.info,  Node: Fair Threads,  Next: Posix Threads,  Prev: Thread Common Functions,  Up: Threads

16.2 Threads
============

Bigloo supports fair threads (see Section *note Thread::), a
specification of cooperative threads.  In this framework a thread must
explicitly or implicitly _yield_ the processor to the scheduler (see
Section *note Scheduler::).  Explicit cooperation is achieved by library
functions such as 'thread-yield!' or 'thread-sleep!'.  The scheduler
does not preempt a running thread to allocate the processor to another
waiting thread.  Fair threads have two drawbacks over preemptive
threads:

   * Cooperative threads are not able to benefit of multi-processors
     platforms.
   * Single threads programs must be adapted in order to be run
     concurrently.

  On the other hand, Fair threads have advantages that make them
suitable for a high level programming language such as Scheme:

   * Fair threads have a strong and well defined semantic.  Multi
     threaded programs using Fair threads are _deterministic_ thus
     programs that deploy Fair threads are _predictable_.
   * Fair threads are easier to program with because they hide most the
     of the concurrent programming pitfalls.  In particular, since Fair
     threads enforce a strong synchronization, there is no need to
     deploy techniques such as _mutex_, _semaphore_ or _condition
     variables_.

* Menu:

* Introduction::
* Fair Threads Api::
* SRFI-18 compatibility::

  This whole chapter has been written in collaboration with F.
Boussinot.  It uses materials on Fair threads that can be found at
<http://www-sop.inria.fr/indes/rp/FairThreads/html/FairThreads.html>.


File: bigloo.info,  Node: Introduction,  Next: Fair Threads Api,  Up: Fair Threads

16.2.1 Introduction to Fair Threads
-----------------------------------

Fair threads are cooperative threads run by a fair scheduler which gives
them equal access to the processor.  Fair threads can communicate using
broadcast events.  Their semantics does not depends on the executing
platform.  Fine control over fair threads execution is possible allowing
the programming of specific user-defined scheduling strategies.

  Contrary to standard sequential programming where the processor
executes a single program, in concurrent programming the processor is a
shared resource which is dispatched to several programs.  The term
_concurrent_ is appropriate because programs can be seen as concurrently
competing to gain access to the processor, in order to execute.

  Threads are a basic means for concurrent programming, and are widely
used in operating systems.  At language level, threads offer a way to
structure programs by decomposing systems in several concurrent
components; in this respect, threads are useful for modularity.

  However, threads are generally considered as low-level primitives
leading to over-complex programming.  Moreover, threads generally have
loose semantics, in particular depending on the underlying executing
platform; to give them a precise semantics is a difficult task, and this
is a clearly identified problem to get portable code.

  Bigloo proposes a new framework with clear and simple semantics, and
with an efficient implementation.  In it, threads are called _fair_;
basically a fair thread is a cooperative thread executed in a context in
which all threads always have equal access to the processor.  Fair
threads have a deterministic semantics, relying on previous work
belonging to the so-called reactive approach.


File: bigloo.info,  Node: Fair Threads Api,  Next: SRFI-18 compatibility,  Prev: Introduction,  Up: Fair Threads

16.2.2 Fair Threads Api
-----------------------

* Menu:

* Thread::
* Scheduler::
* Signal::

The Fair Thread library relies on the Posix Thread one, but you don't
need to import the 'pthread' library, as it is done automatically when
importing the 'fthread' one.

  The functions listed in *note Thread Common Functions:: can be used to
manipulates the Fair Thread, but 'thread-start-joinable!', as a fair
thread can always join any other fair thread in the same scheduler.


File: bigloo.info,  Node: Thread,  Next: Scheduler,  Up: Fair Threads Api

16.2.2.1 Thread
...............

 -- Bigloo syntax: instantiate::fthread (body THUNK) [(name NAME)]
     Returns a new thread which is not started yet.  The body of the
     thread is the body of the procedure THUNK.  The optional argument
     NAME can be use to identify the thread.  It can be any Bigloo
     value.

          (instantiate::fthread (body (lambda () (print 1) (thread-yield!)
          (print 2)))
                                (name 'my-thread))

     The former thread-start function can be rewritten as follow:

          (define (make-thread body . name)
            (if (pair? name)
                (instantiate::fthread (body body) (name (car name)))
                (instantiate::fthread (body body))))

 -- SRFI-18 function: thread-start! THREAD [SCHEDULER]
     Runs a thread created with 'make-thread'.  If SCHEDULER is
     provided, the THREAD is started in this particular scheduler.
     Otherwise, it is started in the current scheduler (see Section
     *note Scheduler::).  Threads are started at the beginning of
     _reactions_ (see Section *note Scheduler::).

 -- SRFI-18 function: thread-yield!
     The current thread _cooperates_.  That is, it is suspended for the
     _reaction_ and the scheduler selects a new thread to be resumed.
     The scheduler resumes the next avaliable thread.  If there is only
     one thread started in the scheduler, the same thread is resumed.  A
     _reaction_ corresponds to the invocation of a 'scheduler-react!'
     call (see Section *note Scheduler::).

 -- SRFI-18 function: thread-sleep! TIMEOUT
     The current thread _cooperates_ during _exactly_ TIMEOUT
     _reactions_ (see *note Scheduler::).  It is suspended and the
     scheduler selects a new thread to be resumed.  If there is only one
     thread started in the scheduler, the same thread will be resumed.

          (let ((t1 (instantiate::fthread
                       (body (lambda () (thread-sleep! 2) (display 'foo)))))
                (t2 (instantiate::fthread
                       (body (lambda () (let loop ((n 1))
                                           (display n)
                                           (thread-yield!)
                                           (if (< n 5)
                                           (loop (+ n 1)))))))))
             (thread-start! t1)
             (thread-start! t2)
             (scheduler-start!)) -| 12foo34

 -- SRFI-18 function: thread-terminate! THREAD
     Terminates THREAD at the end of the current reaction.

 -- SRFI-18 function: thread-join! THREAD [TIMEOUT [TIMEOUT-VAL]]
     The current thread waits until THREAD terminates or until TIMEOUT
     is reached (when supplied).  If the TIMEOUT is reached,
     'thread-join!' returns TIMEOUT-VAL.  If THREAD terminates,
     'thread-join!' returns the end-result of the THREAD or the
     end-exception if that thread terminates abnormally.

     If several threads wait for the termination of the same thread,
     they are all notified of the termination during the current
     reaction.

          (let* ((t1 (thread-start!
                      (instantiate::fthread
                         (body (lambda () (thread-sleep! 3) 'foo)))))
                 (t2 (thread-start!
                      (instantiate::fthread
                         (body (lambda () (print "t1: " (thread-join! t1 1)))))))
                 (t3 (thread-start!
                      (instantiate::fthread
                         (body (lambda () (print "t2: " (thread-join! t1 2 'bar)))))))
                 (t3 (thread-start!
                      (instantiate::fthread
                         (body (lambda () (print "t3: " (thread-join! t1)))))))
                 (t4 (thread-start!
                      (instantiate::fthread
                         (body (lambda () (print "t4: " (thread-join! t1))))))))
             (scheduler-start!))
             -| t1: #|%uncaught-exception [reason: (exception . join-timeout)]|
                t2: bar
                t3: foo
                t4: foo

     'thread-join!' can be used to wait for a Posix Thread termination.
     The 'pthread' object must be started with 'thread-start-joinable!'.

 -- Bigloo function: thread-suspend! THREAD
 -- Bigloo function: thread-resume! THREAD
     Suspends/resumes the THREAD at the end of reaction.  While
     suspended a thread is not eligible to get the processor by the
     scheduler.

 -- Bigloo function: thread-await! SIGNAL [TIMEOUT]

     Blocks the thread until SIGNAL has been broadcast or until TIMEOUT
     has elapsed.  The function 'thread-await!' returns the value
     associated with the previous emissions of the signal that took
     place during the reaction.

          (let ((t1 (thread-start! (instantiate::fthread
                                      (body (lambda ()
                                               (display (thread-await! 'foo))
                                               (display (thread-await! 'bar)))))))
                (t2 (thread-start! (instantiate::fthread
                                      (body (lambda ()
                                               (broadcast! 'foo 'val1-foo)
                                               (broadcast! 'foo 'val2-foo))))))
                (t3 (thread-start! (instantiate::fthread
                                      (body (lambda ()
                                               (thread-sleep! 2)
                                               (broadcast! 'bar 'val-bar)))))))
             (let loop ((n 1))
                (display n)
                (scheduler-react! (default-scheduler))
                (loop (+ n 1))))
            -| 1val2-foo23val-bar456...

     The function 'thread-await!' cannot be used to intercept all the
     signals broadcast during a reaction.  This is illustrated by the
     following example where obviously 'thread-await!' cannot intercept
     the emission of the signal:

          (thread-start! (instantiate::fthread (body (lambda ()
                                                        (thread-await! 'foo)
                                                        (broadcast! 'foo 1)))))
          (thread-start! (instantiate::fthread (body (lambda ()
                                                        (broadcast! 'foo 2)))))

 -- Bigloo function: thread-get-values! SIGNAL

     Terminates the instant for the thread (as 'thread-yield!') and
     returns, hence at the next instant, all the values associated with
     broadcast SIGNAL (see Section *note Signal::) during the previous
     scheduler reaction (see Section *note Scheduler::).

     Example:
          (thread-start! (instantiate::fthread
                            (body (lambda ()
                                     (for-each print (thread-get-values! 'foo))))))
          (thread-start! (instantiate::fthread
                            (body (lambda ()
                                     (broadcast! 'foo 1)
                                     (broadcast! 'foo 'foo)
                                     (broadcast! 'foo "blabla")))))
             -| 1
                foo
                blabla

     Example:
          (let ((t1 (thread-start!
                     (instantiate::fthread
                        (body (lambda ()
                                 (for-each print (thread-get-values! 'foo))))
                        (name 't1))))
                (t2 (thread-start!
                     (instantiate::fthread
                        (body (lambda ()
                                 (broadcast! 'foo (current-thread))
                                 (thread-yield!)
                                 ;; this second broadcast won't be intercepted
                                 ;; because it occurs during the next reaction
                                 (broadcast! 'foo (current-thread))))
                        (name 't2))))
                (t3 (thread-start!
                     (instantiate::fthread
                        (body (lambda ()
                                 (broadcast! 'foo (current-thread))
                                 (broadcast! 'foo (current-thread))))
                        (name 't3)))))
             (scheduler-start!))
             -| #<thread:t2>
                #<thread:t3>
                #<thread:t3>

 -- Bigloo function: thread-await-values! SIGNAL [TIMEOUT]
     This blocks the current thread until SIGNAL has been broadcast.  It
     then returns, at the next instant, all the values associated with
     all the broadcasts that took place during the instant.  It can be
     defined as:

          (define (thread-await-values! signal . tmt)
             (apply thread-await! signal tmt)
             (thread-get-values signal))

 -- Bigloo function: thread-await*! SIGNALS [TIMEOUT]
     Wait for one of a list of signals.  The function 'thread-await*!'
     can be compared to the Unix 'select' function.  The argument
     SIGNALS is a list of signal identifier.  The function
     'thread-await*!' blocks the current thread until one of the signal
     in the list SIGNALS is broadcast or until the optional numerical
     argument TIMEOUT is elapsed.  If the thread unblocks because the
     timeout is elapsed, 'thread-await*!' returns '#f'.  Otherwise it
     returns two values that have to be collected with
     'multiple-value-bind' (see *note Control Features::).  The first
     one is the value of the broadcast signal.  The second one is the
     broadcast signal.

     Example:
          (let ((res #f))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (let ((sig* (list 'foo 'bar)))
                             (multiple-value-bind (val1 sig1)
                                (thread-await*! sig*)
                                (multiple-value-bind (val2 sig2)
                                   (thread-await*! sig*)
                                   (thread-yield!)
                                   (multiple-value-bind (val3 sig3)
                                      (thread-await*! sig*)
                                      (set! res (list sig1 sig2 sig3))))))))))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-sleep! 2)
                          (broadcast! 'foo 1)))))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-sleep! 3)
                          (broadcast! 'bar 2)))))
             (scheduler-start!)
             res)
            => '(foo foo bar)

     A second example using timeouts:
          (let ((res #f))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (let ((sig* (list 'foo 'bar)))
                             (multiple-value-bind (val1 sig1)
                                (thread-await*! sig* 1)
                                (thread-yield!)
                                (multiple-value-bind (val2 sig2)
                                   (thread-await*! sig* 1)
                                   (thread-yield!)
                                   (multiple-value-bind (val3 sig3)
                                      (thread-await*! sig* 2)
                                      (set! res (list sig1 sig2 sig3))))))))))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-sleep! 2)
                          (broadcast! 'foo 1)))))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-sleep! 3)
                          (broadcast! 'bar 2)))))
             (scheduler-start!)
             res)
            => '(#f foo bar)

 -- Bigloo function: thread-get-values*! SIGNALS
     Terminates the instant for the thread (as 'thread-yield!') and
     returns, hence at the next instant, all the values associated with
     all broadcast SIGNALS (see Section *note Signal::) during the
     previous scheduler reaction (see Section *note Scheduler::).  The
     function 'thread-get-values*!' returns an _alist_ made of the
     scanned signal and their values.  That is the length of the returns
     list is the length of the list SIGNALS.  If a signal of the list
     SIGNALS has not been broadcast, its associated entry the list
     returned by 'thread-get-values*!' has an empty 'cdr'.

     Example:
          (let ((s1 'foo)
                (s2 'bar)
                (s3 'gee)
                (res #f))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-sleep! 2)
                          (broadcast! 'foo (current-time))
                          (broadcast! 'bar 0)))))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-await*! (list s1 s2 s3))
                          (set! res (thread-get-values*! (list s1 s2 s3)))))))
             (thread-start!
              (instantiate::fthread
                 (body (lambda ()
                          (thread-sleep! 2)
                          (broadcast! 'bar (current-time))))))
             (scheduler-start!)
             res)
            => ((foo 3) (bar 3 0) (gee))

     Used with asynchronous signal, the functions 'thread-await*!' and
     'thread-get-values*!' can be used to read concurrently, in a non
     blocking way, several files.

 -- Bigloo function: thread-await-values*! SIGNALS [TIMEOUT]
     This blocks the current thread until at least one of SIGNALS has
     been broadcast.  It then returns, at the next instant, all the
     values associated with all the broadcasts that took place during
     the instant.  It can be defined as:

          (define (thread-await-values*! signal . tmt)
             (apply thread-await*! signal tmt)
             (thread-get-values*! signal))


File: bigloo.info,  Node: Scheduler,  Next: Signal,  Prev: Thread,  Up: Fair Threads Api

16.2.2.2 Scheduler
..................

 -- Bigloo function: make-scheduler [STRICT-ORDER?] [ENVS]
     Creates a new scheduler.  The optional boolean argument
     'strict-order?' is used to ask the scheduler to always schedule the
     threads in the same order, it defaults to '#f'.  The optional
     arguments ENVS are _fair thread environments_ which will be defined
     in forthcoming Bigloo releases.

 -- Bigloo function: scheduler-strict-order?
 -- Bigloo function: scheduler-strict-order?-set! BOOL
     Gets or sets the strict scheduling policy of the scheduler.  If
     set, the threads will always be scheduled in the same order, until
     their termination.  By default, it is set to false, which improve
     performances when there is a lot of thread to schedule.

 -- Bigloo function: scheduler? OBJ
     Returns '#t' if OBJ is a scheduler.  Otherwise returns '#f'.

 -- Bigloo function: scheduler? OBJ
     Returns '#t' if OBJ is a scheduler.  Otherwise returns '#f'.

 -- Bigloo function: current-scheduler
     Returns the current scheduler.  The current scheduler is the
     scheduler which currently schedules the current thread.  This value
     is not mutable, as it is set during the call to 'thread-start!'.

 -- Bigloo function: default-scheduler [SCHEDULER]
     Sets or gets the default scheduler.  The default scheduler is the
     scheduler that will be used in the calls to 'scheduler-react!',
     'scheduler-start!' or 'thread-start!' if not specified.  It always
     exists a default scheduler.  That is, it is optional for an
     application to create a scheduler.

 -- Bigloo function: scheduler-react! [SCHEDULER]
     Executes all the threads started (see 'thread-start!', Section
     *note Thread::) in the scheduler until all the threads are blocked.
     A thread is blocked if the has explicitly yield the processor
     ('thread-yield!' and 'thread-sleep!') or because it is waiting a
     signal ('thread-await!').  A thread can be selected several times
     during the same reaction.  The function 'scheduler-react!' returns
     a symbol denoting the _state_ of the scheduler.  The possible
     states are:

        * 'ready' The Scheduler is ready to execute some threads.
        * 'done' All the threads started in the scheduler have
          terminated.
        * 'await' All the threads started in the scheduler are waiting
          for a signal.

     An invocation of 'scheduler-react!' is called a _reaction_.

 -- Bigloo function: scheduler-start! [ARG [SCHEDULER]]
     Executes 'scheduler-react!' as long as the scheduler is not done.
     If the optional argument SCHEDULER is not provided,
     'scheduler-start!' uses the current scheduler (see
     'current-scheduler').  The optional ARG can either be:
        * An integer standing for the number of times 'scheduler-react!'
          must be called.
        * A procedure F of one argument.  The procedure F is invoked
          after each reaction.  It is passed a value I which is the
          iteration number of the scheduler.  The reactions of the
          scheduler continue while F returns '#f'.

          (let* ((s (make-scheduler))
                 (t (instantiate::fthread
                       (body (lambda ()
                                (let loop ((n 0))
                                   (display n)
                                   (thread-yield!)
                                   (loop (+ 1 n))))))))
             (scheduler-start! 10 s))
            -| 0123456789

          (let* ((s (make-scheduler))
                 (t (instantiate::fthread
                       (body (lambda ()
                                (let loop ((n 0))
                                   (display n)
                                   (thread-yield!)
                                   (loop (+ 1 n))))))))
             (scheduler-start! (lambda (i) (read-char)) s))
            -| 0123456789

 -- Bigloo function: scheduler-terminate! [SCHEDULER]
     Terminates all the threads in SCHEDULER.

 -- Bigloo function: scheduler-instant [SCHEDULER]
     Returns the current reaction number of SCHEDULER.  The reaction
     number is the number of times 'scheduler-react!' has been invoked
     passing SCHEDULER as argument.


File: bigloo.info,  Node: Signal,  Prev: Scheduler,  Up: Fair Threads Api

16.2.2.3 Signal
...............

 -- Bigloo function: broadcast! SIGNAL [VAL]
     Broadcasts SIGNAL to all threads started in SCHEDULER
     _immediately_, that is during the reaction.  This function can only
     be called from within a running thread.  If the optional argument
     VAL is omitted, the SIGNAL is broadcast with an unspecified value.

          (thread-start! (instantiate::fthread
                            (body (lambda ()
                                     (thread-await! 'foo)
                                     (print (scheduler-instant (current-scheduler)))))))
          (thread-start! (instantiate::fthread
                            (body (lambda ()
                                     (broadcast! 'foo)))))
          (scheduler-start!)
            -| 1

 -- Bigloo function: scheduler-broadcast! SCHEDULER SIGNAL [VAL]
     At the next react broadcasts SIGNAL to all threads started in
     SCHEDULER.  This is used to impact running threads from outside any
     threads.  If the optional argument VAL is omitted, the SIGNAL is
     broadcast with an unspecified value.

 -- Bigloo function: make-asynchronous-signal PROC
     This function invokes in the background, the procedure PROC.  This
     function takes one parameter which is the signal that is broadcast
     when the invocation returns.  When the host operating system
     supports parallel executions, the invocation of PROC is executed in
     parallel with the waiting thread.

     Asynchronous signals can be used to implement non blocking system
     operations, such as input/output.  Here is an example that
     illustrates how to implement concurrent programs that behaves
     similarly with Fair Threads and Posix Threads.

          (define-expander read
             (lambda (x e)
                (cond-expand
          	 (fthread
          	  (thread-await!
          	   (make-aynchronous-signal
          	    (lambda (s)
          	       (read ,@(map (lambda (x) (e x e)) (cdr x)))))))
          	 (else
          	  `(read ,@(map (lambda (x) (e x e)) (cdr x)))))))


File: bigloo.info,  Node: SRFI-18 compatibility,  Prev: Fair Threads Api,  Up: Fair Threads

16.2.3 SRFI-18
--------------

This section presents the functions that are not necessary to Bigloo but
supported for compliance with SRFI-18, provided by the Fair Thread
library.

 -- SRFI-18 function: current-time [SCHEDULER]
     Returns the reaction number of SCHEDULER.

 -- SRFI-18 function: time? OBJ
 -- SRFI-18 function: time->seconds OBJ

 -- SRFI-18 function: join-timeout-exception? OBJ
 -- SRFI-18 function: abandoned-mutex-exception? OBJ
 -- SRFI-18 function: terminated-thread-exception? OBJ
 -- SRFI-18 function: uncaught-exception? OBJ
 -- SRFI-18 function: uncaught-exception-reason EXC


File: bigloo.info,  Node: Posix Threads,  Next: Mixing Thread APIs,  Prev: Fair Threads,  Up: Threads

16.3 Posix Threads
==================

This section describes two Posix-Like multi-threading Bigloo libraries.
The two libraries, 'pthread', and 'srfi-18' are all the same but the
'mutex-state' function that returns different results.  Because of these
differences that might seem thin at first glance, the 'pthread' library
is significantly faster than the 'srfi-18' library.  For that reason, it
is recommended to use the 'pthread' library instead of the 'srfi-18'
library that is mostly supported for backward compatibility.

  As much as possible, the names exported by this library are compatible
with the Fair Threads library (see Section *note Fair Threads::).

* Menu:

* Using Posix Threads::
* Threads API::
* Mutexes API::
* Condition Variables API::
* Semaphores API::
* SRFI-18::


File: bigloo.info,  Node: Using Posix Threads,  Next: Threads API,  Up: Posix Threads

16.3.1 Using Posix Threads
--------------------------

The Bigloo modules initialization model does not permit to create
threads before the main function is started.  In other words, it is
unsafe to use the Posix Threads API at the top level of modules.  On
some particular applications this might work correctly.  On other it
could produce an error message stating the threads cannot be created or
started before the pthread library is initialized.


File: bigloo.info,  Node: Threads API,  Next: Mutexes API,  Prev: Using Posix Threads,  Up: Posix Threads

16.3.2 Threads
--------------

 -- Bigloo syntax: instantiate::pthread (body THUNK) [(name NAME)]
 -- SRFI-18 function: make-thread THUNK [NAME]
     Returns a new thread which is not started yet.  The body of the
     thread is the body of the procedure THUNK.  The optional argument
     NAME can be use to identify the thread.  It can be any Bigloo
     value.

     Warning: the 'make-thread' function is deprecated, but still
     provided for a backward compatibility with previous release of
     Bigloo.  The use of this function is _highly_ discouraged, in favor
     of the 'instantiate::pthread' form.

          (module example
             (library pthread)
             (main main))

          (define (main argv)
             (make-thread
              (lambda ()
                 (print 1)
                 (thread-yield!)
                 (print 2))
              'my-thread))

 -- SRFI-18 function: thread-start! THREAD
 -- SRFI-18 function: thread-start-joinable! THREAD
     Runs a thread created with 'instantiate::pthread'.  By default,
     threads are detached, and thus, they cannot be joined.

 -- SRFI-18 function: thread-yield!
     The current thread _cooperates_.

 -- SRFI-18 function: thread-sleep! TIMEOUT

     The current thread _sleeps_ for a certain period.  It is suspended
     and the scheduler is free to select a new thread to be resumed.  If
     there is only one thread started in the scheduler, the same thread
     will be resumed.  The time of TIMEOUT is used to determine the time
     the thread must sleep.

     Here are the possible types for TIMEOUT.

        * 'date': the thread sleeps at least until the date TIMEOUT.
        * 'real': the thread sleeps at least TIMEOUT seconds.
        * 'fixum', 'elong', 'llong': the thread sleeps at least TIMEOUT
          milli-seconds.

 -- SRFI-18 function: thread-terminate! THREAD
     Terminates THREAD as soon as possible.

 -- SRFI-18 function: thread-join! THREAD [TIMEOUT]
     The current thread waits until the THREAD terminates.  If THREAD
     terminates, 'thread-join!' returns the end-result of the THREAD or
     the end-exception if that thread terminates abnormally.

     It is possible to wait for the termination of the a thread if and
     only if it has been started with 'thread-start-joinable!'.  In
     particular, threads started with 'thread-start!' cannot be joined.

     The optional argument TIMEOUT, forces to wait at for TIMEOUT
     milli-seconds for the thread to terminate.  Note that not all
     systems support this facility.  When supported, the 'cond-expand'
     (see *note SRFIs::) 'pthread-timedjoin' is defined.  When the
     timeout expires some systems, raise an error.  Other systems abort
     silently.

 -- SRFI-18 function: terminated-thread-exception? OBJ
 -- SRFI-18 function: uncaught-exception? OBJ
 -- SRFI-18 function: uncaught-exception-reason EXC


File: bigloo.info,  Node: Mutexes API,  Next: Condition Variables API,  Prev: Threads API,  Up: Posix Threads

16.3.3 Mutexes
--------------

Thread locking mechanism is common to Fair Threads and Posix Threads
(see *note Thread Common Functions::).

 -- SRFI-18 function: mutex-state mutex
     Returns the symbol 'locked' when the mutex is locked by a thread.
     Otherwise, it returns the symbol 'unlocked'.


File: bigloo.info,  Node: Condition Variables API,  Next: Semaphores API,  Prev: Mutexes API,  Up: Posix Threads

16.3.4 Condition Variables
--------------------------

Posix thread condition variables follows the common thread API (see
*note Thread Common Functions::).

     (module example
       (library pthread)
       (main main))

     (define (main argv)
        (let ((res #f)
     	 (lock (make-mutex))
     	 (cv (make-condition-variable)))
           (thread-join!
            (thread-start-joinable!
     	(instantiate::pthread
     	   (body (lambda ()
     		    (mutex-lock! lock)
     		    (thread-start!
     		     (instantiate::pthread
     			(body (lambda ()
     				 (mutex-lock! lock)
     				 (condition-variable-signal! cv)
     				 (mutex-unlock! lock)))))
     		    (condition-variable-wait! cv lock)
     		    (set! res 23)
     		    (mutex-unlock! lock))))))
           res))


File: bigloo.info,  Node: Semaphores API,  Next: Condition Variables API,  Prev: Condition Variables API,  Up: Posix Threads

16.3.5 Semaphores
-----------------

 -- bigloo function: open-semaphore NAME [CREATE #t] [EXCL #t] [READ #t]
          [WRITE #t] [VALUE 1]
 -- bigloo function: close-semaphore SEM
 -- bigloo function: delete-semaphore NAME
     Opens, closes, and deletes a named semaphore.

 -- bigloo function: semaphore? OBJ
     Returns '#t' if OBJ is a semaphore, returns '#f' otherwise.

 -- bigloo function: semaphore-post SEM
 -- bigloo function: semaphore-wait SEM
 -- bigloo function: semaphore-trywait SEM
     Wait and post for a semaphore.

 -- bigloo function: semaphore-value SEM
     Returns the current value of the semaphoe


File: bigloo.info,  Node: SRFI-18,  Prev: Semaphores API,  Up: Posix Threads

16.3.6 SRFI-18
--------------

 -- SRFI-18 function: mutex-state mutex

     Returns information about the state of the mutex.  The possible
     results are:

        * thread T: the mutex is in the locked/owned state and thread T
          is the owner of the mutex
        * symbol 'not-owned': the mutex is in the locked/not-owned state
        * symbol 'abandoned': the mutex is in the unlocked/abandoned
          state
        * symbol 'not-abandoned': the mutex is in the
          unlocked/not-abandoned state

     Examples:

          (mutex-state (make-mutex))
             => not-abandoned

          (define (thread-alive? thread)
             (let ((mutex (make-mutex)))
                (mutex-lock! mutex #f thread)
                (let ((state (mutex-state mutex)))
                   (mutex-unlock! mutex) ; avoid space leak
                   (eq? state thread))))



File: bigloo.info,  Node: Mixing Thread APIs,  Prev: Posix Threads,  Up: Threads

16.4 Mixing Thread APIs
=======================

The *note Fair Threads:: library is "Posix Threads" safe, which means it
is possible to use at the same time both libraries.  In other words, it
is possible to embed one fair scheduler into a Posix thread.

  Here is a little example with two schedulers started into two
different Posix threads, each schedulers running two fair threads.

     (module mix_threads
        (library fthread pthread)
        (main main))

     (define *f1* 0)
     (define *f2* 0)

     (define (main args)
        (let ((s1 (make-scheduler #t))
     	 (s2 (make-scheduler #t))

     	 (f1a (instantiate::fthread
     		 (body (lambda ()
     			  (let loop ()
     			     (print "f1a: " *f1* " " (current-thread))
     			     (set! *f1* (+ 1 *f1*))
     			     (thread-yield!)
     			     (loop))))))

     	 (f1b (instantiate::fthread
     		 (body (lambda ()
     			  (let loop ()
     			     (print "f1b: " *f1* " " (current-thread))
     			     (set! *f1* (+ 1 *f1*))
     			     (thread-yield!)
     			     (loop))))))

     	 (f2a (instantiate::fthread
     		 (body (lambda ()
     			  (let loop ()
     			     (print "f2a: " *f2* " " (current-thread))
     			     (set! *f2* (+ 1 *f2*))
     			     (thread-yield!)
     			     (loop))))))

     	 (f2b (instantiate::fthread
     		 (body (lambda ()
     			  (let loop ()
     			     (print "f2b: " *f2* " " (current-thread))
     			     (set! *f2* (+ 1 *f2*))
     			     (thread-yield!)
     			     (loop)))))))

           (let* ((p1 (instantiate::pthread
     		   (body (lambda ()
     			    ;; Sets the thread's specific scheduler
     			    (default-scheduler s1)
     			    (scheduler-start! 5)))))

     	    (p2 (instantiate::pthread
     		   (body (lambda ()
     			    ;; Sets the thread's specific scheduler
     			    (default-scheduler s2)
     			    ;; One reaction for s2
     			    (scheduler-react!)
     			    ;; Starts s1
     			    (thread-start-joinable! p1)
     			    ;; Do three reactions
     			    (scheduler-start! 3)
     			    ;; Waits for p1/s1 termination
     			    (thread-join! p1)
     			    ;; The final reaction
     			    (scheduler-react!))))))

     	 (thread-start! f1a s1)
     	 (thread-start! f1b s1)
     	 (thread-start! f2a s2)
     	 (thread-start! f2b s2)

     	 (thread-join! (thread-start-joinable! p2)))))


File: bigloo.info,  Node: Database,  Next: Multimedia,  Prev: Threads,  Up: Top

17 Database
***********

Bigloo supports database programming.  The current version proposes a
SQLite binding.

* Menu:

* SQLite::


File: bigloo.info,  Node: SQLite,  Up: Database

17.1 SQLite
===========

The Bigloo's C back-end supports SQL queries.  It relies on the SQLite
library (<http://www.sqlite.org/>).  The SQLite binding is accessible to
Bigloo via the 'sqlite' library.  Here is an example of module that uses
this library.

     (module example1
       (library sqlite))

     (let ((db (instantiate::sqlite)))
       ...)

 -- bigloo sqlite class: sqlite
          (class sqlite
             (path::bstring read-only (default ":memory:")))

     The instances of the class 'sqlite' hold SQLite databases.  A
     database may be permanently stored on a disk or loaded in memory.
     The class attribute 'path' is the location on the disk where the
     database is stored.  The special path ':memory:' denotes in-memory
     databases.  When an instance is created a SQLite database is
     _opened_.

     Example:
          (define db1 (instantiate::sqlite (path "/tmp/foo.db")))
          (define db2 (instantiate::sqlite))

     Binds the global variable DB1 to a database that is stored on the
     file system at location '/tmp/foo.db'.  This example also binds the
     global variable DB2 to an in-memory SQLite database.

 -- bigloo sqlite function: sqlite-close SQLITE
     This function closes a database previously opened by creating an
     instance of the class 'sqlite'.

     Example:
          (let ((db (instantiate::sqlite)))
            (sqlite-exec db "CREATE TABLE table1 (x INTEGER, y INTEGER)")
            (sqlite-exec db "INSERT INTO table1 VALUES( ~a, ~a )" 1 4)
            (sqlite-close db))

 -- bigloo sqlite function: sqlite-format STRING ARG ...
     Constructs a string of characters representing an SQLite commands.
     This function acts as 'format' (see *note Input and Output::).  It
     is augmented with three additional escape sequence: '~q', '~k', and
     '~l'.  The first one build a string of characters where the
     characters denoting SQL strings (i.e., the character ''') is
     automatically escaped.  The escape character '~k' introduces a list
     of SQL strings.  The escape character '~l' introduces a SQL list.

     Summary of all escape codes:
        * '~a' The corresponding value is inserted into the string as if
          printed with display.
        * '~s' The corresponding value is inserted into the string as if
          printed with write.
        * '~%' A newline is inserted.
        * '~~' A tilde '~' is inserted.
        * '~q' An SQL escaped string.
        * '~l' Introduces a list (comma separated).
        * '~k' Introduces a list of SQL strings.

     Examples:

          (module example
             (library sqlite))

          (sqlite-format "~a" "foo'bar") => "foo'bar"
          (sqlite-format "~q" "foo'bar") => "'foo''bar'"
          (sqlite-format "~a" '("foo'bar" "foo")) => "(foo'bar foo)"
          (sqlite-format "~k" '("foo'bar" "foo")) => "'foo''bar','foo'"
          (sqlite-format "~l" '("foo'bar" "foo")) => "foo'bar,foo"

 -- bigloo sqlite function: sqlite-exec SQLITE STRING ARG ...
     The function 'sqlite-exec' _executes_ an SQLite command.  The
     command is the built by implicitly invoking 'sqlite-format' on
     STRING and the optional ARG arguments.  This function returns a
     single element, the first one returned by the SQL engine.

     Example:
          (module example
             (library sqlite))

          (define *db* (instantiate::sqlite))

          (sqlite-exec *db* "CREATE TABLE foo (x INTEGER, y INTEGER)")
          (for-each (lambda (x)
          		(sqlite-exec *db*  "INSERT INTO foo VALUES(~A, ~A)" x (* x x)))
          	     (iota 10))
          (sqlite-exec *db* "SELECT * FROM foo")
             => 9

 -- bigloo sqlite function: sqlite-eval SQLITE PROCEDURE STRING ARG ...
     The function 'sqlite-eval' invokes a SQLite command built by
     implicitly invoking 'sqlite-format' on STRING and the optional ARG
     arguments.  The result of the function is built by applying
     PROCEDURE to the first value returned by the SQLite call.

     Note: user callback (PROCEDURE) must not exit.  That is they must
     not invoke a function create by 'bind-exit'.  Exiting from a
     callback will leave the database in a inconsistent state that
     prevent transactions to be rolled back.

 -- bigloo sqlite function: sqlite-map SQLITE PROCEDURE STRING ARG ...
     The function 'sqlite-map' invokes a SQLite command built by
     implicitly invoking 'sqlite-format' on STRING and the optional ARG
     arguments.  The result is a list whose elements are built by
     applying PROCEDURE to all the values returned by the SQLite call.

     Note: user callback (PROCEDURE) must not exit.  That is they must
     not invoke a function create by 'bind-exit'.  Exiting from a
     callback will leave the database in a inconsistent state that
     prevent transactions to be rolled back.  Example:

          (module example
             (library sqlite))

          (define *db* (instantiate::sqlite))

          (sqlite-exec *db* "CREATE TABLE foo (x INTEGER, y INTEGER)")
          (for-each (lambda (x)
          		(sqlite-exec *db*  "INSERT INTO foo VALUES(~A, ~A)" x (* x x)))
          	     (iota 10))
          (sqlite-map *db*
            (lambda (s1 s2) (+ (string->integer s1) (string->integer s2)))
            "SELECT * FROM foo")
             => (0 2 6 12 20 30 42 56 72 90)

  Example2:
     (module example
        (library sqlite))

     (define *db* (instantiate::sqlite))

     (sqlite-exec *db* "CREATE TABLE foo (x INTEGER, y INTEGER)")
     (for-each (lambda (x)
     		(sqlite-exec *db*  "INSERT INTO foo VALUES(~A, ~A)" x (* x x)))
     	     (iota 10))
     (sqlite-map *db* vector "SELECT * FROM foo")
        => '(#("0" "0")
     	#("1" "1")
     	#("2" "4")
     	#("3" "9")
     	#("4" "16")
     	#("5" "25")
     	#("6" "36")
     	#("7" "49")
     	#("8" "64")
     	#("9" "81"))

 -- bigloo sqlite function: sqlite-name-of-tables SQLITE
     Returns the name of tables in the database.  This list can also be
     obtained with

          (sqlite-map db
             (lambda (x) x)
             "SELECT name FROM sqlite_master WHERE type='table'")

 -- bigloo sqlite function: sqlite-table-name-of-columns SQLITE TABLE
     Returns the name of columns in the table.

 -- bigloo sqlite function: sqlite-last-insert-rowid SQLITE
     Returns the SQLite _rowid_ of the last inserted row.


File: bigloo.info,  Node: Multimedia,  Next: Mail,  Prev: Database,  Up: Top

18 Multimedia
*************

Bigloo provides various facilities for programming multimedia
applications.  It provides functions for parsing images and sounds and
functions for controlling music players.  All the functions, variables,
and classes presented in the document are accessible via the
'multimedia' library.  Here is an example of module that uses this
library:

     ;; Extract the thumbnail of a digital photography.
     (module thumbnail
        (library multimedia)
        (main main))

     (define (main argv)
        (when (and (pair? (cdr argv)) (file-exists? (cadr argv)))
           (let ((ex (jpeg-exif (cadr argv))))
               (when (exif? ex)
                  (display (exif-thumbnail ex))))))

* Menu:

* Photography::
* Music::
* Color::


File: bigloo.info,  Node: Photography,  Next: Music,  Up: Multimedia

18.1 Photography
================

The multimedia library provides functions for accessing the metadata
generated by digital camera.

 -- bigloo multimedia procedure: jpeg-exif file-name
     The function 'jpeg-exif' extracts the EXIF
     (<http://en.wikipedia.org/wiki/Exif>) metadata of a JPEG file as
     created by digital camera.  The argument FILE-NAME is the name of
     the JPEG file.  If the file contains an EXIF section it is returned
     as an instance of the 'exif' class.  Otherwise, this function
     returns '#f'.

 -- bigloo multimedia procedure: jpeg-exif-comment-set! file-name text

     Set the comment of the EXIF metadata section of the file file-name
     to TEXT.

 -- bigloo multimedia class: exif
          (class exif
             (version (default #f))
             (jpeg-encoding (default #f))
             (jpeg-compress (default #f))
             (comment (default #f))
             (commentpos (default #f))
             (commentlen (default #f))
             (date (default #f))
             (make (default #f))
             (model (default #f))
             (orientation (default 'landscape))
             (width (default #f))
             (height (default #f))
             (ewidth (default #f))
             (eheight (default #f))
             (xresolution (default #f))
             (yresolution (default #f))
             (resolution-unit (default #f))
             (focal-length (default #f))
             (flash (default #f))
             (fnumber (default #f))
             (iso (default #f))
             (shutter-speed-value (default #f))
             (exposure-time (default #f))
             (exposure-bias-value (default #f))
             (aperture (default #f))
             (metering-mode (default #f))
             (cdd-width (default #f))
             (focal-plane-xres (default #f))
             (focal-plane-units (default #f))
             (thumbnail (default #f))
             (thumbnail-path (default #f))
             (thumbnail-offset (default #f))
             (thumbnail-length (default #f)))

     The instance of the 'exif' class maps the EXIF metadata found in
     JPEG files into Bigloo objects.  Since all fields are optional they
     are untyped.

 -- bigloo multimedia procedure: exif-date->date
     Parses an exif date, i.e., a string of characters, and returns
     corresponding date.  Raises an &IO-PARSE-ERROR if the string does
     not represents an exif date whose syntax is given by the following
     regular expression:

            [0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9] :[0-9][0-9]:[0-9][0-9]:[0-9][0-9]


File: bigloo.info,  Node: Music,  Next: Color,  Prev: Photography,  Up: Multimedia

18.2 Music
==========

The multimedia library provides an extensive set of functions for
dealing with music.  It provides functions for accessing the metadata of
certain music file formats, it provides functions for controlling the
volume of the hardware mixers and it provides functions for playing and
controlling music playback.

* Menu:

* Metadata and Playlist::
* Mixer::
* Playback::
* MPD::


File: bigloo.info,  Node: Metadata and Playlist,  Next: Mixer,  Up: Music

18.2.1 Metadata and Playlist
----------------------------

 -- bigloo multimedia procedure: read-m3u input-port
 -- bigloo multimedia procedure: write-m3u list output-port

     The function 'read-m3u' reads a playlist expressed in the M3U
     format from INPUT-PORT and returns a list of songs.  The function
     'write-m3u' encode such a list encoded in the M3U format to an
     output port.

 -- bigloo multimedia procedure: file-musictag file-name
 -- bigloo multimedia procedure: mp3-musictag file-name
 -- bigloo multimedia procedure: ogg-musictag file-name
 -- bigloo multimedia procedure: flac-musictag file-name
     These functions extract the metadata of a music file named
     FILE-NAME.

     The function 'mp3-musictag' returns the ID3 tag section if it
     exists.  Otherwise, it returns '#f'.  The function 'ogg-musictag'
     and 'flac-musictag' returns the vorbis comment if it exists.

 -- bigloo multimedia class: musictag
          (abstract-class musictag
             (title::bstring read-only)
             (artist::bstring read-only)
             (orchestra::obj read-only (default #f))
             (interpret::obj read-only (default #f))
             (album::bstring read-only)
             (year::int read-only)
             (comment::bstring read-only)
             (genre::bstring read-only)
             (track::int (default -1)))

     This class is used as the base class of music tag formats.

 -- bigloo multimedia class: id3::musictag
          (class id3::musictag
             version::bstring
             (orchestra::obj read-only (default #f))
             (conductor::obj read-only (default #f))
             (recording read-only (default #f))
             (cd::obj (default #f)))

     This class is used to reify the ID3 metadata used in the MP3
     format.

 -- bigloo multimedia class: vorbis::musictag
          (class vorbis::musictag)

     This class is used to reify the Vorbis comments of OGG and Flac
     files.


File: bigloo.info,  Node: Mixer,  Next: Playback,  Prev: Metadata and Playlist,  Up: Music

18.2.2 Mixer
------------

Bigloo proposes various functions and classes for controlling the audio
volume of sound cards.

 -- bigloo multimedia class: mixer
          (class mixer
             (devices::pair-nil (default '())))

     The field DEVICES is a list of available channels.

 -- bigloo multimedia procedure: mixer-close mix
     Closes a mixer.  The argument MIX must be an instance of the
     'mixer' class.

 -- bigloo multimedia procedure: mixer-volume-get mix channel
 -- bigloo multimedia procedure: mixer-volume-set! mix channel leftv
          rightv
     The function MIXER-VOLUME-GET returns the left and right volume
     levels (two values) of the CHANNEL of the mixer MIX.  The CHANNEL
     is denoted by its name and is represented as a string of
     characters.  The argument MIX is an instance of the 'mixer' class.

     The function MIXER-VOLUME-SET! changes the audio level of a mixer
     channel.

 -- bigloo multimedia class: soundcard::mixer
          (class soundcard::mixer
             (device::bstring read-only))

     The instances of the class 'soundcard', a subclass of the 'mixer'
     class, are used to access physical soundcard as supported by
     operating systems.  The class field DEVICE stands for the name of
     the system device (e.g., '"/dev/mixer"' for the Linux OS). During
     the initialization of the instance, the device is opened and
     initialized.


File: bigloo.info,  Node: Playback,  Next: MPD,  Prev: Mixer,  Up: Music

18.2.3 Playback
---------------

Bigloo supports various functions for playing music.  These functions
rely on two data structure: _music players_ and _music status_.  The
first ones are used to control player back-ends.  The second ones are
used to get information about the music being played.  The following
example shows how a simple music player using either MPlayer, MPG123, or
MPC can be programmed with Bigloo.

     (module musicplay
        (library multimedia)
        (main main))

     (define (main args)
        (let ((files '())
     	 (backend 'mplayer)
     	 (command #f))
           (args-parse (cdr args)
     	 (("--mpg123" (help "Select the mpg123 back-end"))
     	  (set! backend 'mpg123))
     	 (("--mpc" (help "Select the mpc back-end"))
     	  (set! backend 'mpc))
     	 (("--mplayer" (help "Select the mplayer back-end"))
     	  (set! backend 'mplayer))
     	 (("--command" ?cmd (help "Set the command path"))
     	  (set! command cmd))
     	 (("--help" (help "This help"))
     	  (print "usage: music [options] file ...")
     	  (args-parse-usage #f)
     	  (exit 0))
     	 (else
     	  (set! files (cons else files))))
           ;; create a music player
           (let ((player (case backend
     		       ((mpg123)
     			(if command
     			    (instantiate::mpg123
     			       (path command))
     			    (instantiate::mpg123)))
     		       ((mplayer)
     			(if command
     			    (instantiate::mplayer
     			       (path command))
     			    (instantiate::mplayer)))
     		       ((mpc)
     			(instantiate::mpc)))))
              ;; fill the music play list
     	 (for-each (lambda (p) (music-playlist-add! player p)) (reverse files))
              ;; start playing
     	 (music-play player)
              ;; run an event loop with call-backs associated to some events
     	 (music-event-loop player
                 :onstate (lambda (status)
                             (with-access::musicstatus status (state song volume)
                                (print "state   : " state)
                                (print "song    : " song)))
     	    :onmeta (lambda (meta)
     		       (print "meta    : " meta))
     	    :onvolume (lambda (volume)
     		       (print "volume  : " volume))))))

 -- bigloo multimedia abstract class: music
          (abstract-class music
             (frequency::long (default 2000000))

     This abstract class is the root class of all music players.

 -- bigloo multimedia class: musicproc::music
          (class musicproc::music
             (charset::symbol (default 'ISO-LATIN-1)))

     This class is used to reify player that are run in an external
     process.

 -- bigloo multimedia class: mplayer::musicproc
          (class mplayer::musicproc
             (path::bstring read-only (default "mplayer"))
             (args::pair-nil read-only (default '("-vo" "null" "-quiet" "-slave" "-idle")))
             (ao::obj read-only (default #unspecified))
             (ac::obj read-only (default #unspecified)))

     A player based on the external software 'MPlayer'.  Creating such a
     player spawns in background a MPlayer process.

 -- bigloo multimedia class: mpg123::musicproc
          (class mpg123::musicproc
             (path::bstring read-only (default "mpg123"))
             (args::pair-nil read-only (default '("--remote"))))

     A player based on the external software 'mpg123'.

 -- bigloo multimedia class: mpc::music
          (class mpc::music
             (hello read-only (default #f))
             (host read-only (default "localhost"))
             (port read-only (default 6600))
             (timeout read-only (default 10008993))
             (prefix (default #f)))

     A MPC client.

        * 'hello': an optional string written when the connection is
          establish with the MPD server.
        * 'prefix': an optional path prefix to be removed from music
          playlist.  This is needed because MPD can only play music
          files registered in is private database.  The file names used
          by MPD are relative a root directory used to fill the
          database.  The 'prefix' field allows programmer to write
          portable code that manages play list file names independently
          of the player selected.

 -- bigloo multimedia class: musicstatus
          (class musicstatus
             (state::symbol (default 'stop))
             (volume::obj (default -1))
             (repeat::bool (default #f))
             (random::bool (default #f))
             (playlistid::int (default -1))
             (playlistlength::int (default 0))
             (xfade::int (default 0))
             (song::int (default 0))
             (songid::int (default 0))
             (songpos (default 0))
             (songlength::int (default 0))
             (bitrate::int (default 0))
             (khz::int (default 0))
             (err::obj (default #f)))

     The instances of the class 'musicstatus' denote that state of a
     player.

 -- bigloo multimedia procedure: music-close music
 -- bigloo multimedia procedure: music-reset! music
 -- bigloo multimedia procedure: music-closed? music
     Closes, resets, and tests the state of a music player.

 -- bigloo multimedia procedure: music-playlist-get music
 -- bigloo multimedia procedure: music-playlist-add! music song
 -- bigloo multimedia procedure: music-playlist-delete! music int
 -- bigloo multimedia procedure: music-playlist-clear! music
     These functions controls the playlist used by a player.

     Note: The SONG argument is an UTF8 encoded string (see Section
     *note Unicode (UCS-2) Strings::) _whatever_ the local file system
     encoding is.  The function 'music-playlist-get' returns a list of
     UTF8 encoded names.

        * 'music-playlist-get': returns the list of songs (UTF8 names)
          of the current playlist.
        * 'music-playlist-add!': adds an extra song (UTF8 name) at the
          end of the playlist.
        * 'music-delete!': removes the song number INT from the
          playlist.
        * 'music-clear!': erases the whole playlist.

 -- bigloo multimedia procedure: music-play music [song]
 -- bigloo multimedia procedure: music-seek music time [song]
 -- bigloo multimedia procedure: music-stop music
 -- bigloo multimedia procedure: music-pause music
 -- bigloo multimedia procedure: music-next music
 -- bigloo multimedia procedure: music-prev music
     These functions changes the state of the music player.  The
     function 'music-seek' seeks the playback position to the position
     TIME, which is an integer denoting a number of seconds.

 -- bigloo multimedia procedure: music-crossfade music int
 -- bigloo multimedia procedure: music-random-set! music bool
 -- bigloo multimedia procedure: music-repeat-set! music bool
     These functions controls how songs playback should follow each
     other.

 -- bigloo multimedia procedure: music-volume-get music
 -- bigloo multimedia procedure: music-volume-set! music vol
     Get and set the audio volume of a player.  Some player use the
     native mixer supported by the operating system some others use a
     software mixer unrelated to the hardware.

 -- bigloo multimedia procedure: music-status music
 -- bigloo multimedia procedure: music-update-status! music status
     The function 'music-status' returns an instance of the
     'musicstatus' class which denotes the state of the player.  The
     function 'music-update-status!' updates this status.

 -- bigloo multimedia procedure: music-song music
 -- bigloo multimedia procedure: music-songpos music
     These two functions return the number of the song being played and
     the position in the song.  These functions are somehow redundant
     with the function 'music-status' because the status also contains
     information about the playback song and playback position.
     However, for some players getting the music song and the playback
     position is cheaper than getting the whole player status.

 -- bigloo multimedia procedure: music-meta music
     Returns the metadata the current song.

 -- bigloo multimedia procedure: music-reset-error! music
     Reset the previous errors detected by a player.

 -- bigloo multimedia procedure: music-event-loop music [:onstate]
          [:onmeta] [:onerror] [:onvolume]

     The function 'music-event-loop' enable event notifications when the
     state of a player changes.  The keyword arguments are:

        * ':onstate', a function of one parameter.  When the player
          state changes, this function is called with an instance of
          'musicstatus' as first actual parameter.
        * ':onmeta', a function of two parameters.  This function is
          called when a metadata is detected in the music currently
          played.
        * ':onerror', a function of one parameter, invoked when an error
          is detected.
        * ':onvolume', a function of one parameter, invoked when the
          volume changes.


File: bigloo.info,  Node: MPD,  Prev: Playback,  Up: Music

18.2.4 Music Player Daemon
--------------------------

Music Player Daemon (MPD in short) allows remote access for playing
music <http://www.musicpd.org>.  MPD is designed for integrating a
computer into a stereo system that provides control for music playback
over a local network.  The Bigloo class 'mpc' implements a 'mpd' client.
All Bigloo players can be access via the MPD protocol, using the

  The following example shows how to access a MPlayer music player using
the MPD protocol with a simple Bigloo program:

     (module mpd
        (library multimedia pthread)
        (main main))

     (define (main argv)
        (let ((db (instantiate::mpd-database
     		(directories (cdr argv))))
     	 (serv (make-server-socket 6600))
     	 (music (instantiate::mplayer)))
           (let loop ()
     	 (thread-start! (make-mpd-connection-thread music db sock))
     	 (loop))))

     (define (make-mpd-connection-thread music db sock)
        (instantiate::pthread
           (body (lambda ()
     	       (let ((pi (socket-input sock))
     		     (po (socket-output sock)))
     		  (input-timeout-set! pi 10000)
     		  (output-timeout-set! po 10000)
     		  (unwind-protect
     		     (mpd music pi po db)
     		     (socket-close sock)))))))

 -- bigloo multimedia procedure: mpd music input-port output-port
          database [:log]
     The function 'mpd' implements a MPD server.  It reads commands from
     the INPUT-PORT and write results to OUTPUT-PORT.  The argument
     DATABASE, an instance of the 'mpd-database' class, describes the
     music material that can be delivered by this player.

 -- bigloo multimedia class: mpd-database
          (class mpd-database
             (directories::pair-nil read-only)

     The field 'directories' contains the list of the directories that
     contains music files.


File: bigloo.info,  Node: Color,  Prev: Music,  Up: Multimedia

18.3 Color
==========

The multimedia library provides functions for dealing with colors.

 -- bigloo multimedia procedure: hsv->rgb h s v
 -- bigloo multimedia procedure: hsl->rgb h s l
 -- bigloo multimedia procedure: rgb-hsv r g b
 -- bigloo multimedia procedure: rgb-hsl r g b
     These procedures converts from and to HSV, HSL, and RGB
     representations.  The argument H is an integer in the range
     '[0..360]', the arguments S, V, and L in the range '[0..100]'.  The
     arguments R, G, and B are in the range '[0..255]'.  These
     procedures returns multiple-values.

          (multiple-value-bind (r g b)
             (hsv->rgb 340 34 56)
             (list r g b)) => (143 94 110)
          (multiple-value-bind (h s v)
             (rgb->hsv 255 0 0)
             (list h s v)) => (0 100 100)


File: bigloo.info,  Node: Mail,  Next: Text,  Prev: Multimedia,  Up: Top

19 Mail
*******

Bigloo provides various facilities for handling mails.  It provides
facilities for parsing many formats commonly used in composing mails
(quoted printable, vcard, mime types).  It also provides facilities for
dealing with mail servers.  For that it proposes an abstracted view of
mail servers with two implementations: 'imap' and 'maildir'.

* Menu:

* RFC 2045:: MIME Part one.
* RFC 2047:: MIME, Part three.
* RFC 2426:: vcard.
* RFC 2822:: Internet Message Format
* Mail servers:: 'imap' and 'maildir'


File: bigloo.info,  Node: RFC 2045,  Next: RFC 2047,  Up: Mail

19.1 RFC 2045 - MIME, Part one
==============================

This section described the functions offered by Bigloo to encode and
decode some of the formats specified in the RFC 2045
<http://tools.ietf.org/html/rfc2045>.

 -- bigloo mail procedure: quoted-printable-encode string
 -- bigloo mail procedure: quoted-printable-decode string
     These functions encode/decode a string into and from the
     'quoted-printable' format.

     Examples:

          (quoted-printable-encode "foo bar") => "foobar=20"
          (quoted-printable-decode "foobar=20") => "foo bar"

 -- bigloo mail procedure: quoted-printable-encode-port ip op
 -- bigloo mail procedure: quoted-printable-decode-port ip op [rfc2047]
     These functions are similar to 'quoted-printable-encode' and
     'quoted-printable-decode' except that they operate on 'input-port's
     and 'output-port's.

     The function 'quoted-printable-decode-port' accepts an optional
     argument: RFC2047.  If this argument is '#t', then the parsing
     stops on the prefix '?=', which is a marker in the mail subject as
     specified by the RFC 2047, (see
     <http://tools.ietf.org/html/rfc2047>) is found.

 -- bigloo mail procedure: mime-content-decode string
 -- bigloo mail procedure: mime-content-decode-port input-port
     These two functions parse respectively a 'string' and an
     'input-port' and return a list of three elements:

        * a content type,
        * a content subtype,
        * options.

     Example:

          (mime-content-type-decode "text/plain; boundary=Apple-Mail-11")
            => (text plain ((boundary . Apple-Mail-11)))

 -- bigloo mail procedure: mime-content-disposition-decode string
 -- bigloo mail procedure: mime-content-disposition-decode-port
          input-port
     These two functions parse respectively a 'string' and an
     'input-port' and return a list describing the content disposition.

     Example:

          (mime-content-disposition-decode "attachment; filename=\"smine.p7s\"")
            => (attachment ((filename . smine.p7s)))

 -- bigloo mail procedure: mime-multipart-decode string boundary
          [recursive]
 -- bigloo mail procedure: mime-multipart-decode-port input-port
          boundary [recursive]
     These two functions parse respectively a 'string' and an
     'input-port' and return a list of mime sections.

     If the optional argument RECURSIVE controls whether subparts of a
     multipart section must be decoded are not.  If the RECURSIVE is
     '#t' then all subparts of the multipart content are decoded.  The
     result is a fully decoded multipart section.  If RECURSIVE is '#f'
     subparts are not decoded and included in the result as plain
     strings.


File: bigloo.info,  Node: RFC 2047,  Next: RFC 2426,  Prev: RFC 2045,  Up: Mail

19.2 RFC 2047 - MIME, Part three
================================

This section described the function offered by Bigloo to decode the RFC
2047 encoding used in mail headers (see
<http://tools.ietf.org/html/rfc2047>).

 -- bigloo mail procedure: rfc2047-decode-port ip op [:charset
          iso-latin-1]
 -- bigloo mail procedure: rfc2047-decode string [:charset iso-latin-1]

     These functions decode mail header fields encoded using the RFC
     2047 specification.  The optional argument CHARSET specified in
     which charset the result should be encoded.  The allowed values
     are:

        * 'utf-8'
        * 'iso-latin-1'
        * 'cp-1252'

     Example:

          (map char->integer
            (string->list (rfc2047-decode "Poste =?ISO-8859-1?Q?t=E9l=E9phonique?=")))
            => (80 111 115 116 101 32 116 233 108 233 112 104 111 110 105 113 117 101)
          (string-for-read (rfc2047-decode "Poste =?ISO-8859-1?Q?t=E9l=E9phonique?=" :charset 'utf8))
            => "Poste t\303\251l\303\251phonique"


File: bigloo.info,  Node: RFC 2426,  Next: RFC 2822,  Prev: RFC 2047,  Up: Mail

19.3 RFC 2426 - MIME, Part three
================================

This section presents the facilities supported by Bigloo for dealing
with 'vcard's.

 -- bigloo mail class: vcard
          (class vcard
            (version::bstring (default "2.1"))
            (uid (default #f))
            (fn (default #f))
            (familyname (default #f))
            (firstname (default #f))
            (nickname (default #f))
            (photo (default #f))
            (sound (default #f))
            (url (default #f))
            (org (default #f))
            (emails::pair-nil (default '()))
            (phones::pair-nil (default '()))
            (birthday (default #f))
            (addresses::pair-nil (default '()))
            (lang (default #f))
            (related (default #f))
            (key (default #f))
            (notes::pair-nil (default '()))
            (x-thumbnail (default #f))
            (x-color (default #f))
            (xx-extras::pair-nil (default '())))

     The class 'vard' is used to reify in memory a vcard as parsed by
     the functions 'port->vcard', 'read-vcard', and 'string->vcard'.

     Except 'emails', 'phones', and 'addresses', all fields are
     optional.  They should be either '#f' or a string.  Field meanings
     are given by the RFC 2426 specification, apart the 'x-thumbnail',
     'x-color', and 'xx-extras' that are Bigloo specific.

        * 'photo' is a flat list of strings.
        * 'x-thumbnail' an optional thumbnail image.
        * 'x-color' an optional color.
        * 'xx-extras' a list of non standard properties.
        * 'phones' is an alist whose elements are pairs of two strings.
        * 'addresses' is a list composed of:
             * the postoffice, a string,
             * a list of strings denoting the street address,
             * a string denoting the city,
             * a string denoting the region,
             * a string denoting the zip code,
             * a string denoting the zip country.

          All street values are required and must be provided.  The
          empty string should be used to denote empty values.

 -- bigloo mail function: read-vcard::vcard ip [:charset-encoder]
 -- bigloo mail function: port->vcard::vcard ip [:charset-encoder]
 -- bigloo mail function: string->vcard::vcard str [:charset-encoder]
     These three functions parse a _vcard_ to produce a 'vcard'
     instance.  The optional argument CHARSET-ENCODER, when provided,
     must be a function of argument: a string to be decoded.  Vcard
     strings are UTF-8 encoded.  The CHARSET-ENCODER can be used to
     encode on-the-fly the strings found in the vcard in a difference
     encoding.

     'port->vcard' is similar to 'read-vcard' but it returns '#f' on
     end-of-file, while 'read-vcard' returns the eof-object.


File: bigloo.info,  Node: RFC 2822,  Next: Mail servers,  Prev: RFC 2426,  Up: Mail

19.4 RFC 2822 - Internet Message Format
=======================================

This section described the functions offered by Bigloo to encode and
decode some of the formats specified in the RFC 2822
(<http://tools.ietf.org/html/rfc2045>).  It mainly supports functions
for parsing email headers and for decoding email addresses.

 -- bigloo mail procedure: mail-header->list obj
     The function 'mail-header->list' parses a mail header that can
     either be implemented as a string or an input port.  It returns a
     list of fields.

     Example:

          (mail-header->list "Return-Path: <foo.bar@inria.fr>
          Received: from eurus.inria.fr ([unix socket])")
            =>
            ((return-path . "<foo.bar@inria.fr>") (received . "from eurus.inria.fr ([unix socket])"))

 -- bigloo mail procedure: email-normalize string

     The function 'email-normalize' extracts the actual email address
     from an email representation.

     Example:
          (email-normalize "foo bar <foo.bar@inria.fr>") => "foo.bar@inria.fr"

 -- bigloo mail procedure: rfc2822-address-display-name string
     Extract the name component of an email.

     Example:
          (rfc2822-address-display-name "Foo Bar <foo.bar@inria.fr>") => "Foo Bar"
          (rfc2822-address-display-name "<foo.bar@inria.fr>") => "foo bar"


File: bigloo.info,  Node: Mail servers,  Prev: RFC 2822,  Up: Mail

19.5 Mail servers - imap and maildir
====================================

Bigloo implements the 'imap' protocol
(<http://tools.ietf.org/html/rfc3501>) and the 'maildir' format.  This
section presents the API for manipulating them both.

* Menu:

* mailbox:: main API for dealing with messages and folders.
* imap:: implementation of the RFC 3501 protocol.
* maildir:: implementation of the mail file system format.


File: bigloo.info,  Node: mailbox,  Prev: imap,  Up: Mail servers

19.5.1 Mailboxes
----------------

 -- bigloo mail class: mailbox
          (abstract-class mailbox
            (label::bstring (default "")))

     The abstract class 'mailbox' is the common ancestors to all the
     mailbox implementations.  It allows the definitions of various
     generic functions that deal with mail messages and mail folders.

 -- bigloo mail class: &mailbox-error
          (abstract-class &mailbox-error::&error)

     The '&mailbox-error' is the super class of all the errors that can
     be raised when accessing mail servers, except the parsing errors
     that inherit from the '&parse-error' super class.

 -- bigloo mail procedure: mailbox-close mailbox
     Close the mailbox connection.

     Example:
          (let ((mbox (if (network-up?)
                          (instantiate::imap (socket ...))
                          (instantiate::maildir (path my-local-cache)))))
             (mailbox-close mbox))

 -- bigloo mail procedure: mailbox-separator mailbox
     Returns a string denoting the separator (commonly '"' or '.') used
     by the MAILBOX.

 -- bigloo mail procedure: mailbox-prefix mailbox
     Returns the prefix of the MAILBOX, a string or '#f'.

 -- bigloo mail procedure: mailbox-hostname mailbox
     Returns the hostname of the MAILBOX, a string or '#f'.

 -- bigloo mail procedure: mailbox-folders mailbox
     Returns a list of strings denoting the folder names of the MAILBOX.

 -- bigloo mail procedure: mailbox-folder-select! mailbox string
     Selects one folder of the MAILBOX.  This function is central to
     mailboxes because all messages are referenced relatively to the
     folder selection.  All the functions that operates on 'uid'
     implicitly access the current folder selection.

 -- bigloo mail procedure: mailbox-folder-unselect! mailbox
     Unselects the MAILBOX current selected folder.

 -- bigloo mail procedure: mailbox-folder-create! mailbox folder
     Creates a new FOLDER denotes by a fully qualified name.

     Example
          (mailbox-create! mbox "INBOX.scheme.bigloo")

 -- bigloo mail procedure: mailbox-folder-delete! mailbox folder
     Deletes an empty FOLDER.

 -- bigloo mail procedure: mailbox-folder-rename! mailbox old new
     Renames a folder.

 -- bigloo mail procedure: mailbox-folder-move! mailbox folder dest
     Moves the FOLDER into the destination folder DEST.

 -- bigloo mail procedure: mailbox-subscribe! mailbox folder
 -- bigloo mail procedure: mailbox-unsubscribe! mailbox folder
     Subscribe/unsubscribe to a folder.  This allows 'imap' servers not
     to present the entire list of folders.  Only subscribed folders are
     returned by 'mailbox-folders'.  These functions have no effect on
     'maildir' servers.

 -- bigloo mail procedure: mailbox-folder-exists? mailbox folder
     Returns '#t' if and only if FOLDER exists in MAILBOX.  Returns '#f'
     otherwise.

 -- bigloo mail procedure: mailbox-folder-status mailbox folder
     Returns the status of the FOLDER.  A status is an alist made of the
     number of unseen mail, the uid validity information, the uid next
     value, the number of recent messages, and the overall number of
     messages.

 -- bigloo mail procedure: mailbox-folder-uids mailbox
     Returns the list of UIDs (a list of integers) of the messages
     contained in the currently selected folder.

 -- bigloo mail procedure: mailbox-folder-dates mailbox
     Returns the list of dates of the messages contained in the
     currently selected folder.

 -- bigloo mail procedure: mailbox-folder-delete-messages! mailbox
     Deletes the messages marked as _deleted_ of the currently selected
     folder.

 -- bigloo mail procedure: mailbox-folder-header-fields mailbox field
     Returns the list of headers FIELDS of the message of the current
     folder.

 -- bigloo mail procedure: mailbox-message mailbox uid
     Returns the message UID in the current folder.

 -- bigloo mail procedure: mailbox-message-path mailbox uid
     Returns the full path name of the message UID.

 -- bigloo mail procedure: mailbox-message-body mailbox uid [len]
     Returns the body of the message UID.  If LEN is provided, only
     returns the first LEN characters of the body.

 -- bigloo mail procedure: mailbox-message-header mailbox uid
     Returns the header as a string of the message UID.

 -- bigloo mail procedure: mailbox-message-header-list mailbox uid
     Returns the header as an alist of the message UID.

 -- bigloo mail procedure: mailbox-message-header-field mailbox uid
          field
     Extracts one field from the message header.

 -- bigloo mail procedure: mailbox-message-size mailbox uid
     Returns the size of the message.

 -- bigloo mail procedure: mailbox-message-info mailbox uid
     Returns the information relative to the message UID.  This a list
     containing the message identifier, its uid, the message date, the
     message size, and the message flags.

 -- bigloo mail procedure: mailbox-message-flags mailbox uid
 -- bigloo mail procedure: mailbox-message-flags-set! mailbox uid lst
     Sets/Gets the flags of the message UID.  This is a list of strings.
     Typical flags are:

        * '\Flagged'
        * '\Answered'
        * '\Deleted'
        * '\Seen'

 -- bigloo mail procedure: mailbox-message-delete! mailbox uid
     Deletes the message UID.

 -- bigloo mail procedure: mailbox-message-move! mailbox uid folder
     Moves the message UID into the new FOLDER (denoted by a string).

 -- bigloo mail procedure: mailbox-message-create! mailbox folder
          content
     Creates a new message in the FOLDER whose content is given the
     string CONTENT.


File: bigloo.info,  Node: imap,  Next: mailbox,  Prev: maildir,  Up: Mail servers

19.5.2 IMAP (RFC 3501)
----------------------

 -- bigloo mail class: imap
          (class imap::mailbox
            (socket::socket read-only))

          (define mbox
            (instantiate::maildir
              (label "My Remote Mailbox")
              (socket (imap-login (make-client-socket "imap.inria.fr" 993)
                                  "serrano" "XXX"))))

 -- bigloo mail class: &imap-parse-error
          (class &imap-parse-error::&io-parse-error)

 -- bigloo mail class: &imap-error
          (class &imap-error::&mailbox-error)

 -- bigloo mail procedure: imap-login socket user password
     Log a user into an imap server.  The SOCKET must have been created
     first.  The argument USER is a string and denotes the user name.
     The argument PASSWORD is a string too and it contains the user
     password.  This function returns as value the SOCKET it has
     received.  If the operation fails the function raises a
     '&imap-error' exception.

     Example:

          (define mbox
             (imap-login (make-client-socket "imap.inria.fr" 993 :timeout 200000)
                         "serrano" "XXX"))

          (print (mailbox-folders mbox))

 -- bigloo mail procedure: imap-logout socket
     Closes an 'imap' connection.

 -- bigloo mail procedure: imap-capability socket
     Returns the list of capabilities supported the 'imap' server.


File: bigloo.info,  Node: maildir,  Next: imap,  Up: Mail servers

19.5.3 Maildir
--------------

 -- bigloo mail class: maildir
          (class maildir::mailbox
            (prefix::bstring read-only (default "INBOX"))
            (path::bstring read-only))

     Example:

          (define mbox
            (instantiate::maildir
              (label "My Mailbox")
              (path (make-file-name (getenv "HOME") ".maildir"))))

          (tprint (mailbox-folders mbox))

 -- bigloo mail class: &maildir-error
          (class &maildir-error::&mailbox-error)


File: bigloo.info,  Node: Text,  Next: CSV,  Prev: Mail,  Up: Top

20 Text
*******

This chapter describes the Bigloo API for processing texts.

* Menu:

* BibTeX::
* Character strings::
* Character encodings::


File: bigloo.info,  Node: BibTeX,  Next: Character strings,  Up: Text

20.1 BibTeX
===========

 -- bigloo text function: bibtex obj
 -- bigloo text function: bibtex-port input-port
 -- bigloo text function: bibtex-file file-name
 -- bigloo text function: bibtex-string string
     These function parse BibTeX sources.  The variable OBJ can either
     be an input-port or a string which denotes a file name.  It returns
     a list of BibTeX entries.

     The functions 'bibtex-port', 'bibtex-file', and 'bibtex-string' are
     mere wrappers that invoke 'bibtex'.

     Example:
          (bibtex (open-input-string "@book{ as:sicp,
            author 	= {Abelson, H. and Sussman, G.},
            title 	= {Structure and Interpretation of Computer Programs},
            year 		= 1985,
            publisher 	= {MIT Press},
            address 	= {Cambridge, Mass., USA},
          }")) => (("as:sicp" BOOK
                            (author ("Abelson" "H.") ("Sussman" "G."))
                            (title . "Structure and Interpretation of Computer Programs")
                            (year . "1985")
                            (publisher . "MIT Press")
                            (address . "Cambridge, Mass., USA")))

 -- bigloo text function: bibtex-parse-authors string
     This function parses the author field of a bibtex entry.

     Example:
          (bibtex-parse-authors "Abelson, H. and Sussman, G.")
          => (("Abelson" "H.") ("Sussman" "G."))


File: bigloo.info,  Node: Character strings,  Next: Character encodings,  Prev: BibTeX,  Up: Text

20.2 Character strings
======================

 -- bigloo text function: hyphenate word hyphens
     The function 'hyphenate' accepts as input a single word and returns
     as output a list of subwords.  The argument HYPHENS is an opaque
     data structure obtained by calling the function 'load-hyphens' or
     'make-hyphens'.

     Example:
          (hyphenate "software" (load-hyphens 'en)) => ("soft" "ware")

 -- bigloo text function: load-hyphens obj
     Loads an hyphens table and returns a data structure suitable for
     'hyphenate'.  The variable OBJ can either be a file name containing
     an hyphens table or a symbol denoting a pre-defined hyphens table.
     Currently, Bigloo supports two tables: 'en' for an English table
     and 'fr' for a French table.  The procedure 'load-hyphens' invokes
     'make-hyphens' to build the hyphens table.

  Example:
     (define (hyphenate-text text lang)
        (let ((table (with-handler
                        (lambda (e)
                           (unless (&io-file-not-found-error? e)
                              (raise e)))
                        (load-hyphens lang)))
              (words (string-split text " ")))
           (if table
               (append-map (lambda (w) (hyphenate w table)) words)
                words)))

  The procedure 'hyphenate-text' hyphenates the words of the 'text'
according to the rules for the language denoted by its code 'lang' if
there is a file LANG'-hyphens.sch'.  If there is no such file, the text
remains un-hyphenated.

 -- bigloo text function: make-hyphens [:language] [:exceptions]
          [:patterns]
     Creates an hyphens table out of the arguments EXCEPTIONS and
     PATTERNS.

     The implementation of the table of hyphens created by
     'make-hyphens' follows closely Frank Liang's algorithm as published
     in his doctoral dissertation _Word Hy-phen-a-tion By Com-pu-ter_
     available on the TeX Users Group site here:
     <http://www.tug.org/docs/liang/>.  This table is a _trie_ (see
     <http://en.wikipedia.org/wiki/Trie> for a definition and an
     explanation).

     Most of this implementation is borrowed from Phil Bewig's work
     available here: <http://sites.google.com/site/schemephil/>, along
     with his paper describing the program from which the Bigloo
     implementation is largely borrowed.

     EXCEPTIONS must be a non-empty list of explicitly hyphenated words.

     Explicitly hyphenated words are like the following:
     '"as-so-ciate"', '"as-so-ciates"', '"dec-li-na-tion"', where the
     hyphens indicate the places where hyphenation is allowed.  The
     words in EXCEPTIONS are used to generate hyphenation patterns,
     which are added to PATTERNS (see next paragraph).

     PATTERNS must be a non-empty list of hyphenation patterns.

     Hyphenation patterns are strings of the form '".anti5s"', where a
     period denotes the beginning or the end of a word, an odd number
     denotes a place where hyphenation is allowed, and an even number a
     place where hyphenation is forbidden.  This notation is part of
     Frank Liang's algorithm created for Donald Knuth's TeX typographic
     system.


File: bigloo.info,  Node: Character encodings,  Prev: Character strings,  Up: Text

20.3 Character encodings
========================

 -- bigloo text function: gb2312->ucs2 string
     Converts a GB2312 (aka cp936) encoded 8bits string into an UCS2
     string.


File: bigloo.info,  Node: CSV,  Next: Eval,  Prev: Text,  Up: Top

21 CSV
******

This chapter describes the Bigloo API for processing CSV spreadsheets.
This chapter has been written by Joseph Donaldson, as the implementation
of the CSV library.

* Menu:

* CSV Overview::    What is this Bigloo csv library?.
* API Reference::
* Example::


File: bigloo.info,  Node: CSV Overview,  Next: API Reference,  Up: CSV

21.1 Overview
=============

The Bigloo csv library supports the parsing of csv and csv-like data.
By default, it enables the parsing of comma, tab, and pipe separated
data.  In addition, facilities are provided that enable extending the
library to support additonal csv-like formats.

  The rest of this document describes the Bigloo csv application
programming interface.


File: bigloo.info,  Node: API Reference,  Next: Example,  Prev: CSV Overview,  Up: CSV

21.2 API Reference
==================

* Menu:

* read-csv-record::
* read-csv-records::
* csv-for-each::
* csv-map::
* make-csv-lexer::
* +csv-lexer+:: comma separated values
* +tsv-lexer+:: tab separated values
* +psv-lexer+:: pipe separated values


File: bigloo.info,  Node: read-csv-record,  Next: read-csv-records,  Up: API Reference

 -- bigloo procedure: read-csv-record input-port [custom-lexer]

     'read-csv-record' has one required argument, the input-port of the
     csv data to parse, and an optional argument indicating the lexer to
     use, by default the lexer supporting standard csv files.  It
     returns a single record, as a list, or '#eof-object'.  Upon error,
     it will throw an '&invalid-port-error' or '&io-parse-error'
     exception.


File: bigloo.info,  Node: read-csv-records,  Next: csv-for-each,  Prev: read-csv-record,  Up: API Reference

 -- bigloo procedure: read-csv-records input-port [custom-lexer]

     'read-csv-records' has one required argument, the input-port of the
     csv data to parse, and an optional argument indicating the lexer to
     use, by default the lexer supporting standard csv files.  It
     returns all of the records, as a list of lists, or '#eof-object'.
     Upon error, it will throw an '&invalid-port-error' or
     '&io-parse-error exception'.


File: bigloo.info,  Node: csv-for-each,  Next: csv-map,  Prev: read-csv-records,  Up: API Reference

 -- bigloo procedure: csv-for-each proc input-port [custom-lexer]

     'csv-for-each' has two required arguments, a procedure to apply to
     each record and the input-port of the csv data to parse, and an
     optional argument indicating the lexer to use, by default the lexer
     supporting standard csv files.  It returns '#unspecified'.  Upon
     error, it will throw an '&invalid-port-error' or '&io-parse-error
     exception'.


File: bigloo.info,  Node: csv-map,  Next: make-csv-lexer,  Prev: csv-for-each,  Up: API Reference

 -- bigloo procedure: csv-map proc input-port [custom-lexer]

     'csv-map' has two required arguments, a procedure to apply to each
     record and the input-port of the csv data to parse, and an optional
     argument indicating the lexer to use, by default the lexer
     supporting standard csv files.  It returnsthe results of applying
     'proc' to each record as a list.  Upon error, it will throw an
     '&invalid-port-error' or '&io-parse-error' exception.


File: bigloo.info,  Node: make-csv-lexer,  Next: +csv-lexer+,  Prev: csv-map,  Up: API Reference

 -- bigloo form: make-csv-lexer sep quot

     'make-csv-lexer' has two required arguments, a character used to
     separate records and a character for quoting.  It returns custom
     lexer.


File: bigloo.info,  Node: +csv-lexer+,  Next: +tsv-lexer+,  Prev: make-csv-lexer,  Up: API Reference

 -- Variable: bigloo variable +csv-lexer+
     +CSV-LEXER+ is a bigloo-csv lexer supporting the standard
     comma-separated value format.


File: bigloo.info,  Node: +tsv-lexer+,  Next: +psv-lexer+,  Prev: +csv-lexer+,  Up: API Reference

 -- Variable: bigloo variable +tsv-lexer+
     +TSV-LEXER+ is a bigloo-csv lexer supporting the tab-separated
     value format.


File: bigloo.info,  Node: +psv-lexer+,  Prev: +tsv-lexer+,  Up: API Reference

 -- Variable: bigloo variable +psv-lexer+
     +PSV-LEXER+ is a bigloo-csv lexer supporting the pipe-separated
     value format.


File: bigloo.info,  Node: Example,  Prev: API Reference,  Up: CSV

The following is a simple example of using the bigloo-csv library.  It
parses a single record from the given csv data and prints it.

     (module example
        (library bigloo-csv)
        (main main))

     (define +csv-data+ "dog,cat,horse\npig,cow,squirrel")

     (define (main args)
        (let ((in (open-input-string +csv-data+)))
           (unwind-protect
              (print (read-csv-record in))
              (close-input-port in))))


File: bigloo.info,  Node: Eval,  Next: Macro Expansion,  Prev: CSV,  Up: Top

22 Eval and code interpretation
*******************************

This chapter describes the Bigloo evaluator.

* Menu:

* Eval compliance::
* Eval standard functions::
* Eval command line options::
* Eval and the foreign interface::


File: bigloo.info,  Node: Eval compliance,  Next: Eval standard functions,  Up: Eval

22.1 Eval compliance
====================

Bigloo includes an interpreter.  Unfortunately, the language accepted by
the interpreter is a proper subset of that accepted by the compiler.
The main differences are:

   * No foreign objects can be handled by interpreter.
   * Classes of the object system cannot be declared within interpreted
     code.
   * The interpreter ignores modules, and has a unique global
     environment.

  Compiled code and interpreted code can be mixed together.  That is,
interpreted code is allowed to call compiled code and vice versa.  This
connection can be use to circumvent the missing features of the
interpreter (see Section *note Module Declaration::, for a description
of how to connect compiled and interpreted code).

  By default the evaluator assumes that operators from the standard
library (e.g., '+', 'car') are immutable.  Hence, it optimizes these
operators's calls.  This optimization can be disabled using the
'bigloo-eval-strict-module' parameter described in the chapter
describing the parameters (see *note Parameters::).


File: bigloo.info,  Node: Eval standard functions,  Next: Eval command line options,  Prev: Eval compliance,  Up: Eval

22.2 Eval standard functions
============================

 -- procedure: eval exp [env]
     This form evaluates EXP.  The second argument is optional.  It can
     be the evaluation of one of these three function forms:
          (scheme-report-environment 5)
          (null-environment 5)
          (interaction-environment)

 -- procedure: scheme-report-environment version
 -- procedure: null-environment VERSION
 -- procedure: interaction-environment VERSION
     These three procedures have the definitions given in the R5RS so
     see *note scheme-report-environment: (r5rs.info)Eval, for more
     details.

 -- bigloo procedure: byte-code-compile exp [env (default-environment)]
 -- bigloo procedure: byte-code-run byte-code
     The function 'byte-code-compile' compiles a Scheme expression into
     a sequence of byte codes that is implemented as a string.  The
     function 'byte-code-run' execute such a sequence.

 -- bigloo procedure: repl
     This invokes the READ-EVAL-PRINT loop.  Several 'repl' can be
     embedded.

     The 'repl' function can be used to implement custom Bigloo
     interpreters.  For instance, one may write:

          (module repl)
          (repl)

     When compiled, this will deliver an executable containing the sole
     Bigloo interpreter.

 -- bigloo procedure: set-repl-error-notifier! notifier
 -- bigloo procedure: get-repl-error-notifier
     Set or get the procedure used in the REPLs to display errors.  The
     notifier is a procedure of one argument, the error or exception
     that has been raised.  Example:

          (set-repl-error-notifier!
             (lambda (e)
                (print "=== error ======)
                (exception-notify e)))

 -- bigloo procedure: quit
     This exits from the currently running 'repl'.  If the current
     'repl' is the first one then this function ends the interpreter.

 -- bigloo procedure: set-prompter! proc
     The argument PROC has to be a procedure of one argument and
     invoking this function sets the 'repl' prompter.  That is, to
     display its prompt, 'repl' invokes PROC giving it the nesting level
     of the current loop as its argument.

 -- bigloo procedure: get-prompter
     Returns the current 'repl' prompter.

 -- bigloo procedure: set-repl-printer! proc
     The argument PROC has to be a procedure accepting one or two
     arguments.  This function sets the 'repl' display function.  That
     is, to display the result of its evaluations, 'repl' invokes PROC
     giving it the evaluated expression as first argument and the
     current output port (or a file in case of transcript) as second
     argument.  'Set-repl-printer!' returns the former 'repl' display
     function.

     For instance, one may write:

          1:=> (define x (cons 1 2))         -| X
          1:=> (define y (cons x x))         -| Y
          1:=> y                             -| (#0=(1 . 2) . #0#)
          1:=> (set-repl-printer! display)   -| #<procedure:83b8c70.-2>
          1:=> y                             -| ((1 . 2) 1 . 2)

 -- bigloo procedure: native-repl-printer
     Returns the native (default) 'repl' display function.

 -- bigloo procedure: expand exp
     Returns the value of EXP after all macro expansions have been
     performed.

 -- bigloo procedure: expand-once exp
     Returns the value of EXP after one macro expansion has been
     performed.

  It is possible to specify files which have to be loaded when the
interpreter is invoked.  For this, see section *note Compiler
Description::.

  If a Bigloo file starts with the line:
      #! bigloo-command-name

  and if this file is executable (in the meaning of the system) and if
the user tries to execute it, Bigloo will evaluate it.  Note also that
SRFI-22 support enables to run any Unix interpreter (*note SRFIs::).

 -- bigloo procedure: load filename
 -- bigloo procedure: loadq filename
     FILENAME should be a string naming an existing file which contains
     Bigloo source code.  This file is searched in the current directory
     and in all the directories mentioned in the variable '*load-path*'.
     The 'load' procedure reads expressions and definitions from the
     file, evaluating them sequentially.  If the file loaded is a module
     (i.e.  if it begins with a regular module clause), load behaves as
     module initialization.  Otherwise, this function returns the result
     of the last evaluation.  The function 'loadq' differs from the
     function 'load' in the sense that 'loadq' does not print any
     intermediate evaluations.

     Both functions return the full path of the loaded file.

 -- bigloo procedure: loada filename
     Loads an "access file", which allows the interpreter to find the
     modules imported by a loaded module.  It returns the full path of
     the loaded file.

 -- bigloo variable: *load-path*
     A list of search paths for the 'load' functions.

 -- bigloo procedure: dynamic-load filename #!optional (init init-point)

     Loads a shared library named 'filename'.  Returns the value of the
     last top-level expression.

     _Important note: _ The function 'dynamic-load' can only be used
     from compiled modules linked against dynamic libraries.  In
     particular, the 'dynamic-load' function can be issued from the
     'bigloo' command if and only if the option '--sharedcompiler=yes'
     has been used when configuring Bigloo.  If the 'bigloo' command is
     not linked against dynamic libraries and if 'dynamic-load' is
     required inside a read-eval-print loop (REPL) it exists a simple
     workaround.  It consists in implementing a new REPL and linking it
     against dynamic libraries.  This can be done as:

          $ cat > new-repl.scm <<EOF
          (module new-repl)
          (repl)
          EOF
          $ bigloo new-repl.scm -o new-repl
          $ new-repl
          1:=> (dynamic-load ...)



     If INIT-POINT is specified and if it is a string and if the library
     defines a function named 'init-point', this function is called when
     the library is loaded.  INIT-POINT is a C identifier, not a Scheme
     identifier.  In order to set the C name a Scheme function, use the
     extern 'export' clause (see Section *note C Interface::).  If the
     INIT-POINT is provided and is not a string, no initialization
     function is called after the library is loaded.  If the INIT-POINT
     value is not provided, once the library is loaded, 'dynamic-load'
     uses the Bigloo default entry point.  Normally you should _not_
     provide an INIT-POINT to 'dynamic-load' unless you known what you
     are doing.  When producing C code, to force the Bigloo compiler to
     emit such a default entry point, use the '-dload-sym' compilation
     option (see Section *note Compiler Description::).  This option is
     useless when using the JVM code generator.  Let's assume a Linux
     system and two Bigloo modules.  The first:

          (module mod1
             (eval (export foo))
             (export (foo x)))

          (define (foo x)
             (print "foo: " x))

          (foo 4)

     The second:

          (module mod2
             (import (mod1 "mod1.scm"))
             (eval (export bar))
             (export (bar x)))

          (define (bar x)
             (print "bar: " x))

          (bar 5)

     If these modules are compiled as:

          $ bigloo mod1.scm -c -o mod1.o
          $ bigloo mod2.scm -c -o mod2.o -dload-sym

     Then, if a shared library is built using these two modules (note
     that on non Linux systems, a different command line is required):

          $ ld -G -o lib.so mod1.o mod2.o

     Then, 'lib.so' cant be dynamically loaded and the variables it
     defines used such as :

          $ bigloo -i
          (dynamic-load "lib.so")
               -| foo: 4
                 bar: 5
          1:=> (foo 6)
               -| foo: 7

     As the example illustrates, when Bigloo modules are dynamically
     loaded, they are initialized.  This initialization is ensure _only_
     if 'dynamic-load' is called with exactly one parameter.  If
     'dynamic-load' is called with two parameters, it is of the
     responsibility of the program to initialize the dynamically loaded
     module before using any Scheme reference.

     _Note: _ In order to let the loaded module accesses the variables
     defined by the loader application, special compilation flags must
     be used (e.g., '-rdynamic' under the Linux operating system).
     'Dynamic-load' is implemented on the top of the 'dlopen' facility.
     For more information read the 'dlopen' and 'ld' manuals.

 -- bigloo procedure: dynamic-unload filename
     On the operating system that supports this facility, unloads a
     shared library.  Returns '#t' on success.  Returns '#f' otherwise.

 -- bigloo variable: *dynamic-load-path*
     A list of search paths for the 'dynamic-load' functions.

 -- bigloo-procedure: dynamic-load-symbol filename name #!optional
          module
 -- bigloo-procedure: dynamic-load-symbol-get dlsym
 -- bigloo-procedure: dynamic-load-symbol-set dlsym val

     The function 'dynamic-load-symbol' looks up for a variable in the
     dynamic library FILENAME.  If found, it returns a 'custom' Bigloo
     object denoting that variable.  Otherwise it returns '#f'.  This
     function assumes that FILENAME has previously been successfully
     loaded with 'dynamic-load'.  If not, an error is raised.  The
     argument FILENAME must be equal (in the sense of 'string=?' to the
     argument used to load the library.

     The C name of the looked up variable is NAME is MODULE is not
     provided.  Otherwise, it is the result of calling
     'bigloo-module-mangle' with NAME and MODULE as arguments.

     The function 'dynamic-load-symbol-get' returns the value of a
     dynamically loaded variable.

     The function 'dynamic-load-symbol-set' sets the value of a dynamic
     loaded variable.  It assumes that the variable is writable, i.e.,
     that it has not been compiled as a C constant.

 -- procedure: transcript-on filename
 -- procedure: transcript-off


File: bigloo.info,  Node: Eval command line options,  Next: Eval and the foreign interface,  Prev: Eval standard functions,  Up: Eval

22.3 Eval command line options
==============================

This section presents the Bigloo compiler options that impact the
interaction between compiled and interpreted code.  The whole list of
the Bigloo compiler options can be found in *note The Bigloo command
line: Compiler Description.

   * '-i' Don't compile a module, interpret it!
   * '-export-all' Make all the bindings _defined_ by the compiled
     module available from the interpreter.
   * '-export-export' Make all the bindings _exported_ by the compiled
     module available from the interpreter.
   * '-export-mutable' Make all the bindings _exported_ by the compiled
     module mutable from outside the module.  This option is
     _dangerous_!  Either all the modules composing the application must
     be compiled with or without '-export-mutable'.  It is impossible to
     mix '-export-mutable' enabled and disabled compilations.


File: bigloo.info,  Node: Eval and the foreign interface,  Prev: Eval command line options,  Up: Eval

22.4 Eval and the foreign interface
===================================

To be able to get access to foreign functions within the Bigloo
interpreter, some extra measurements have to be taken.  The foreign
functions have to be present in the interpreter binary, which means you
have to compile a custom interpreter.  This is described in Section
*note Using C bindings within the interpreter::.


File: bigloo.info,  Node: Macro Expansion,  Next: Parameters,  Prev: Eval,  Up: Top

23 Macro expansion
******************

Bigloo makes use of two macro expansion system.  The one based on the
expansion passing style [Dybvig et al.  86] and the one advocated by the
R5RS, for which see *note (r5rs.info)Top::.

23.1 Expansion passing style macros
===================================

 -- bigloo syntax: define-expander name proc
     This form defines an expander, NAME, where PROC is a procedure of
     two arguments: a form to macro-expand, and an expander.

 -- bigloo syntax: define-macro (name [args]...) body
     This form is itself macro-expanded into a 'define-expander' form.

     Macro expanders cannot be exported or imported since there is no
     way to specify expanders in a module declaration.

     Macros defined with 'define-expander' and 'define-macro' are used
     by both the compiler and the interpreter.

  Here is an example of an expander:
     (define-expander when
        (lambda (x e)
           (match-case x
              ((?- ?test . ?exps)
               (e `(if ,test (begin ,@exps)) e))
              (else
                (error "when" "illegal form" x)))))

     (when (> a 0) (print a) a)
        ==> (if (> a 0) (begin (print a) a))

  The same example can written with a 'define-macro' form:
     (define-macro (when test . exps)
        `(if ,test (begin ,@exps)))

23.2 Revised(5) macro expansion
===============================

Bigloo support the Revised(5) Report on the Scheme programming language.
For a detailed documentation see *Note r5rs.info: (R5RS)Expressions.

 -- syntax: let-syntax (binding...) body
 -- syntax: letrec-syntax (binding...) body
 -- syntax: define-syntax keyword transformer
 -- syntax: syntax-rules literals rule...
     These three forms are compatible with the description of the
     Revised(5) Report on the Algorithmic Language Scheme.

     Implementation Note: Current Bigloo does not ensure hygiene for
     'let-syntax' and 'letrec-syntax'.  Hygienic expansion is only
     guaranteed for 'define-syntax'.


File: bigloo.info,  Node: Parameters,  Next: Explicit Typing,  Prev: Macro Expansion,  Up: Top

24 Parameters
*************

The Bigloo parameters drive the global behavior programs.  A parameter
is accessed via a pair of functions: a reader and a setter.  The type of
the value is given, in this documentation, by the name of the parameter
of the setter.

 -- bigloo function: bigloo-strict-r5rs-strings
 -- bigloo function: bigloo-strict-r5rs-strings-set! boolean

     Traditional syntax conforms to the Revised Report if the parameter
     'bigloo-strict-r5rs-strings' is not '#f'.  Otherwise constant
     strings specified by the '"([^"]|\")*"' are considered as foreign
     strings.

     For example, after reading the expression '"1\n23\t4\"5"', the
     following string is built, which is equal to '(string #\1 #\n #\2
     #\3 #\t #\4 #\" #\5)' if '(bigloo-strict-r5rs-strings)' is not
     '#f'.  It is '(string #\1 #\n #\2 #\3 #\tab #\4 #\" #\5)'
     otherwise.

     Printing this string will produce: '1n23t4"5'.

     The new foreign syntax allows C escape sequences to be recognized.
     For example, the expression '#"1\n23\t4\"5"' builds a string equal
     to:

     '(string #\1 #\newline #\2 #\3 #\t #\4 #\" #\5)'

     and printing this string will then produce:
          1
          23    4"5

 -- bigloo function: bigloo-compiler-debug
 -- bigloo function: bigloo-compiler-debug-set! integer
 -- bigloo function: bigloo-debug
 -- bigloo function: bigloo-debug-set! integer
 -- bigloo function: bigloo-warning
 -- bigloo function: bigloo-warning-set! integer
     These parameters control the debugging and warning level.  The
     'bigloo-compiler-debug' is automatically controlled by the compiler
     command line '-g' option (see *note Command Line Parsing::).

     When a program is compiled in debug mode LVL, the compiler
     introduces a call to '(bigloo-debug-set! lvl)' before the
     evaluation of the first expression.

     The 'bigloo-debug' parameter is used to control traces (see *note
     Tracing::).

 -- bigloo function: bigloo-trace
 -- bigloo function: bigloo-trace-set! list

     Specifies the active trace (see 'with-trace' form).  The argument
     LIST is the list of symbols which are active and which triggers the
     display of a 'with-trace' form.

 -- bigloo function: bigloo-trace-color
 -- bigloo function: bigloo-trace-color-set! bool
     Enables/disables traces coloring (see *note Tracing::).

 -- bigloo function: bigloo-trace-stack-depth
 -- bigloo function: bigloo-trace-stack-depth-set! integer
     Controls the depth of the stack trace to be displayed on errors.
     With systems that supports shell variables (such as Unix) this
     parameter is dynamically adjusted according to the value of the
     SHELL variable BIGLOOSTACKDEPTH.

 -- bigloo function: bigloo-eval-strict-module
 -- bigloo function: bigloo-eval-strict-module-set! bool
     When set to '#t' enables eval optimization that inlines operators
     calls.  This optimization reduces the memory footprint of an
     application and it reduces the execution time.

 -- bigloo function: bigloo-dns-enable-cache
 -- bigloo function: bigloo-dns-enable-cache-set! bool

     Enable/disable DNS name caching.

 -- bigloo function: bigloo-dns-cache-validity-timeout
 -- bigloo function: bigloo-dns-cache-validity-timeout-set! integer

     Get/set the validity period for the DNS cache entries.  It is
     expressed in seconds.


File: bigloo.info,  Node: Explicit Typing,  Next: C Interface,  Prev: Parameters,  Up: Top

25 Explicit typing
******************

Bigloo supports _type annotation_ or _type information_.  As shown in
Section ref Module declaration, these annotations can be written both in
the module clauses and in module bodies although module body type
information is optional.  It helps the compiler to produce better
quality code and to reject incorrectly typed programs.  Type annotations
can describe both the result and formal parameter types for global
functions and also types for local variable.  Due to our module language
design (in particular module initialization), Scheme global variables
_cannot_ support type information.

  Types are either atomic types (see *note Atomic types::), foreign
types (see *note Defining an extern type::), or a classes (see *note
Class declaration::).

  _Warning: _ All type annotations are ignored by the interpreter.

  Module body type annotations are introduced by the following special
forms.

 -- bigloo syntax: define (f[::type] [a[::type]]...) body
 -- bigloo syntax: define-inline (f[::type] [a[::type]]...) body
 -- bigloo syntax: let ((var[::type] ...) ...) body
 -- bigloo syntax: let loop ((var[::type] ...) ...) body
 -- bigloo syntax: let* ((var[::type] ...) ...) body
 -- bigloo syntax: letrec ((var[::type] ...) ...) body
 -- bigloo syntax: labels ((var[::type] (var[::type]...) b) ...) body

     Type annotations are optional.  That is, for any of these
     constructions, if a type annotation is missing, Bigloo uses the
     default generic type 'obj' instead.

     Here is an example of type annotated program:

          (module example
             (export (vector-fill!::vector ::vector ::obj)))

          (define (vector-fill! v filler)
            (let loop ((i::long (- (vector-length v) 1)))
               (if (< i 0)
                   v
                   (begin
                      (vector-set! v i filler)
                      (loop (- i 1))))))

          (let ((v::vector (make-vector 3 4)))
             (vector-fill! v "dummy"))

     The types that can be used in annotations are any of:

        * the basic Scheme types 'pair', 'null', 'bstring', 'bint'
          (presented in Section *note Defining an extern type::).
        * the basic extern types 'long', 'int', 'char', 'string'
          presented in Section *note Defining an extern type::.
        * the compound extern types described in Section *note Defining
          an extern type::.
        * the types introduced by class declarations (Section *note
          Class declaration::).

     When a function that contains type annotation is exported, the type
     annotations must be written in the prototype of the function in the
     export clause.  In that case the type annotation need to be written
     in the function definition:

          (module foo
             (export (succ::int ::int)))

          (define (succ x) (+ 1 x))


File: bigloo.info,  Node: C Interface,  Next: Java Interface,  Prev: Explicit Typing,  Up: Top

26 The C interface
******************

We call all the pieces of program devoted to the interactions between
Scheme and another language a _foreign interface_.  In Bigloo, the
foreign interface allows Scheme's functions and variables to be exported
to a foreign language and foreign functions and variables to be imported
into the Scheme code.  Using the foreign interface requires two kind of
operations.

   * Declarations -- type declarations, import declarations or export
     declarations.
   * Foreign reference in the Scheme code.

  Declarations take place in a special module clause, see *note Module
Declaration::, and reference to foreign variables within Scheme code
requires no special construction.  The current release of Bigloo
includes a C and a Java interface.  The Java connection is specified by
the means of a 'java' clause (*note Java Interface::).  The C interface
is active (that is the 'extern' module clauses are read) only when
compiling to C. So, when compiling to Jvm the binding declared in an
'extern' clause are not bound.

  Connecting Bigloo code with C is generally straightforward.  To
illustrate this simplicity, let us consider a simple example involving
two source files.  First a simple C file 'sum.c' containing a single
declaration:

     int sum(int x, int y) { return x + y; }

  Then, let us assume a Bigloo source code 'main.scm' that makes uses of
that C function:

     (module foo
        (extern (sum::int (::int ::int) "sum"))
        (main main))

     (define (main x)
        (print (sum (length x) 10)))

  With a Unix installation of Bigloo, this program can be compiled and
executed with the following commands:

       $ gcc sum.c -c
       $ bigloo main.scm sum.o -o main
       $ ./main 1 2 3

  The connection between Scheme and C is made particularly easy by
Bigloo because the programmer is free from inserting conversion between
Scheme values and C values.  When needed, these are automatically
inserted by the compiler.

* Menu:

* The syntax of the foreign declarations::
* The very dangerous pragma Bigloo special forms::
* Name mangling::
* Embedded Bigloo applications::
* Using C bindings within the interpreter::


File: bigloo.info,  Node: The syntax of the foreign declarations,  Next: The very dangerous pragma Bigloo special forms,  Prev: C Interface,  Up: C Interface

26.1 The syntax of the foreign declarations
===========================================

The syntax of _foreign_ clauses is defined by:

     <extern> ==> <variable-clause>
          | <function-clause>
          | <include-clause>
          | <export-clause>
          | <type-clause>

  Foreign clauses are automatically "transmitted" by the importation
process.  That is, if module 'module1' imports a module 'module2',
'module' treats the 'extern' clauses of 'module2' as though they were
included in its own module declaration.  Redefinition of a variable or a
function already defined in an foreign clause is an error.

* Menu:

* Automatic extern clauses generation::
* Importing an extern variable::
* Importing an extern function::
* Including an extern file::
* Exporting a Scheme variable::
* Defining an extern type::


File: bigloo.info,  Node: Automatic extern clauses generation,  Next: Importing an extern variable,  Prev: The syntax of the foreign declarations,  Up: The syntax of the foreign declarations

26.1.1 Automatic extern clauses generation
------------------------------------------

Extern clauses can be automatically generated using the Cigloo program
which is distributed in the same package as Bigloo.  Using Cigloo may be
a good way to understand how C prototypes (and types) have to be
declared in Bigloo.  Cigloo reads C files and generates the Bigloo
extern clauses for that files.


File: bigloo.info,  Node: Importing an extern variable,  Next: Importing an extern function,  Prev: Automatic extern clauses generation,  Up: The syntax of the foreign declarations

26.1.2 Importing an extern variable
-----------------------------------

The <variable-clause> denotes importation of variables.
     <variable-clause> ==> ( <typed-ident> <c-name>)
          | (macro <typed-ident> <string>)
          | (macro <typed-ident> (<typed-ident>+) <string>)
          | (infix macro <typed-ident> (<typed-ident>+) <string>)

  Only extern "non-'macro'" variables are mutable (that is mutable using
the 'set!' construction).  Bigloo does not emit "extern C prototype" for
variables introduced by a 'macro' clause.  <string> is the C name of
variable.  The Scheme name of that variable is extracted from the
<typed-ident>.

  Here is an example of variable importations:
     (module example
        (extern (c-var::double "c_var")
                (macro bufsiz::long "BUFSIZ")))

     (print "c-var: " c-var)
     (set! c-var (+ 1.0 c-var))
     (print "c-var: " c-var)
     (print "bufsize: " BUFSIZ)


File: bigloo.info,  Node: Importing an extern function,  Next: Including an extern file,  Prev: Importing an extern variable,  Up: The syntax of the foreign declarations

26.1.3 Importing an extern function
-----------------------------------

Function are imported using the <function-clause>.

     <function-clause> ==> (<typed-ident> (<typed-ident>*) <string>)
          | (<typed-ident> (<typed-ident>+ . <typed-ident>) <string>)
          | (macro <typed-ident> (<typed-ident>*) <string>)
          | (macro <typed-ident> (<typed-ident>+ . <typed-ident>) <string>)

  The function result type and Scheme name are extracted from the
<typed-ident>; the <typed-ident> denotes the type of the function
arguments and <string> is the C name of the function.  Bigloo does not
produce "C extern prototype" for macro functions (those introduced by
'macro' clauses).  If the typed identifier of the function does not
contain any type information.  Bigloo will emit a warning message when
compiling and it will use a default C type (e.g.  the 'int' C type) as
the return type of the function.

     (module example
        (extern (macro prn::int (::string . ::long) "printf")))

     (let ((n (read)))
        (prn #"fib(%d): %d\n" n (fib n)))


File: bigloo.info,  Node: Including an extern file,  Next: Exporting a Scheme variable,  Prev: Importing an extern function,  Up: The syntax of the foreign declarations

26.1.4 Including an extern file
-------------------------------

C files can be included in the C code produced by using
<include-clause>.

     <include-clause> ==> (include <string>)


File: bigloo.info,  Node: Exporting a Scheme variable,  Next: Defining an extern type,  Prev: Including an extern file,  Up: The syntax of the foreign declarations

26.1.5 Exporting a Scheme variable
----------------------------------

A Scheme variable (or function) can be exported to the foreign world if
and only if it is also exported using an 'export' clause.  Type
information is given in the Scheme exportation, thus, the only
requirement for a variable to be extern exported is to be given a
foreign name.  The foreign <export-clause> does this:

     <export-clause> ==> (export <ident> <string>)

  Here is an example of exportation:

     (module example
        (export (fib::long ::long))
        (extern (export fib "scheme_fib")))

     (define (fib x) (if (< x 2) 1 ...))


File: bigloo.info,  Node: Defining an extern type,  Prev: Exporting a Scheme variable,  Up: The syntax of the foreign declarations

26.1.6 Defining an extern type
------------------------------

New Bigloo types can be defined using extern <type-clause>.  These newly
introduced types can be used in any declaration (that is in any extern
_or_ Scheme module clause and in any Scheme variable or function
definition).  The syntax of <type-clause> is:

     <type-clause> ==> (type <ident> <type-def> <string>)
     <type-def> ==> <atomic-type>
          | <ident>
          | <struct-type>
          | <union-type>
          | <function-type>
          | <array-type>
          | <pointer-type>
          | <enum-type>
          | <opaque-type>

  The symbol <ident> is the Scheme name of the introduced type and
<string> is the C name of the type.  When Bigloo produces the definition
of a variable v of type s, it produces the following C code: s v';'.
This rules applies unless s is a pointer or an array and then, to
produce a C definition, the name of the elements of the array or the
elements pointed by the pointer type are used.  Hence, if v is for
instance 'foo' and s is '(array int)' the produced C code will be: 'int
*foo'.

* Menu:

* Atomic types::
* C structures and unions::
* C pointers::
* C null pointers::
* C arrays::
* C functions::
* C enums::
* C opaques::


File: bigloo.info,  Node: Atomic types,  Next: C structures and unions,  Up: Defining an extern type

26.1.6.1 Atomic types
.....................

The atomic types are the pre-existing ones, defined in the standard
Bigloo's library.
     <atomic-type> ==> <bigloo-type>
          | <c-type>
     <bigloo-type> ==> obj procedure
          | pair | nil | pair-nil
          | bint | blong | belong | bllong
          | bignum  |real | bbool | cnst
          | bstring | ucs2string | bchar | bucs2
          | vector | tvector | struct
          | tstruct | output-port | input-port
          | binary-port | unspecified | symbol | keyword
          | cell | date | process | exit
          | mutex | condvar | mmap
          | s8vector | u8vector | s16vector | u16vector
          | s32vector | u32vector | s64vector | u64vector
          | f32vector | f64vector
          | dynamic-env | opaque | foreign
     <c-type> ==> cobj char
          | uchar | short
          | ushort | int | uint | long
          | ulong | slong | elong | llong
          | bool | string
          | file | double | float | void
          | function

  The type 'obj' denotes the super type of all Bigloo types (i.e., all
Bigloo types, such as 'procedure', 'pair', ...)  is an 'obj'.  The type
'cobj' denotes the super of all C types (i.e., all preexisting C types
such as 'char', 'uchar', 'schar', 'short', ...).  The type 'pair-nil'
denotes values that are either pairs or the '()' value.


File: bigloo.info,  Node: C structures and unions,  Next: C pointers,  Prev: Atomic types,  Up: Defining an extern type

26.1.6.2 Struct and Union types
...............................

C struct and Union types can be declared in Bigloo using <struct-type>
clauses:

     <struct-type> ==> (struct (<typed-ident> <string>)^+)
     <union-type> ==> (union (<typed-ident> <string>)^+)

  This clause declared a C struct but C structure values _cannot_ be
handled by Bigloo.  Instead Bigloo is able to handle _pointers to_ C
structure.  Thus, in order to help the definition of extern types, when
a struct named STRUCT is defined, if it does not exists yet, Bigloo
automatically defines a type _pointer to the structure_.  This type is
named STRUCT'*'.

  When a pointer to a structure type is defined, Bigloo automatically
produces functions to manipulate objects of this type.  Let us suppose
the type definition of STRUCT'*':

     (type STRUCT
        (struct (ID1::TYPE1 NAME1)
                ...
                (IDN::TYPEN NAMEN))

  The following functions are created:

   * A creator:
          (STRUCT*::STRUCT* ::TYPE_1 ... ::TYPE_N)

     This function allocates a fresh STRUCT'*' (in the same heap as any
     Scheme value) and fills the fields of the C structure with the
     proper values provided in the call.

   * A type checker:
          (STRUCT*?::bool OBJ::obj)

     This function returns '#t' if and only if the argument OBJ is of
     type STRUCT'*'.

   * A null checker:
          (STRUCT*-null?::bool ::STRUCT*)
     This function returns '#t' if and only if its argument is 'Null'.

   * A null creator:
          (make-null-STRUCT::STRUCT*)
     This function creates a 'NULL' value of type STRUCT*.

   * An equality checker:
          (=STRUCT*?::bool ::STRUCT* ::STRUCT*)

     This function returns '#t' if and only if its arguments are equal.

   * Accessors and mutators:
          (STRUCT*-ID_1::TYPE_1 ::STRUCT*)
          (STRUCT*-ID_1-set!::obj ::STRUCT* ::TYPE_1)
          ...
     These functions read and store field values.

  Here is an example of structure usage:

     (module foo
        (extern
          (include "named_point_declaration.h")
          (type named-point
                (struct (x::double "x")
                        (y::double "y")
                        (name::string "name"))
                "struct named_point")
          (c-print-point::int (named-point*) "ppoint")))

     (define (scheme-print-point point)
        (print "point*-name: " point
               " x: " (named-point*-x point)
               " y: " (named-point*-y point)))

     (let ((orig (named-point* 0.0 0.0 "orig")))
        (if (named-point*-null? orig)
            (error "bigloo" "cannot allocate point" orig)
            (begin
               (c-print-point orig)
               (scheme-print-point orig))))


File: bigloo.info,  Node: C pointers,  Next: C null pointers,  Prev: C structures and unions,  Up: Defining an extern type

26.1.6.3 C pointers
...................

C pointers are defined by the <pointer-type>

     <pointer-type> ==> (pointer <ident>)

  <ident> is the name of a previously defined type.  Let us suppose the
pointer type declaration:
     (type PTR (pointer IDENT) ...)

  If IDENT is the name of a structure type, Bigloo automatically creates
structure accessors (*note C structures and unions::).  Otherwise, it
creates the following functions:

   * A creator:
          (make-PTR::PTR NB::long)

     This function allocates memory for NB elements of type IDENT and
     returns a PTR to this zone.  The memory is filled with the C 'Null'
     value.

   * A type checker:
          (PTR?::bool OBJ::obj)
     This function returns '#t' the argument OBJ is of type PTR and '#f'
     otherwise.

   * A null checker:
          (PTR-null?::bool ::PTR)
     This function returns '#t' if its argument is 'Null' and '#f'
     otherwise.

   * A null creator:
          (make-null-PTR::PTR*)
     This function creates a 'NULL' value of type PTR*.

   * An equality checker:
          (=PTR*?::bool ::PTR* ::PTR*)
     This function returns '#t' if its arguments are equal and '#f'
     otherwise.

   * Accessors and mutators:
          (PTR-ref::IDENT ::PTR ::long)
          (PTR-set!::obj ::PTR ::long ::IDENT)
     These functions read and store field values.

  Here is an example of a program using pointer types:

     (module foo
        (extern
         (type double* (pointer double) "double *")))

     (define (make-vect::double* x y z)
        (let ((vect (make-double* 3)))
           (double*-set! vect 0 x)
           (double*-set! vect 1 y)
           (double*-set! vect 2 z)
           vect))

     (define (vect-norm vect::double*)
        (sqrt (+ (expt (double*-ref vect 0) 2)
     	    (expt (double*-ref vect 1) 2)
     	    (expt (double*-ref vect 2) 2))))

     (print (vect-norm (make-vect 1.2 4.5 -4.5)))


File: bigloo.info,  Node: C null pointers,  Next: C arrays,  Prev: C pointers,  Up: Defining an extern type

26.1.6.4 C null pointers
........................

It may be convenient to build C null pointers.  Several means can be
used.  In particular, foreign structures and pointers are provided with
Null creators.  For other foreign types, the easiest one is likely to be
a 'pragma' form.  For instance, in order to create a null pointer to a
'double*' type, one may use:

     (pragma::double* "((double *)0L)")

 -- bigloo procedure: string-ptr-null? string
 -- bigloo procedure: void*-null? void*
     These two predicates checks if there argument is the C 'NULL'
     value.

 -- bigloo procedure: make-string-ptr-null
 -- bigloo procedure: make-void*-null
     These two constructors creates _null_ foreign values.


File: bigloo.info,  Node: C arrays,  Next: C functions,  Prev: C null pointers,  Up: Defining an extern type

26.1.6.5 C arrays
.................

C arrays are defined by the <array-type>

     <array-type> ==> (array <ident>)

  <ident> is the name of a previously defined type.  Array types are
similar to pointer types except that they include their size in their
type definition string.  Let us suppose the array type declaration:
     (type ARRAY (array IDENT) ...)

  If IDENT is the name of a structure type, Bigloo automatically creates
structures accessors (*note C structures and unions::).  Otherwise, it
creates the following functions:

   * A creator:
          (make-ARRAY::ARRAY)
     This function allocates memory for the array ARRAY.  The memory is
     filled with the C 'Null' value.

   * A type checker:
          (ARRAY?::bool OBJ::obj)
     This function returns '#t' if the argument OBJ is of type _array_
     and '#f' otherwise.

   * A null checker:
          (null-ARRAY?::bool ::ARRAY)
     This function returns '#t' if the argument OBJ is 'Null' and '#f'
     otherwise.

   * An equality checker:
          (=ARRAY*?::bool ::ARRAY* ::ARRAY*)
     This function returns '#t' if its arguments are equal and '#f'
     otherwise.

   * Accessors and mutators:
          (ARRAY-ref::IDENT ::ARRAY ::long)
          (ARRAY-set!::obj ::ARRAY ::long ::IDENT)
     These functions read and store field values.

  Here is an example of a program using array types:

     (module foo
        (extern
         (type double* (array double) "double [ 10 ]")))

     (define (make-vect::double* x y z)
        (let ((vect (make-double*)))
           (double*-set! vect 0 x)
           (double*-set! vect 1 y)
           (double*-set! vect 2 z)
           vect))

     (define (vect-norm vect::double*)
        (sqrt (+ (expt (double*-ref vect 0) 2)
     	    (expt (double*-ref vect 1) 2)
     	    (expt (double*-ref vect 2) 2))))


     (print (vect-norm (make-vect 1.2 4.5 -4.5)))


File: bigloo.info,  Node: C functions,  Next: C enums,  Prev: C arrays,  Up: Defining an extern type

26.1.6.6 C functions
....................

C function types are introduced by the <function-type> clause:

     <function-type> ==> (function <ident> (<ident>*))

  Let us suppose the array type declaration:
     (type FUN (function RES (ARG*)) ...)

  Bigloo creates the following functions:

   * A type checker:
          (FUN?::bool OBJ::obj)
     This function returns '#t' if the argument OBJ is of type FUN and
     '#f' otherwise.

   * An equality checker:
          (=FUN*?::bool ::FUN* ::FUN*)
     This function returns '#t' if and only if its arguments are equal.

   * Caller:
          (FUN-call::RES F::FUN A::TA ...)
     This function invokes F with the arguments A ... AN.

  Suppose we have to use in Scheme the following C variable:

     double (*convert)(char *);

  It can be done as in:

     (module foo
        (extern
         (type *string->double
               (function double (string))
                         "double (*)(char *)")
         (macro cv::*string->double "convert")))

     (print (*string->double-call cv "3.14"))


File: bigloo.info,  Node: C enums,  Next: C opaques,  Prev: C functions,  Up: Defining an extern type

26.1.6.7 C enums
................

This form defines 'enum' types.

     <enum-type> ==> (enum (<ident> <string>)... ...)

  Let us suppose the type:
     (type ENUM
        (enum (ID_1 NAME_1)
              ...
              (ID_N NAME_N)))

  Bigloo creates the following functions:

   * Creators:
          (ENUM-ID_1::ENUM)
          ...
          (ENUM-ID_N::ENUM)
     These functions create ENUM values.

   * A type checker:
          (ENUM?::bool OBJ::obj)
     This function returns '#t' if the argument OBJ is of type ENUM and
     '#f' otherwise.

   * An equality checker:
          (=ENUM?::bool ::ENUM ::ENUM)
     This function returns '#t' if the arguments are equal and '#f'
     otherwise.

  Here is an example of Scheme code using ENUM type.

     (module foo
        (extern
         (type gizmo
               (enum (titi "titi")
                     (tutu "tutu")
                     (tata "tata"))
               "enum toto")))

     (let ((v1 (gizmo-titi))
           (v2 (gizmo-tutu)))
        (print (=gizmo? v1 v2)))


File: bigloo.info,  Node: C opaques,  Prev: C enums,  Up: Defining an extern type

26.1.6.8 C opaques
..................

This form defines 'opaque' types.

     <opaque-type> ==> (opaque)

  Let us suppose the type:
     (type OPA (opaque) ...)

  Bigloo creates the following functions:

   * A type checker:
          (OPA?::bool OBJ::obj)
     This function returns '#t' if the argument OBJ is of type OPA and
     '#f' otherwise.

   * An equality checker:
          (=OPA?::bool ::OPA ::OPA)
     This function returns '#t' if the arguments are equal and '#f'
     otherwise.

  Opaque types are relevant when a C value must transit via a Scheme
function from a C function to another C function.  The value can't be
used in Scheme because no accessors are defined over that type it can
only be send back to a C function.

  Here is an example of Scheme code using OPAQUE type.

     (module foo
        (extern (type filedes (opaque) "FILE *")
                (macro _fopen::filedes (::string ::string) "fopen")
                (_fgetc::int (::filedes) "fgetc")
                (_fclose (::filedes) "fclose"))
        (export (fopen::filedes ::bstring ::bstring)
                (fclose ::filedes)
                (fgetc::char ::filedes)))

     (define (fopen fname mode)
        (_fopen fname mode))

     (define (fclose filedes)
        (_fclose filedes))

     (define (fgetc filedes)
        (integer->char (_fgetc filedes)))

  _Note: _ To illustrate the default type compilation of extern
function, we have voluntary introduced an incomplete declaration for the
'fclose' function.  This will make Bigloo to produce a warning when
compiling that module.


File: bigloo.info,  Node: The very dangerous pragma Bigloo special forms,  Next: Name mangling,  Prev: The syntax of the foreign declarations,  Up: C Interface

26.2 The very dangerous "pragma" Bigloo special forms
=====================================================

Bigloo has a special form which allows the inclusion of C text into the
produced code.  It is _only_ applicable to the C back-end.  In
particular, the JVM back-end (see Chapter *note Java Interface::) does
not support it.

 -- bigloo syntax: pragma::IDENT string [args]
 -- bigloo syntax: free-pragma::IDENT string [args]
 -- bigloo syntax: pragma::IDENT IDENT
     This force Bigloo to include STRING in the produced C code as a
     regular C fragment of code.  This form must not be used without an
     in depth understanding of Bigloo C code production; with unskilled
     use, the produced C file may be unacceptable to the C compiler.

     Values can be passed to a 'pragma' form, being referenced in STRING
     by expressions of the form '$NUMBER'.  Such expression are replaced
     by the corresponding values, the number of referenced values in
     STRING being exactly the number of values provided.  Here is an
     example of 'pragma' usage:

          (define (fibo x::long)
             (pragma "printf( \"fib(%d):%d\\n\", $1, $2 );"
                     x
                     (fib x)))

     Arguments provided to a pragma form are not converted during
     compilation.  Hence, pragma arguments can be of any types,
     including, foreign types.

     A pragma result type can be specified using the notation
     'pragma::NAME' where the default type is 'unspecified'.  Then, for
     instance, the expression '(pragma::bool "$1 == 0" x)' will be
     considered to be returning a object of type 'bool' (C boolean)
     while the expression '(pragma "$1 == 0" x)' will be considered by
     Bigloo to be returning the 'unspecified' typed object.

     The compiler assumes that a 'pragma' forms operates a side effects
     and that it writes into its parameters.  This assumption no long
     holds with 'free-pragma'.  This is the only difference between the
     two forms.

     The last form 'pragma IDENT' enables "injecting" a Scheme mangled
     identifier into the generated C code.  Example:

          (let ((x/y 3))
            (pragma "$1 = BINT( 24 )" (pragma x/y))
            (print x/y))


File: bigloo.info,  Node: Name mangling,  Next: Embedded Bigloo applications,  Prev: The very dangerous pragma Bigloo special forms,  Up: C Interface

26.3 Name mangling
==================

In order to avoid name clashes, Bigloo uses name mangling when compiling
to C or to Jvm.  The name mangling for a Scheme identifier may be
overridden by the means of an extern 'export' clause (see Section *note
Exporting a Scheme variable::).

  Four public functions may be used to mangle and to demangle Scheme
identifiers:

 -- bigloo procedure: bigloo-mangle string
     Mangle the identifier STRING.

 -- bigloo procedure: bigloo-module-mangle string1 string2
     Mangle the identifier STRING1 that belongs to module STRING2.

 -- bigloo procedure: bigloo-mangled? string
     Returns '#t' if STRING has been computed by the 'bigloo-mangle' or
     'bigloo-module-mangle' function.

 -- bigloo procedure: bigloo-class-mangled? string
     Returns '#t' if STRING is a mangled name of a Bigloo class.

 -- bigloo procedure: bigloo-need-mangling string
     Returns '#t' if STRING requires name mangling because it is not a C
     or Jvm valid identifier.

 -- bigloo procedure: bigloo-demangle string
     Demangle previously mangled identifiers:

          (let ((id "foo!")
                (module "a-module"))
             (let ((mangled (bigloo-module-mangle id module)))
                (multiple-value-bind (new-id new-module)
                   (bigloo-demangle mangled)
                   (and (string=? id new-id) (string=? module new-module)))))
             => #t

 -- bigloo procedure: bigloo-class-demangle string
     Demangle previously mangled class identifier.


File: bigloo.info,  Node: Embedded Bigloo applications,  Next: Using C bindings within the interpreter,  Prev: Name mangling,  Up: C Interface

26.4 Embedded Bigloo applications
=================================

It is possible to design and realize embedded Bigloo applications.  This
facility is useful for adding a new Scheme part to an already existing C
program.  The C part of the program has only to enter the Bigloo
initialization, hence, it can call any Bigloo function.

  Normally, Bigloo creates an initialization function called ' main'
when it reads a 'main' module clause.  To use an embedded Bigloo
program, such an initialization function would have to be created but
with a different name.  Changing the name can be be done using the
following Bigloo option: '-copt "-DBIGLOO_MAIN=<NEW-NAME>"'.  To prevent
exit from the program after <NEW-NAME> is executed, the following Bigloo
option must be used: '-copt "-DBIGLOO_EXIT='BUNSPEC,'"'.

  A very important part of designing embedded Bigloo programs is being
sure that all used Bigloo modules are correctly initialized and the
normal way to initialize them is to use 'with' clauses in the module
which contains the 'main' clause.

  An example of an embedded program can be found in the distribution's
examples directory.


File: bigloo.info,  Node: Using C bindings within the interpreter,  Prev: Embedded Bigloo applications,  Up: C Interface

26.5 Using C bindings within the interpreter
============================================

To be able to get access to foreign functions within the Bigloo
interpreter, some extra measurements have to be taken.  The foreign
functions have to be present in the interpreter binary, which means you
have to compile a custom interpreter.  Fortunately, this is easy.  What
has to be done is to wrap the foreign functions within Scheme and make
an interpreter module.

  Let us consider an example where a C function 'get_system_time'
returning and 'int' is used in an interpreter.  (When linking, be sure
to add the '.o' file containing the 'get_system_time'.)

  The 'ffi-interpreter.scm' file:

     (module ExtendendInterpreter
        (import (wrapper "wrapper.scm"))
        (main main))

     (define (main argv)
        (repl))

  The 'wrapper.scm' file:

     (module wrapper
        (extern (macro %get-system-time::int () "get_system_time"))
        (export (get-system-time))
        (eval (export-exports))

     (define (get-system-time)
       (%get-system-time))

  Compile and link your application with something like:

     cc gettime.c -c gettime.o
     bigloo wrapper.scm -c
     bigloo ffi-interpreter.scm wrapper.o gettime.o


File: bigloo.info,  Node: Java Interface,  Next: Bigloo Libraries,  Prev: C Interface,  Up: Top

27 The Java interface
*********************

When the Bigloo is configured for a JVM back-end support, the compiler
is able to produce Java class file instead of C files.  In order to
produce JVM class files, use the '-jvm' compiler option.  Example:

     $ cat > foo.scm
     (module foo (main main))
     (define (main argv)
        (print "Hello world: " argv))
     $ bigloo -jvm foo.scm
     $ a.out
       -| Hello world: (a.out)

* Menu:

* Compiling with the JVM back-end::
* JVM back-end and SRFI-0::
* Limitation of the JVM back-end::
* Connecting Scheme and Java code::
* Performance of the JVM back-end::


File: bigloo.info,  Node: Compiling with the JVM back-end,  Next: JVM back-end and SRFI-0,  Up: Java Interface

27.1 Compiling with the JVM back-end
====================================

27.1.1 Compiler JVM options
---------------------------

All the compiler options that control the compilation (optimization
options, debugging options, etc.), can be used in conjunction with the
'-jvm' option.  However, the '-jvm' option _MUST_ be the first compiler
option on the command line.

  In order to prevent the compiler to produce a script shell file to run
the program, it is required to use simultaneously the '-jvm' and '-c'
options.

27.1.2 Compiling multi-modules applications
-------------------------------------------

In order to compile and link multi-modules applications, it is required
to specify the association between Scheme source modules and Java
qualified type names.  This task is generally complex because of the
annoying mapping that exists from Java class names and the operating
file system names.  In order to get rid of this problem, the Bigloo
standard distribution contains a tool, 'jfile', that automatically
produces Bigloo Module/Java classes association files.  The default name
for such a table is '.jfile'.  When compiling a module, Bigloo checks if
a '.jfile' exists in the current directory, if it exists, the file is
read.  The compilation option '-jfile' may be used to specify an
alternative jfile name.  Example:

     $ cat > foo.scm
     (module foo (export (foo))) (define (foo) 'foo)
     $ cat > bar.scm
     (module bar (export (bar))) (define (bar) 'bar)
     $ cat > hux.scm
     (module hux (export (hux))) (define (hux) 'hux)
     $ cat > main.scm
     (module main (main main) (import foo bar hux)
     (define (main argv)
        (print (foo))
        (print (bar))
        (print (fhux)))
     $ afile *.scm > .afile
     $ jfile *.scm > .jfile
     $ bigloo -jvm -c foo.scm
     $ bigloo -jvm -c bar.scm
     $ bigloo -jvm -c hux.scm
     $ bigloo -jvm main.scm foo.class bar.class hux.class

  For an explanation about the '.afile', *note Modules::.


File: bigloo.info,  Node: JVM back-end and SRFI-0,  Next: Limitation of the JVM back-end,  Prev: Compiling with the JVM back-end,  Up: Java Interface

27.2 JVM back-end and SRFI-0
============================

The currently running back-end may be tested by the means of the SRFI-0
'cond-expand' form (*note SRFIs::).  That is, when the JVM is ran, the
'bigloo-jvm' clause is true.  Otherwise, the 'bigloo-c' is true.
Example:

     $ cat > foo.scm
     (module foo (main main))
     (define (main argv)
        (cond-expand
           (bigloo-jvm (print "JVM back-end"))
           (bigloo-c (print "C back-end"))
           (else (error "main" "unsupported back-end" #unspecified))))
     $ bigloo -jvm foo.scm
     $ a.out
       -| JVM back-end
     $ bigloo foo.scm
     $ a.out
       -| C back-end


File: bigloo.info,  Node: Limitation of the JVM back-end,  Next: Connecting Scheme and Java code,  Prev: JVM back-end and SRFI-0,  Up: Java Interface

27.3 Limitation of the JVM back-end
===================================

The JVM back-end supports the entire Bigloo source language but the
'call/cc' function.  More precisely, using the JVM back-end, the
continuation reified in a 'call/cc' form can only be invoked in the
dynamic extent of that form.

  The other restrictions of the C back-end apply to the JVM back-end.
Mainly,

   * Bigloo is not able to compile all the tail recursive call without
     stack consumption (however, most of the tail recursive calls are
     optimized by Bigloo and don't use stack activation frames).

   * Bigloo compiled applications do not check for arithmetic overflow.

   * When compiling to Jvm, the 'extern' module clauses are not used.

   * Jvm runtime system does support the following function 'chdir'.

   * Jvm runtime system support for 'chmod' is restricted.

   * In order to read a shell variable from a Bigloo compiled Jvm
     program, you have to use the Bigloo link option '-jvm-env' _when_
     linking that program.  However, some shell variables are
     automatically defined ('HOME', 'USER', 'CLASSPATH' and 'TMPDIR'.

   * JVM code generation does not support 'pragma' forms.


File: bigloo.info,  Node: Connecting Scheme and Java code,  Next: Performance of the JVM back-end,  Prev: Limitation of the JVM back-end,  Up: Java Interface

27.4 Connecting Scheme and Java code
====================================

When compiling and linking with the JVM back-end, Bigloo source code may
use the Java API. That is, Bigloo Scheme source code may use (refer or
set) Java static variables, Bigloo source code may call static or
virtual Java methods.  In addition, Bigloo variables and functions may
be exported to Java, that is use, set or called in Java source code.
Java module clauses are enabled (read and parsed) only when compiling to
JVM byte code.

  Java definitions are declared in Bigloo modules by the mean of a
Bigloo module clause: the _java_ module clause.  The syntax of a _Java_
clause is defined by:

     <java> ==> <declare-class-clause>
          | <declare-abstract-class-clause>
          | <extend-class-clause>
          | <array-clause>
          | <export-clause>

  As for the _extern_ clause, _java_ clauses are automatically
"transmitted" by the importation process.  That is, if module 'module1'
imports a module 'module2', 'module' treats the 'java' clauses of
'module2' as though they were included in its own module declaration.
Redefinition of a variable or a function already defined in an java
clause is an error.  However, the definition of a Java class or an Java
abstract class may be enriched from module to module.

* Menu:

* Automatic Java clauses generation::
* Declaring Java classes::
* Declaring abstract Java classes::
* Extending Java classes::
* Declaring Java arrays::
* Exporting Scheme variables::
* Bigloo module initialization::


File: bigloo.info,  Node: Automatic Java clauses generation,  Next: Declaring Java classes,  Prev: Connecting Scheme and Java code,  Up: Connecting Scheme and Java code

27.4.1 Automatic Java clauses generation
----------------------------------------

Java clauses can be automatically generated using the Jigloo program
which is distributed in the same package as Bigloo.  Using Jigloo may be
a good way to understand how Java classes, methods, and variables have
to be declared in Bigloo.  Jigloo reads Java _class_ files and generate
the Bigloo java clauses for that classes.


File: bigloo.info,  Node: Declaring Java classes,  Next: Declaring abstract Java classes,  Prev: Automatic Java clauses generation,  Up: Connecting Scheme and Java code

27.4.2 Declaring Java classes
-----------------------------

The <declare-class-clause> clause denotes importation of Java classes.

     <declare-class-clause> ==> (class <typed-ident> <slot>* <string>)
     <slot> ==> <field> | <method> | <constructor>
     <field> ==> (field <modifier> <typed-ident> <string>)
     <method> ==> (method <modifier> <typed-ident> (<typed-ident>*) <string>)
     <constructor> ==> (constructor <ident> (<typed-ident>*))
     <modifier> ==> public | private | protected
          | static | final | synchronized | abstract

  When the compiler encounters a Java class declaration, it
automatically creates a predicate.  If the class identifier is 'id', the
predicate is named 'id?'.  In addition, the compiler generates functions
that fetch and set the field values.  For a field named 'f', these
functions are named 'id-f' and 'id-f-set!'.  Methods and constructors
are also always prefixed the name of the class.  That is, for a method
named 'm' of a class 'k', the Scheme name of the method is 'k-m'.

  Example:

     (module java-example
        (java (class point
                 (constructor new-default ())
                 (field x::int "x")
                 (method show::void (::point) "show")
                 (method static statistics::int () "PointStatistics")
                 "Point")
              (class point-3d::point
                 "Point3D")))

     (let ((p (point-new-default)))
        (print (point? p))   -| #t
        (point-x-set! p 3)
        (print (point-x p))) -| 3


File: bigloo.info,  Node: Declaring abstract Java classes,  Next: Extending Java classes,  Prev: Declaring Java classes,  Up: Connecting Scheme and Java code

27.4.3 Declaring abstract Java classes
--------------------------------------

A Bigloo abstract Java class declaration corresponds to a Java
interface.  It cannot be instantiate but regular classes may inherit
from it.

     <declare-abstract-class-clause> ==> (abstract-class <typed-ident> <slot>* <string>)


File: bigloo.info,  Node: Extending Java classes,  Next: Declaring Java arrays,  Prev: Declaring abstract Java classes,  Up: Connecting Scheme and Java code

27.4.4 Extending Java classes
-----------------------------

A class definition may be split into several pieces.  One class
declaration (see <declare-class-clause>) and several extensions.  The
syntax for a Java class extension is:

     <extend-class-clause> ==> (class <typed-ident> <slot>*)

  Example:

     (module java-example2
        (import java-example)
        (java (class point
                 (field y::int "y")
                 (field static num::int "point_num")
                 (constructor new (::int ::int)))))


File: bigloo.info,  Node: Declaring Java arrays,  Next: Exporting Scheme variables,  Prev: Extending Java classes,  Up: Connecting Scheme and Java code

27.4.5 Declaring Java arrays
----------------------------

Java arrays may be allocated and used inside Scheme code.  The syntax of
a Java array module clause is:

     <array-clause> ==> (array <ident> <typed-ident>)

  The <typed-ident> must refer to the name of an existing type (i.e., a
primitive Bigloo type, a Bigloo class, an already defined Java class or
an already defined Java array).  For an array named 'ar', Bigloo
generates:

   * a creator named 'make-ar' which is a function of one integer
     argument.

   * a predicate named 'ar?'.

   * a getter named 'ar-ref' which is a function of two arguments: an
     array and one integer argument.

   * a setter named 'ar-set!' which is a function of three arguments: an
     array, an integer, and a value of the array item types.

   * a length named 'ar-length'.

  Example:

     (module foo
        (java (array int* ::int)
              (class bar
                 (method static hello::int (::int*) "hello")
                 "bar"))
        (main main))

     (define (main argv)
        (let ((tab (make-int* 2)))
           (int*-set! tab 0 3)
           (int*-set! tab 1 6)
           (print (bar-hello tab))))


File: bigloo.info,  Node: Exporting Scheme variables,  Next: Bigloo module initialization,  Prev: Declaring Java arrays,  Up: Connecting Scheme and Java code

27.4.6 Exporting Scheme variables
---------------------------------

As for the C connection, a Scheme variable (or function) can be exported
to the Java world if and only if it is also exported using an 'export'
Java clause.  Type information is given in the Scheme exportation, thus,
the only requirement for a variable to be Java exported is to be given a
Java name.  The Java <export-clause> does this:

     <export-clause> ==> (export <ident> <string>)

  Here is an example of exportation:

     (module example
        (export (fib::long ::long))
        (java (export fib "scheme_fib")))

     (define (fib x) (if (< x 2) 1 ...))

  Here is a concrete example that illustrates how to invoke Scheme
functions from Java code.  First, here is a Java source file:

     // arr.java
     import java.io.*;
     import bigloo.*;

     public class arr {
        public static int hello( int[] tab ) {
           System.out.println( "tab[ 0 ]: " + tab[ 0 ] );
           System.out.println( "tab[ 1 ]: " + tab[ 1 ] );
           System.out.println( "<callback: " + main.callback( 10 ) + ">" );
           System.out.println("tab = main.squareJavaArray(tab);");
           tab = main.squareJavaArray(tab);
           System.out.println( "tab[ 0 ]: " + tab[ 0 ] );
           System.out.println( "tab[ 1 ]: " + tab[ 1 ] );
           return tab.length;
        }
     }

  Then the main Scheme file

     ;; main.scm
     (module main
        (java (array int* ::int)

           (class bar
     	 (method static hello::int (::int*) "hello")
     	 "bar"))

        (export (callback::int ::int))
        (export (squareJavaArray::int* ::int*))

        (main main))

     (define (main argv)
       (let ((tab (make-int* 2)))
         (print "tab length: " (int*-length tab))
         (int*-set! tab 0 3)
         (int*-set! tab 1 6)
         (display (int*-ref tab 1))
         (newline)

         (display "(java-array-int->scheme-vector tab) : ")
         (display (java-array-int->scheme-vector tab))
         (newline)
         (display "(square-vector (java-array-int->scheme-vector tab)) : ")
         (display (square-vector (java-array-int->scheme-vector tab)))
         (newline)

         (print (bar-hello tab))))

     (define (callback x)
        (+ 1 x))

     (define (square x)
        (* x x))

     (define (square-list L)
        (map square L))

     (define (square-vector V)
        (vector-map square V))

     (define (sum-vector V)
        (apply + (vector->list V)))

     (define (squareJavaArray A)
        (scheme-vector->java-array-int
           (square-vector (java-array-int->scheme-vector A))))

     (define (java-array-int->scheme-vector A)
        (let* ((len (int*-length A))
     	  (res (make-vector len)))
           (display len)
           (newline)
           (let loop ((i 0))
     	 (vector-set! res i (int*-ref A i))
     	 (if (= i (- len 1))
     	     res
     	     (loop (+ i 1))))))

     (define (scheme-vector->java-array-int V)
        (let* ((len (vector-length V))
     	  (res (make-int* len)))
           (let loop ((i 0))
     	 (int*-set! res i (vector-ref V i))
     	 (if (= i (- len 1))
     	     res
     	     (loop (+ i 1))))))

  To compile this example:

     $ bigloo -jvm main.scm -c
     $ javac arr.java -classpath=bigloo.zip:.
     $ bigloo -jvm main.scm arr.class

  To run it:

     $ ./a.out
     tab length: 2
     6
     (java-array-int->scheme-vector tab) : 2
     #(3 6)
     (square-vector (java-array-int->scheme-vector tab)) : 2
     #(9 36)
     tab[ 0 ]: 3
     tab[ 1 ]: 6
     <callback: 11>
     tab = main.squareJavaArray(tab);
     2
     tab[ 0 ]: 9
     tab[ 1 ]: 36
     2

