1.1 --- a/README	Sun Aug 24 22:32:57 2014 +0200
     1.2 +++ b/README	Mon Aug 25 02:52:47 2014 +0200
     1.3 @@ -1,5 +1,5 @@
     1.4 -seqlua: Extended sequences and iterators in Lua
     1.5 -===============================================
     1.6 +seqlua: Extension for handling sequential data in Lua
     1.7 +=====================================================
     1.8  
     1.9  This is an experimental package to extend Lua in the following manner:
    1.10  
    1.11 @@ -15,6 +15,122 @@
    1.12  ``__ipairs`` metamethod takes precedence over ``__index``, while the
    1.13  ``__len`` metamethod is never used.
    1.14  
    1.15 +Metamethod handling in detail is explained in the last section
    1.16 +("Respected metamethods") at the bottom of this README.
    1.17 +
    1.18 +
    1.19 +
    1.20 +Motivation
    1.21 +----------
    1.22 +
    1.23 +Sequential data (such as arrays or streams) is often represented in two
    1.24 +different ways:
    1.25 +
    1.26 +* as an ordered set of values (usually implemented as an array in other
    1.27 +  programming languages, or as a sequence in Lua: a table with numeric keys
    1.28 +  {1..n} associated with a value each),
    1.29 +* as some sort of data stream (sometimes implemented as a class of objects
    1.30 +  providing certain methods, or as an iterator function in Lua: a function that
    1.31 +  returns the next value with every call, where nil indicates the end of the
    1.32 +  stream).
    1.33 +
    1.34 +Quite often, when functions work on sequential data, it shouldn't matter in
    1.35 +which form the sequential data is being provided to the function. As an
    1.36 +example, consider a function that is writing a sequence of strings to a file.
    1.37 +Such function could either be fed with an array of strings (a table with
    1.38 +numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
    1.39 +function in Lua).
    1.40 +
    1.41 +A function in Lua that accepts a table, might look like as follows:
    1.42 +
    1.43 +    function write_lines(lines)
    1.44 +      for i, line in ipairs(lines) do
    1.45 +        io.stdout:write(line)
    1.46 +        io.stdout:write("\n")
    1.47 +      end
    1.48 +    end
    1.49 +
    1.50 +In contrast, a function in Lua that accepts an iterator function would have to
    1.51 +be implemented differently:
    1.52 +
    1.53 +    function write_lines(get_next_line)
    1.54 +      for line in get_next_line do
    1.55 +        io.stdout:write(line)
    1.56 +        io.stdout:write("\n")
    1.57 +      end
    1.58 +    end
    1.59 +
    1.60 +If one wanted to create a function that accepts either a sequence in form of a
    1.61 +table or an iterator function, then one might need to write:
    1.62 +
    1.63 +    function write_lines(lines)
    1.64 +      local iter1, iter2, iter3
    1.65 +      if type(lines) == "function" then
    1.66 +        iter1 = lines
    1.67 +      else
    1.68 +        iter1, iter2, iter3 = ipairs(lines)
    1.69 +      end
    1.70 +      for i, line in iter1, iter2, iter3 do
    1.71 +        io.stdout:write(line)
    1.72 +        io.stdout:write("\n")
    1.73 +      end
    1.74 +    end
    1.75 +
    1.76 +Obviously, this isn't something we want to write in every function that accepts
    1.77 +sequential data. Therefore, we usually decide for one of the two first forms
    1.78 +and therefore disallow the other possible representation of sequential data to
    1.79 +be passed to the function.
    1.80 +
    1.81 +This extension, however, modifies Lua's ``ipairs`` statement in such way that
    1.82 +it automatically accepts either a table or an iterator function as argument.
    1.83 +Thus, the first of the three functions above will accept both (table) sequences
    1.84 +and (function) iterators.
    1.85 +
    1.86 +In addition to the modification of ``ipairs``, it also provides C functions and
    1.87 +macros to iterate over values in the same manner as a generic loop statement
    1.88 +with ``ipairs`` would do.
    1.89 +
    1.90 +Note that this extension doesn't aim to supersede Lua's concept of iterator
    1.91 +functions. While metamethods (see section "Respected metamethods" below) may be
    1.92 +used to customize iteration behavior on values, this extension isn't thought to
    1.93 +replace the common practice to use function closures as iterators. Consider the
    1.94 +following example:
    1.95 +
    1.96 +    local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
    1.97 +    write_lines(result:get_column_entries("name"))
    1.98 +
    1.99 +The ``get_column_entries`` method can return a simple function closure that
   1.100 +returns the next entry in the "name" column (returning ``nil`` to indicate the
   1.101 +end). Such a closure can then be passed to another function that iterates
   1.102 +through a sequence of values by invoking ``ipairs`` with the general for-loop
   1.103 +(as previously shown).
   1.104 +
   1.105 +Where desired, it is also possible to use metamethods to customize iteration
   1.106 +behavior. Consider:
   1.107 +
   1.108 +    function process_row(database_row)
   1.109 +      -- some function
   1.110 +    end
   1.111 +    local result = sql_query("SELECT * FROM actor")
   1.112 +    process_row(result)  -- without writing :rows() or similar
   1.113 +
   1.114 +This extension, however, doesn't respect the ``__len`` metamethod due to the
   1.115 +following reasons:
   1.116 +
   1.117 +* An efficient implementation seems to be impossible, where
   1.118 +  ``for i, v in ipairs(tbl) do ... end`` does neither create a closure
   1.119 +  nor repeatedly evaluate ``#tbl``.
   1.120 +* Respecting ``__len`` could be used to implement sparse arrays, but this would
   1.121 +  require iterating functions to expect ``nil`` as a potential value. This may
   1.122 +  lead to problems because ``nil`` is usually also used to indicate the absence
   1.123 +  of a value.
   1.124 +
   1.125 +If such behavior is desired, though, it can still be implemented through the
   1.126 +``__ipairs`` metamethod.
   1.127 +
   1.128 +Unless manually done by the user in the ``__ipairs`` metamethod, this extension
   1.129 +never creates any closures or other values that need to be garbage collected.
   1.130 +
   1.131  
   1.132  
   1.133  Lua part of the library
   1.134 @@ -94,6 +210,7 @@
   1.135      -- prints: alpha,beta
   1.136  
   1.137  
   1.138 +
   1.139  C part of the library
   1.140  ---------------------
   1.141  
   1.142 @@ -144,6 +261,7 @@
   1.143  ``seqlua_iterloopauto`` is used.
   1.144  
   1.145  
   1.146 +
   1.147  Respected metamethods
   1.148  ---------------------
   1.149