seqlua
view README @ 51:06c5f2f9ec41
Added Lua version number 5.2 to README file
| author | jbe |
|---|---|
| date | Mon Aug 25 12:12:46 2014 +0200 (2014-08-25) |
| parents | 598f61d93402 |
| children | 3362ec36cb09 |
line source
1 seqlua: Extension for handling sequential data in Lua
2 =====================================================
4 This package is an extension for the Lua programming language (version 5.2)
5 which:
7 * allows ``ipairs(seq)`` to accept either tables or functions (i.e function
8 iterators) as an argument,
9 * adds a new function ``string.concat(separator, seq)`` that concats either
10 table entries or function return values,
11 * provides auxiliary C functions and macros to simplify iterating over both
12 tables and iterator functions with a generic statement.
14 Existing ``__ipairs`` or ``__index`` (but not ``__len``) metamethods are
15 respected by both the Lua functions and the C functions and macros. The
16 ``__ipairs`` metamethod takes precedence over ``__index``, while the
17 ``__len`` metamethod is never used.
19 Metamethod handling in detail is explained in the last section
20 ("Respected metamethods") at the bottom of this README.
22 In Lua, this extension is loaded by ``require "seqlua"``. In order to use the
23 auxiliary C functions and macros, add ``#include <seqlualib.h>`` to your C file
24 and ensure that the functions implemented in ``seqlualib.c`` are statically or
25 dynamically linked with your C Lua library.
29 Motivation
30 ----------
32 Sequential data (such as arrays or streams) is often represented in two
33 different ways:
35 * as an ordered set of values (usually implemented as an array in other
36 programming languages, or as a sequence in Lua: a table with numeric keys
37 {1..n} associated with a value each),
38 * as some sort of data stream (sometimes implemented as a class of objects
39 providing certain methods, or as an iterator function in Lua: a function that
40 returns the next value with every call, where nil indicates the end of the
41 stream).
43 Quite often, when functions work on sequential data, it shouldn't matter in
44 which form the sequential data is being provided to the function. As an
45 example, consider a function that is writing a sequence of strings to a file.
46 Such function could either be fed with an array of strings (a table with
47 numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
48 function in Lua).
50 A function in Lua that accepts a table, might look like as follows:
52 function write_lines(lines)
53 for i, line in ipairs(lines) do
54 io.stdout:write(line)
55 io.stdout:write("\n")
56 end
57 end
59 In contrast, a function in Lua that accepts an iterator function would have to
60 be implemented differently:
62 function write_lines(get_next_line)
63 for line in get_next_line do
64 io.stdout:write(line)
65 io.stdout:write("\n")
66 end
67 end
69 If one wanted to create a function that accepts either a sequence in form of a
70 table or an iterator function, then one might need to write:
72 do
73 local function write_line(line)
74 io.stdout:write(line)
75 io.stdout:write("\n")
76 end
77 function write_lines(lines)
78 if type(lines) == "function" then
79 for line in lines do
80 write_line(line)
81 end
82 else
83 for i, line in ipairs(lines) do
84 write_line(line)
85 end
86 end
87 end
88 end
90 Obviously, this isn't something we want to do in every function that accepts
91 sequential data. Therefore, we usually decide for one of the two first forms
92 and thus disallow the other possible representation of sequential data to be
93 passed to the function.
95 This extension, however, modifies Lua's ``ipairs`` statement in such way that
96 it automatically accepts either a table or an iterator function as argument.
97 Thus, the first of the three ``write_lines`` functions above will accept both
98 (table) sequences and (function) iterators.
100 In addition to the modification of ``ipairs``, it also provides C functions and
101 macros to iterate over values in the same manner as a generic loop statement
102 with ``ipairs`` would do.
104 Note that this extension doesn't aim to supersede Lua's concept of iterator
105 functions. While metamethods (see section "Respected metamethods" below) may be
106 used to customize iteration behavior on values, this extension isn't thought to
107 replace the common practice to use function closures as iterators. Consider the
108 following example:
110 local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
111 write_lines(result:get_column_entries("name"))
113 The ``get_column_entries`` method can return a simple function closure that
114 returns the next entry in the "name" column (returning ``nil`` to indicate the
115 end). Such a closure can then be passed to another function that iterates
116 through a sequence of values by invoking ``ipairs`` with the general for-loop
117 (as previously shown).
119 Where desired, it is also possible to use metamethods to customize iteration
120 behavior:
122 function print_rows(rows)
123 for i, row in ipairs(rows) do
124 print_row(row)
125 end
126 end
127 local result = sql_query("SELECT * FROM actor ORDER BY birthday")
128 assert(type(result) == "userdata")
130 -- we may rely on the ``__index`` or ``__ipairs`` metamethod to
131 -- iterate through all result rows here:
132 print_rows(result) -- no need to use ":rows()" or a similar syntax
134 -- but we can also still pass an individual set of result rows to the
135 -- print_rows function:
136 print_rows{result[1], result[#result]}
138 This extension, however, doesn't respect the ``__len`` metamethod due to the
139 following considerations:
141 * An efficient implementation where ``for i, v in ipairs(tbl) do ... end`` does
142 neither create a closure nor repeatedly evaluate ``#tbl`` seems to be
143 impossible.
144 * Respecting ``__len`` could be used to implement sparse arrays, but this would
145 require iterating functions to expect ``nil`` as a potential value. This may
146 lead to problems because ``nil`` is usually also used to indicate the absence
147 of a value.
149 Though, if such behavior is desired, it can still be implemented through the
150 ``__ipairs`` metamethod.
152 Unless manually done by the user in the ``__ipairs`` metamethod, the ``ipairs``
153 function as well as the corresponding C functions and macros provided by this
154 extension never create any closures or other values that need to be garbage
155 collected.
159 Lua part of the library
160 -----------------------
162 The modified ``ipairs(seq)`` and the new ``string.concat(sep, seq)`` functions
163 accept either a table or a function as ``seq``. This is demonstrated in the
164 following examples:
166 require "seqlua"
168 t = {"a", "b", "c"}
170 for i, v in ipairs(t) do
171 print(i, v)
172 end
173 -- prints:
174 -- 1 a
175 -- 2 b
176 -- 3 c
178 print(string.concat(",", t))
179 -- prints: a,b,c
181 function alphabet()
182 local letter = nil
183 return function()
184 if letter == nil then
185 letter = "a"
186 elseif letter == "z" then
187 return nil
188 else
189 letter = string.char(string.byte(letter) + 1)
190 end
191 return letter
192 end
193 end
195 for i, v in ipairs(alphabet()) do
196 print(i, v)
197 end
198 -- prints:
199 -- 1 a
200 -- 2 b
201 -- 3 c
202 -- ...
203 -- 25 y
204 -- 26 z
206 print(string.concat(",", alphabet()))
207 -- prints: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
209 function filter(f)
210 return function(seq)
211 return coroutine.wrap(function()
212 for i, v in ipairs(seq) do f(v) end
213 end)
214 end
215 end
217 alpha_beta_x = filter(function(v)
218 if v == "a" then
219 coroutine.yield("alpha")
220 elseif v == "b" then
221 coroutine.yield("beta")
222 elseif type(v) == "number" then
223 for i = 1, v do
224 coroutine.yield("X")
225 end
226 end
227 end)
229 print((","):concat(alpha_beta_x{"a", 3, "b", "c", "d"}))
230 -- prints: alpha,X,X,X,beta
232 print((","):concat(alpha_beta_x(alphabet())))
233 -- prints: alpha,beta
237 C part of the library
238 ---------------------
240 In ``seqlualib.h``, the following macro is defined:
242 #define seqlua_iterloop(L, iter, idx) \
243 for ( \
244 seqlua_iterinit((L), (iter), (idx)); \
245 seqlua_iternext(iter); \
246 )
248 and
250 #define seqlua_iterloopauto(L, iter, idx) \
251 for ( \
252 seqlua_iterinit((L), (iter), (idx)); \
253 seqlua_iternext(iter); \
254 lua_pop((L), 1) \
255 )
257 This macro allows iteration over either tables or iterator functions as the
258 following example function demonstrates:
260 int printcsv(lua_State *L) {
261 seqlua_Iterator iter;
262 seqlua_iterloop(L, &iter, 1) {
263 if (seqlua_itercount(&iter) > 1) fputs(",", stdout);
264 fputs(luaL_tolstring(L, -1, NULL), stdout);
265 // two values need to be popped (the value pushed by
266 // seqlua_iternext and the value pushed by luaL_tolstring)
267 lua_pop(L, 2);
268 }
269 fputs("\n", stdout);
270 return 0;
271 }
273 printcsv{"a", "b", "c"}
274 -- prints: a,b,c
276 printcsv(assert(io.open("testfile")):lines())
277 -- prints: line1,line2,... of "testfile"
279 NOTE: During iteration using ``seqlua_iterloop``, ``seqlua_iterloopauto``, or
280 ``seqlua_iterinit``, three extra elements are stored on the stack (additionally
281 to the value). These extra elements are removed automatically when the loop ends
282 (i.e. when ``seqlua_iternext`` returns zero). The value pushed onto the stack
283 for every iteration step has to be removed manually from the stack, unless
284 ``seqlua_iterloopauto`` is used.
288 Respected metamethods
289 ---------------------
291 Regarding the behavior of the Lua functions and the C functions and macros
292 provided by this extension, an existing ``__index`` metamethod will be
293 respected automatically. An existing ``__ipairs`` metamethod, however, takes
294 precedence.
296 If the ``__ipairs`` field of a value's metatable is set, then it must always
297 refer to a function. When starting iteration over a value with such a
298 metamethod being set, then this function is called with ``self`` (i.e. the
299 value itself) passed as first argument. The return values of the ``__ipairs``
300 metamethod may take one of the following 4 forms:
302 * ``return function_or_callable, static_argument, startindex`` causes the three
303 arguments to be returned by ``ipairs`` without further modification. Using
304 the C macros and functions for iteration, the behavior is according to the
305 generic loop statement in Lua:
306 ``for i, v in function_or_callable, static_argument, startindex do ... end``
307 * ``return "raw", table`` will result in iteration over the table ``table``
308 using ``lua_rawgeti``
309 * ``return "index", table_or_userdata`` will result in iteration over the table
310 or userdata while respecting any ``__index`` metamethod of the table or
311 userdata value
312 * ``return "call", function_or_callable`` will use the callable value as
313 (function) iterator where the function is expected to return a single value
314 without any index (the index is inserted automatically when using the
315 ``ipairs`` function for iteration)
317 These possiblities are demonstrated by the following example code:
319 require "seqlua"
321 do
322 local function ipairsaux(t, i)
323 i = i + 1
324 if i <= 3 then
325 return i, t[i]
326 end
327 end
328 custom = setmetatable(
329 {"one", "two", "three", "four", "five"},
330 {
331 __ipairs = function(self)
332 return ipairsaux, self, 0
333 end
334 }
335 )
336 end
337 print(string.concat(",", custom))
338 -- prints: one,two,three
339 -- (note that "four" and "five" are not printed)
341 tbl = {"alpha", "beta"}
343 proxy1 = setmetatable({}, {__index = tbl})
344 for i, v in ipairs(proxy1) do print(i, v) end
345 -- prints:
346 -- 1 alpha
347 -- 2 beta
349 proxy2 = setmetatable({}, {
350 __ipairs = function(self)
351 return "index", proxy1
352 end
353 })
354 for i, v in ipairs(proxy2) do print(i, v) end
355 -- prints:
356 -- 1 alpha
357 -- 2 beta
358 print(proxy2[1])
359 -- prints: nil
361 cursor = setmetatable({
362 "alice", "bob", "charlie", pos=1
363 }, {
364 __call = function(self)
365 local value = self[self.pos]
366 if value == nil then
367 self.pos = 1
368 else
369 self.pos = self.pos + 1
370 end
371 return value
372 end,
373 __ipairs = function(self)
374 return "call", self
375 end
376 })
377 for i, v in ipairs(cursor) do print(i, v) end
378 -- prints:
379 -- 1 alice
380 -- 2 bob
381 -- 3 charlie
382 print(cursor())
383 -- prints: alice
384 for i, v in ipairs(cursor) do print(i, v) end
385 -- prints:
386 -- 1 bob
387 -- 2 charlie
388 -- (note that "alice" has been returned earlier)
390 coefficients = setmetatable({1.25, 3.14, 17.5}, {
391 __index = function(self) return 1 end,
392 __ipairs = function(self) return "raw", self end
393 })
394 for i, v in ipairs(coefficients) do print(i, v) end
395 -- prints:
396 -- 1 1.25
397 -- 2 3.14
398 -- 3 17.5
399 -- (note that iteration terminates even if coefficients[4] == 1)
400 print(coefficients[4])
401 -- prints: 1