seqlua
view README @ 49:598f61d93402
Changed introduction in README
| author | jbe |
|---|---|
| date | Mon Aug 25 12:03:02 2014 +0200 (2014-08-25) |
| parents | facf29831f6f |
| children | 06c5f2f9ec41 |
line source
1 seqlua: Extension for handling sequential data in Lua
2 =====================================================
4 This package is an extension for the Lua language which:
6 * allows ``ipairs(seq)`` to accept either tables or functions (i.e function
7 iterators) as an argument,
8 * adds a new function ``string.concat(separator, seq)`` that concats either
9 table entries or function return values,
10 * provides auxiliary C functions and macros to simplify iterating over both
11 tables and iterator functions with a generic statement.
13 Existing ``__ipairs`` or ``__index`` (but not ``__len``) metamethods are
14 respected by both the Lua functions and the C functions and macros. The
15 ``__ipairs`` metamethod takes precedence over ``__index``, while the
16 ``__len`` metamethod is never used.
18 Metamethod handling in detail is explained in the last section
19 ("Respected metamethods") at the bottom of this README.
21 In Lua, this extension is loaded by ``require "seqlua"``. In order to use the
22 auxiliary C functions and macros, add ``#include <seqlualib.h>`` to your C file
23 and ensure that the functions implemented in ``seqlualib.c`` are statically or
24 dynamically linked with your C Lua library.
28 Motivation
29 ----------
31 Sequential data (such as arrays or streams) is often represented in two
32 different ways:
34 * as an ordered set of values (usually implemented as an array in other
35 programming languages, or as a sequence in Lua: a table with numeric keys
36 {1..n} associated with a value each),
37 * as some sort of data stream (sometimes implemented as a class of objects
38 providing certain methods, or as an iterator function in Lua: a function that
39 returns the next value with every call, where nil indicates the end of the
40 stream).
42 Quite often, when functions work on sequential data, it shouldn't matter in
43 which form the sequential data is being provided to the function. As an
44 example, consider a function that is writing a sequence of strings to a file.
45 Such function could either be fed with an array of strings (a table with
46 numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
47 function in Lua).
49 A function in Lua that accepts a table, might look like as follows:
51 function write_lines(lines)
52 for i, line in ipairs(lines) do
53 io.stdout:write(line)
54 io.stdout:write("\n")
55 end
56 end
58 In contrast, a function in Lua that accepts an iterator function would have to
59 be implemented differently:
61 function write_lines(get_next_line)
62 for line in get_next_line do
63 io.stdout:write(line)
64 io.stdout:write("\n")
65 end
66 end
68 If one wanted to create a function that accepts either a sequence in form of a
69 table or an iterator function, then one might need to write:
71 do
72 local function write_line(line)
73 io.stdout:write(line)
74 io.stdout:write("\n")
75 end
76 function write_lines(lines)
77 if type(lines) == "function" then
78 for line in lines do
79 write_line(line)
80 end
81 else
82 for i, line in ipairs(lines) do
83 write_line(line)
84 end
85 end
86 end
87 end
89 Obviously, this isn't something we want to do in every function that accepts
90 sequential data. Therefore, we usually decide for one of the two first forms
91 and thus disallow the other possible representation of sequential data to be
92 passed to the function.
94 This extension, however, modifies Lua's ``ipairs`` statement in such way that
95 it automatically accepts either a table or an iterator function as argument.
96 Thus, the first of the three ``write_lines`` functions above will accept both
97 (table) sequences and (function) iterators.
99 In addition to the modification of ``ipairs``, it also provides C functions and
100 macros to iterate over values in the same manner as a generic loop statement
101 with ``ipairs`` would do.
103 Note that this extension doesn't aim to supersede Lua's concept of iterator
104 functions. While metamethods (see section "Respected metamethods" below) may be
105 used to customize iteration behavior on values, this extension isn't thought to
106 replace the common practice to use function closures as iterators. Consider the
107 following example:
109 local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
110 write_lines(result:get_column_entries("name"))
112 The ``get_column_entries`` method can return a simple function closure that
113 returns the next entry in the "name" column (returning ``nil`` to indicate the
114 end). Such a closure can then be passed to another function that iterates
115 through a sequence of values by invoking ``ipairs`` with the general for-loop
116 (as previously shown).
118 Where desired, it is also possible to use metamethods to customize iteration
119 behavior:
121 function print_rows(rows)
122 for i, row in ipairs(rows) do
123 print_row(row)
124 end
125 end
126 local result = sql_query("SELECT * FROM actor ORDER BY birthday")
127 assert(type(result) == "userdata")
129 -- we may rely on the ``__index`` or ``__ipairs`` metamethod to
130 -- iterate through all result rows here:
131 print_rows(result) -- no need to use ":rows()" or a similar syntax
133 -- but we can also still pass an individual set of result rows to the
134 -- print_rows function:
135 print_rows{result[1], result[#result]}
137 This extension, however, doesn't respect the ``__len`` metamethod due to the
138 following considerations:
140 * An efficient implementation where ``for i, v in ipairs(tbl) do ... end`` does
141 neither create a closure nor repeatedly evaluate ``#tbl`` seems to be
142 impossible.
143 * Respecting ``__len`` could be used to implement sparse arrays, but this would
144 require iterating functions to expect ``nil`` as a potential value. This may
145 lead to problems because ``nil`` is usually also used to indicate the absence
146 of a value.
148 Though, if such behavior is desired, it can still be implemented through the
149 ``__ipairs`` metamethod.
151 Unless manually done by the user in the ``__ipairs`` metamethod, the ``ipairs``
152 function as well as the corresponding C functions and macros provided by this
153 extension never create any closures or other values that need to be garbage
154 collected.
158 Lua part of the library
159 -----------------------
161 The modified ``ipairs(seq)`` and the new ``string.concat(sep, seq)`` functions
162 accept either a table or a function as ``seq``. This is demonstrated in the
163 following examples:
165 require "seqlua"
167 t = {"a", "b", "c"}
169 for i, v in ipairs(t) do
170 print(i, v)
171 end
172 -- prints:
173 -- 1 a
174 -- 2 b
175 -- 3 c
177 print(string.concat(",", t))
178 -- prints: a,b,c
180 function alphabet()
181 local letter = nil
182 return function()
183 if letter == nil then
184 letter = "a"
185 elseif letter == "z" then
186 return nil
187 else
188 letter = string.char(string.byte(letter) + 1)
189 end
190 return letter
191 end
192 end
194 for i, v in ipairs(alphabet()) do
195 print(i, v)
196 end
197 -- prints:
198 -- 1 a
199 -- 2 b
200 -- 3 c
201 -- ...
202 -- 25 y
203 -- 26 z
205 print(string.concat(",", alphabet()))
206 -- prints: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
208 function filter(f)
209 return function(seq)
210 return coroutine.wrap(function()
211 for i, v in ipairs(seq) do f(v) end
212 end)
213 end
214 end
216 alpha_beta_x = filter(function(v)
217 if v == "a" then
218 coroutine.yield("alpha")
219 elseif v == "b" then
220 coroutine.yield("beta")
221 elseif type(v) == "number" then
222 for i = 1, v do
223 coroutine.yield("X")
224 end
225 end
226 end)
228 print((","):concat(alpha_beta_x{"a", 3, "b", "c", "d"}))
229 -- prints: alpha,X,X,X,beta
231 print((","):concat(alpha_beta_x(alphabet())))
232 -- prints: alpha,beta
236 C part of the library
237 ---------------------
239 In ``seqlualib.h``, the following macro is defined:
241 #define seqlua_iterloop(L, iter, idx) \
242 for ( \
243 seqlua_iterinit((L), (iter), (idx)); \
244 seqlua_iternext(iter); \
245 )
247 and
249 #define seqlua_iterloopauto(L, iter, idx) \
250 for ( \
251 seqlua_iterinit((L), (iter), (idx)); \
252 seqlua_iternext(iter); \
253 lua_pop((L), 1) \
254 )
256 This macro allows iteration over either tables or iterator functions as the
257 following example function demonstrates:
259 int printcsv(lua_State *L) {
260 seqlua_Iterator iter;
261 seqlua_iterloop(L, &iter, 1) {
262 if (seqlua_itercount(&iter) > 1) fputs(",", stdout);
263 fputs(luaL_tolstring(L, -1, NULL), stdout);
264 // two values need to be popped (the value pushed by
265 // seqlua_iternext and the value pushed by luaL_tolstring)
266 lua_pop(L, 2);
267 }
268 fputs("\n", stdout);
269 return 0;
270 }
272 printcsv{"a", "b", "c"}
273 -- prints: a,b,c
275 printcsv(assert(io.open("testfile")):lines())
276 -- prints: line1,line2,... of "testfile"
278 NOTE: During iteration using ``seqlua_iterloop``, ``seqlua_iterloopauto``, or
279 ``seqlua_iterinit``, three extra elements are stored on the stack (additionally
280 to the value). These extra elements are removed automatically when the loop ends
281 (i.e. when ``seqlua_iternext`` returns zero). The value pushed onto the stack
282 for every iteration step has to be removed manually from the stack, unless
283 ``seqlua_iterloopauto`` is used.
287 Respected metamethods
288 ---------------------
290 Regarding the behavior of the Lua functions and the C functions and macros
291 provided by this extension, an existing ``__index`` metamethod will be
292 respected automatically. An existing ``__ipairs`` metamethod, however, takes
293 precedence.
295 If the ``__ipairs`` field of a value's metatable is set, then it must always
296 refer to a function. When starting iteration over a value with such a
297 metamethod being set, then this function is called with ``self`` (i.e. the
298 value itself) passed as first argument. The return values of the ``__ipairs``
299 metamethod may take one of the following 4 forms:
301 * ``return function_or_callable, static_argument, startindex`` causes the three
302 arguments to be returned by ``ipairs`` without further modification. Using
303 the C macros and functions for iteration, the behavior is according to the
304 generic loop statement in Lua:
305 ``for i, v in function_or_callable, static_argument, startindex do ... end``
306 * ``return "raw", table`` will result in iteration over the table ``table``
307 using ``lua_rawgeti``
308 * ``return "index", table_or_userdata`` will result in iteration over the table
309 or userdata while respecting any ``__index`` metamethod of the table or
310 userdata value
311 * ``return "call", function_or_callable`` will use the callable value as
312 (function) iterator where the function is expected to return a single value
313 without any index (the index is inserted automatically when using the
314 ``ipairs`` function for iteration)
316 These possiblities are demonstrated by the following example code:
318 require "seqlua"
320 do
321 local function ipairsaux(t, i)
322 i = i + 1
323 if i <= 3 then
324 return i, t[i]
325 end
326 end
327 custom = setmetatable(
328 {"one", "two", "three", "four", "five"},
329 {
330 __ipairs = function(self)
331 return ipairsaux, self, 0
332 end
333 }
334 )
335 end
336 print(string.concat(",", custom))
337 -- prints: one,two,three
338 -- (note that "four" and "five" are not printed)
340 tbl = {"alpha", "beta"}
342 proxy1 = setmetatable({}, {__index = tbl})
343 for i, v in ipairs(proxy1) do print(i, v) end
344 -- prints:
345 -- 1 alpha
346 -- 2 beta
348 proxy2 = setmetatable({}, {
349 __ipairs = function(self)
350 return "index", proxy1
351 end
352 })
353 for i, v in ipairs(proxy2) do print(i, v) end
354 -- prints:
355 -- 1 alpha
356 -- 2 beta
357 print(proxy2[1])
358 -- prints: nil
360 cursor = setmetatable({
361 "alice", "bob", "charlie", pos=1
362 }, {
363 __call = function(self)
364 local value = self[self.pos]
365 if value == nil then
366 self.pos = 1
367 else
368 self.pos = self.pos + 1
369 end
370 return value
371 end,
372 __ipairs = function(self)
373 return "call", self
374 end
375 })
376 for i, v in ipairs(cursor) do print(i, v) end
377 -- prints:
378 -- 1 alice
379 -- 2 bob
380 -- 3 charlie
381 print(cursor())
382 -- prints: alice
383 for i, v in ipairs(cursor) do print(i, v) end
384 -- prints:
385 -- 1 bob
386 -- 2 charlie
387 -- (note that "alice" has been returned earlier)
389 coefficients = setmetatable({1.25, 3.14, 17.5}, {
390 __index = function(self) return 1 end,
391 __ipairs = function(self) return "raw", self end
392 })
393 for i, v in ipairs(coefficients) do print(i, v) end
394 -- prints:
395 -- 1 1.25
396 -- 2 3.14
397 -- 3 17.5
398 -- (note that iteration terminates even if coefficients[4] == 1)
399 print(coefficients[4])
400 -- prints: 1