seqlua
view README @ 57:da4b9d6a5b7e
Fixed bug due to problem combining luaL_buffinit with seqlua_iterinit
| author | jbe |
|---|---|
| date | Tue Jan 05 04:26:49 2016 +0100 (2016-01-05) |
| parents | c3976eacc6ab |
| children |
line source
1 seqlua: Extension for handling sequential data in Lua
2 =====================================================
4 This package is an experimental extension for the Lua 5.2 programming language
5 which:
7 * allows ``ipairs(seq)`` to accept either tables or functions (i.e function
8 iterators) as an argument,
9 * adds a new function ``string.concat(separator, seq)`` that concats either
10 table entries or function return values,
11 * provides auxiliary C functions and macros to simplify iterating over both
12 tables and iterator functions with a generic statement.
14 Existing ``__ipairs`` or ``__index`` (but not ``__len``) metamethods are
15 respected by both the Lua functions and the C functions and macros. The
16 ``__ipairs`` metamethod takes precedence over ``__index``, while the
17 ``__len`` metamethod is never used.
19 Metamethod handling in detail is explained in the last section
20 ("Respected metamethods") at the bottom of this README.
22 In Lua, this extension is loaded by ``require "seqlua"``. In order to use the
23 auxiliary C functions and macros, add ``#include <seqlualib.h>`` to your C file
24 and ensure that the functions implemented in ``seqlualib.c`` are statically or
25 dynamically linked with your C Lua library.
29 Motivation
30 ----------
32 Sequential data (such as arrays or streams) is often represented in two
33 different ways:
35 * as an ordered set of values (usually implemented as an array in other
36 programming languages, or as a sequence in Lua: a table with numeric keys
37 {1..n} associated with a value each),
38 * as some sort of data stream (sometimes implemented as a class of objects
39 providing certain methods, or as an iterator function in Lua: a function that
40 returns the next value with every call, where nil indicates the end of the
41 stream).
43 Quite often, when functions work on sequential data, it shouldn't matter in
44 which form the sequential data is being provided to the function. As an
45 example, consider a function that is writing a sequence of strings to a file.
46 Such function could either be fed with an array of strings (a table with
47 numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
48 function in Lua).
50 A function in Lua that accepts a table, might look like as follows:
52 function write_lines(lines)
53 for i, line in ipairs(lines) do
54 io.stdout:write(line)
55 io.stdout:write("\n")
56 end
57 end
59 In contrast, a function in Lua that accepts an iterator function would have to
60 be implemented differently:
62 function write_lines(get_next_line)
63 for line in get_next_line do
64 io.stdout:write(line)
65 io.stdout:write("\n")
66 end
67 end
69 If one wanted to create a function that accepts either a sequence in form of a
70 table or an iterator function, then one might need to write:
72 do
73 local function write_line(line)
74 io.stdout:write(line)
75 io.stdout:write("\n")
76 end
77 function write_lines(lines)
78 if type(lines) == "function" then
79 for line in lines do
80 write_line(line)
81 end
82 else
83 for i, line in ipairs(lines) do
84 write_line(line)
85 end
86 end
87 end
88 end
90 Obviously, this isn't something we want to do in every function that accepts
91 sequential data. Therefore, we usually decide for one of the two first forms
92 and thus disallow the other possible representation of sequential data to be
93 passed to the function.
95 This extension, however, modifies Lua's ``ipairs`` statement in such way that
96 it automatically accepts either a table or an iterator function as argument.
97 Thus, the first of the three ``write_lines`` functions above will accept both
98 (table) sequences and (function) iterators.
100 In addition to the modification of ``ipairs``, it also provides C functions and
101 macros to iterate over values in the same manner as a generic loop statement
102 with ``ipairs`` would do.
104 This extension doesn't aim to supersede Lua's concept of iterator functions.
105 While metamethods (see section "Respected metamethods" below) may be used to
106 customize iteration behavior on values, this extension isn't thought to replace
107 the common practice to use function closures as iterators. Consider the
108 following example:
110 function write_lines(lines)
111 for i, line in ipairs(lines) do
112 io.stdout:write(line)
113 io.stdout:write("\n")
114 end
115 end
116 local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
117 -- assert(type(result:get_column_entries("name")) == "function")
118 write_lines(result:get_column_entries("name"))
120 Note, however, that in case of repeated or nested loops, using function
121 iterators may not be feasible:
123 function print_list_twice(seq)
124 for i = 1, 2 do
125 for i, v in ipairs(seq) do
126 print(v)
127 end
128 end
129 end
130 print_list_twice(io.stdin:lines()) -- won't work as expected
132 Where desired, it is possible to use metamethods to customize iteration
133 behavior:
135 function print_rows(rows)
136 for i, row in ipairs(rows) do
137 print_row(row)
138 end
139 end
140 local result = sql_query("SELECT * FROM actor ORDER BY birthday")
141 assert(type(result) == "userdata")
143 -- we may rely on the ``__index`` or ``__ipairs`` metamethod to
144 -- iterate through all result rows here:
145 print_rows(result) -- no need to use ":rows()" or a similar syntax
147 -- but we can also still pass an individual set of result rows to the
148 -- print_rows function:
149 print_rows{result[1], result[#result]}
151 This extension, however, doesn't respect the ``__len`` metamethod due to the
152 following considerations:
154 * An efficient implementation where ``for i, v in ipairs(tbl) do ... end`` does
155 neither create a closure nor repeatedly evaluate ``#tbl`` seems to be
156 impossible.
157 * Respecting ``__len`` could be used to implement sparse arrays, but this would
158 require iterating functions to expect ``nil`` as a potential value. This may
159 lead to problems because ``nil`` is usually also used to indicate the absence
160 of a value.
162 Though, if such behavior is desired, it can still be implemented through the
163 ``__ipairs`` metamethod.
165 Unless manually done by the user in the ``__ipairs`` metamethod, the ``ipairs``
166 function as well as the corresponding C functions and macros provided by this
167 extension never create any closures or other values that need to be garbage
168 collected.
172 Lua part of the library
173 -----------------------
175 The modified ``ipairs(seq)`` and the new ``string.concat(sep, seq)`` functions
176 accept either a table or a function as ``seq``. This is demonstrated in the
177 following examples:
179 require "seqlua"
181 t = {"a", "b", "c"}
183 for i, v in ipairs(t) do
184 print(i, v)
185 end
186 -- prints:
187 -- 1 a
188 -- 2 b
189 -- 3 c
191 print(string.concat(",", t))
192 -- prints: a,b,c
194 function alphabet()
195 local letter = nil
196 return function()
197 if letter == nil then
198 letter = "a"
199 elseif letter == "z" then
200 return nil
201 else
202 letter = string.char(string.byte(letter) + 1)
203 end
204 return letter
205 end
206 end
208 for i, v in ipairs(alphabet()) do
209 print(i, v)
210 end
211 -- prints:
212 -- 1 a
213 -- 2 b
214 -- 3 c
215 -- ...
216 -- 25 y
217 -- 26 z
219 print(string.concat(",", alphabet()))
220 -- prints: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
222 function filter(f)
223 return function(seq)
224 return coroutine.wrap(function()
225 for i, v in ipairs(seq) do f(v) end
226 end)
227 end
228 end
230 alpha_beta_x = filter(function(v)
231 if v == "a" then
232 coroutine.yield("alpha")
233 elseif v == "b" then
234 coroutine.yield("beta")
235 elseif type(v) == "number" then
236 for i = 1, v do
237 coroutine.yield("X")
238 end
239 end
240 end)
242 print((","):concat(alpha_beta_x{"a", 3, "b", "c", "d"}))
243 -- prints: alpha,X,X,X,beta
245 print((","):concat(alpha_beta_x(alphabet())))
246 -- prints: alpha,beta
250 C part of the library
251 ---------------------
253 In ``seqlualib.h``, the following macro is defined:
255 #define seqlua_iterloop(L, iter, idx) \
256 for ( \
257 seqlua_iterinit((L), (iter), (idx)); \
258 seqlua_iternext(iter); \
259 )
261 and
263 #define seqlua_iterloopauto(L, iter, idx) \
264 for ( \
265 seqlua_iterinit((L), (iter), (idx)); \
266 seqlua_iternext(iter); \
267 lua_pop((L), 1) \
268 )
270 This macro allows iteration over either tables or iterator functions as the
271 following example function demonstrates:
273 int printcsv(lua_State *L) {
274 seqlua_Iterator iter;
275 seqlua_iterloop(L, &iter, 1) {
276 if (seqlua_itercount(&iter) > 1) fputs(",", stdout);
277 fputs(luaL_tolstring(L, -1, NULL), stdout);
278 // two values need to be popped (the value pushed by
279 // seqlua_iternext and the value pushed by luaL_tolstring)
280 lua_pop(L, 2);
281 }
282 fputs("\n", stdout);
283 return 0;
284 }
286 printcsv{"a", "b", "c"}
287 -- prints: a,b,c
289 printcsv(assert(io.open("testfile")):lines())
290 -- prints: line1,line2,... of "testfile"
292 NOTE: During iteration using ``seqlua_iterloop``, ``seqlua_iterloopauto``, or
293 ``seqlua_iterinit``, three extra elements are stored on the stack (additionally
294 to the value). These extra elements are removed automatically when the loop ends
295 (i.e. when ``seqlua_iternext`` returns zero). The value pushed onto the stack
296 for every iteration step has to be removed manually from the stack, unless
297 ``seqlua_iterloopauto`` is used.
301 Respected metamethods
302 ---------------------
304 Regarding the behavior of the Lua functions and the C functions and macros
305 provided by this extension, an existing ``__index`` metamethod will be
306 respected automatically. An existing ``__ipairs`` metamethod, however, takes
307 precedence.
309 If the ``__ipairs`` field of a value's metatable is set, then it must always
310 refer to a function. When starting iteration over a value with such a
311 metamethod being set, then this function is called with ``self`` (i.e. the
312 value itself) passed as first argument. The return values of the ``__ipairs``
313 metamethod may take one of the following 4 forms:
315 * ``return function_or_callable, static_argument, startindex`` causes the three
316 arguments to be returned by ``ipairs`` without further modification. Using
317 the C macros and functions for iteration, the behavior is according to the
318 generic loop statement in Lua:
319 ``for i, v in function_or_callable, static_argument, startindex do ... end``
320 * ``return "raw", table`` will result in iteration over the table ``table``
321 using ``lua_rawgeti``
322 * ``return "index", table_or_userdata`` will result in iteration over the table
323 or userdata while respecting any ``__index`` metamethod of the table or
324 userdata value
325 * ``return "call", function_or_callable`` will use the callable value as
326 (function) iterator where the function is expected to return a single value
327 without any index (the index is inserted automatically when using the
328 ``ipairs`` function for iteration)
330 These possiblities are demonstrated by the following example code:
332 require "seqlua"
334 do
335 local function ipairsaux(t, i)
336 i = i + 1
337 if i <= 3 then
338 return i, t[i]
339 end
340 end
341 custom = setmetatable(
342 {"one", "two", "three", "four", "five"},
343 {
344 __ipairs = function(self)
345 return ipairsaux, self, 0
346 end
347 }
348 )
349 end
350 print(string.concat(",", custom))
351 -- prints: one,two,three
352 -- (note that "four" and "five" are not printed)
354 tbl = {"alpha", "beta"}
356 proxy1 = setmetatable({}, {__index = tbl})
357 for i, v in ipairs(proxy1) do print(i, v) end
358 -- prints:
359 -- 1 alpha
360 -- 2 beta
362 proxy2 = setmetatable({}, {
363 __ipairs = function(self)
364 return "index", proxy1
365 end
366 })
367 for i, v in ipairs(proxy2) do print(i, v) end
368 -- prints:
369 -- 1 alpha
370 -- 2 beta
371 print(proxy2[1])
372 -- prints: nil
374 cursor = setmetatable({
375 "alice", "bob", "charlie", pos=1
376 }, {
377 __call = function(self)
378 local value = self[self.pos]
379 if value == nil then
380 self.pos = 1
381 else
382 self.pos = self.pos + 1
383 end
384 return value
385 end,
386 __ipairs = function(self)
387 return "call", self
388 end
389 })
390 for i, v in ipairs(cursor) do print(i, v) end
391 -- prints:
392 -- 1 alice
393 -- 2 bob
394 -- 3 charlie
395 print(cursor())
396 -- prints: alice
397 for i, v in ipairs(cursor) do print(i, v) end
398 -- prints:
399 -- 1 bob
400 -- 2 charlie
401 -- (note that "alice" has been returned earlier)
403 coefficients = setmetatable({1.25, 3.14, 17.5}, {
404 __index = function(self) return 1 end,
405 __ipairs = function(self) return "raw", self end
406 })
407 for i, v in ipairs(coefficients) do print(i, v) end
408 -- prints:
409 -- 1 1.25
410 -- 2 3.14
411 -- 3 17.5
412 -- (note that iteration terminates even if coefficients[4] == 1)
413 print(coefficients[4])
414 -- prints: 1