seqlua
view README @ 48:facf29831f6f
Two changes in README
| author | jbe |
|---|---|
| date | Mon Aug 25 11:38:47 2014 +0200 (2014-08-25) |
| parents | 31a78781a1e0 |
| children | 598f61d93402 |
line source
1 seqlua: Extension for handling sequential data in Lua
2 =====================================================
4 This is an experimental package to extend Lua in the following manner:
6 * allow ``ipairs(seq)`` to accept either tables or functions (i.e function
7 iterators) as an argument,
8 * add a new function ``string.concat(separator, seq)`` that concats either
9 table entries or function return values,
10 * provide auxiliary C functions and macros to simplify iterating over both
11 tables and iterator functions with a generic statement.
13 Existing ``__ipairs`` or ``__index`` (but not ``__len``) metamethods are
14 respected by both the Lua functions and the C functions and macros. The
15 ``__ipairs`` metamethod takes precedence over ``__index``, while the
16 ``__len`` metamethod is never used.
18 Metamethod handling in detail is explained in the last section
19 ("Respected metamethods") at the bottom of this README.
23 Motivation
24 ----------
26 Sequential data (such as arrays or streams) is often represented in two
27 different ways:
29 * as an ordered set of values (usually implemented as an array in other
30 programming languages, or as a sequence in Lua: a table with numeric keys
31 {1..n} associated with a value each),
32 * as some sort of data stream (sometimes implemented as a class of objects
33 providing certain methods, or as an iterator function in Lua: a function that
34 returns the next value with every call, where nil indicates the end of the
35 stream).
37 Quite often, when functions work on sequential data, it shouldn't matter in
38 which form the sequential data is being provided to the function. As an
39 example, consider a function that is writing a sequence of strings to a file.
40 Such function could either be fed with an array of strings (a table with
41 numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
42 function in Lua).
44 A function in Lua that accepts a table, might look like as follows:
46 function write_lines(lines)
47 for i, line in ipairs(lines) do
48 io.stdout:write(line)
49 io.stdout:write("\n")
50 end
51 end
53 In contrast, a function in Lua that accepts an iterator function would have to
54 be implemented differently:
56 function write_lines(get_next_line)
57 for line in get_next_line do
58 io.stdout:write(line)
59 io.stdout:write("\n")
60 end
61 end
63 If one wanted to create a function that accepts either a sequence in form of a
64 table or an iterator function, then one might need to write:
66 do
67 local function write_line(line)
68 io.stdout:write(line)
69 io.stdout:write("\n")
70 end
71 function write_lines(lines)
72 if type(lines) == "function" then
73 for line in lines do
74 write_line(line)
75 end
76 else
77 for i, line in ipairs(lines) do
78 write_line(line)
79 end
80 end
81 end
82 end
84 Obviously, this isn't something we want to do in every function that accepts
85 sequential data. Therefore, we usually decide for one of the two first forms
86 and thus disallow the other possible representation of sequential data to be
87 passed to the function.
89 This extension, however, modifies Lua's ``ipairs`` statement in such way that
90 it automatically accepts either a table or an iterator function as argument.
91 Thus, the first of the three ``write_lines`` functions above will accept both
92 (table) sequences and (function) iterators.
94 In addition to the modification of ``ipairs``, it also provides C functions and
95 macros to iterate over values in the same manner as a generic loop statement
96 with ``ipairs`` would do.
98 Note that this extension doesn't aim to supersede Lua's concept of iterator
99 functions. While metamethods (see section "Respected metamethods" below) may be
100 used to customize iteration behavior on values, this extension isn't thought to
101 replace the common practice to use function closures as iterators. Consider the
102 following example:
104 local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
105 write_lines(result:get_column_entries("name"))
107 The ``get_column_entries`` method can return a simple function closure that
108 returns the next entry in the "name" column (returning ``nil`` to indicate the
109 end). Such a closure can then be passed to another function that iterates
110 through a sequence of values by invoking ``ipairs`` with the general for-loop
111 (as previously shown).
113 Where desired, it is also possible to use metamethods to customize iteration
114 behavior:
116 function print_rows(rows)
117 for i, row in ipairs(rows) do
118 print_row(row)
119 end
120 end
121 local result = sql_query("SELECT * FROM actor ORDER BY birthday")
122 assert(type(result) == "userdata")
124 -- we may rely on the ``__index`` or ``__ipairs`` metamethod to
125 -- iterate through all result rows here:
126 print_rows(result) -- no need to use ":rows()" or a similar syntax
128 -- but we can also still pass an individual set of result rows to the
129 -- print_rows function:
130 print_rows{result[1], result[#result]}
132 This extension, however, doesn't respect the ``__len`` metamethod due to the
133 following considerations:
135 * An efficient implementation where ``for i, v in ipairs(tbl) do ... end`` does
136 neither create a closure nor repeatedly evaluate ``#tbl`` seems to be
137 impossible.
138 * Respecting ``__len`` could be used to implement sparse arrays, but this would
139 require iterating functions to expect ``nil`` as a potential value. This may
140 lead to problems because ``nil`` is usually also used to indicate the absence
141 of a value.
143 Though, if such behavior is desired, it can still be implemented through the
144 ``__ipairs`` metamethod.
146 Unless manually done by the user in the ``__ipairs`` metamethod, the ``ipairs``
147 function as well as the corresponding C functions and macros provided by this
148 extension never create any closures or other values that need to be garbage
149 collected.
153 Lua part of the library
154 -----------------------
156 The modified ``ipairs(seq)`` and the new ``string.concat(sep, seq)`` functions
157 accept either a table or a function as ``seq``. This is demonstrated in the
158 following examples:
160 require "seqlua"
162 t = {"a", "b", "c"}
164 for i, v in ipairs(t) do
165 print(i, v)
166 end
167 -- prints:
168 -- 1 a
169 -- 2 b
170 -- 3 c
172 print(string.concat(",", t))
173 -- prints: a,b,c
175 function alphabet()
176 local letter = nil
177 return function()
178 if letter == nil then
179 letter = "a"
180 elseif letter == "z" then
181 return nil
182 else
183 letter = string.char(string.byte(letter) + 1)
184 end
185 return letter
186 end
187 end
189 for i, v in ipairs(alphabet()) do
190 print(i, v)
191 end
192 -- prints:
193 -- 1 a
194 -- 2 b
195 -- 3 c
196 -- ...
197 -- 25 y
198 -- 26 z
200 print(string.concat(",", alphabet()))
201 -- prints: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
203 function filter(f)
204 return function(seq)
205 return coroutine.wrap(function()
206 for i, v in ipairs(seq) do f(v) end
207 end)
208 end
209 end
211 alpha_beta_x = filter(function(v)
212 if v == "a" then
213 coroutine.yield("alpha")
214 elseif v == "b" then
215 coroutine.yield("beta")
216 elseif type(v) == "number" then
217 for i = 1, v do
218 coroutine.yield("X")
219 end
220 end
221 end)
223 print((","):concat(alpha_beta_x{"a", 3, "b", "c", "d"}))
224 -- prints: alpha,X,X,X,beta
226 print((","):concat(alpha_beta_x(alphabet())))
227 -- prints: alpha,beta
231 C part of the library
232 ---------------------
234 In ``seqlualib.h``, the following macro is defined:
236 #define seqlua_iterloop(L, iter, idx) \
237 for ( \
238 seqlua_iterinit((L), (iter), (idx)); \
239 seqlua_iternext(iter); \
240 )
242 and
244 #define seqlua_iterloopauto(L, iter, idx) \
245 for ( \
246 seqlua_iterinit((L), (iter), (idx)); \
247 seqlua_iternext(iter); \
248 lua_pop((L), 1) \
249 )
251 This macro allows iteration over either tables or iterator functions as the
252 following example function demonstrates:
254 int printcsv(lua_State *L) {
255 seqlua_Iterator iter;
256 seqlua_iterloop(L, &iter, 1) {
257 if (seqlua_itercount(&iter) > 1) fputs(",", stdout);
258 fputs(luaL_tolstring(L, -1, NULL), stdout);
259 // two values need to be popped (the value pushed by
260 // seqlua_iternext and the value pushed by luaL_tolstring)
261 lua_pop(L, 2);
262 }
263 fputs("\n", stdout);
264 return 0;
265 }
267 printcsv{"a", "b", "c"}
268 -- prints: a,b,c
270 printcsv(assert(io.open("testfile")):lines())
271 -- prints: line1,line2,... of "testfile"
273 NOTE: During iteration using ``seqlua_iterloop``, ``seqlua_iterloopauto``, or
274 ``seqlua_iterinit``, three extra elements are stored on the stack (additionally
275 to the value). These extra elements are removed automatically when the loop ends
276 (i.e. when ``seqlua_iternext`` returns zero). The value pushed onto the stack
277 for every iteration step has to be removed manually from the stack, unless
278 ``seqlua_iterloopauto`` is used.
282 Respected metamethods
283 ---------------------
285 Regarding the behavior of the Lua functions and the C functions and macros
286 provided by this extension, an existing ``__index`` metamethod will be
287 respected automatically. An existing ``__ipairs`` metamethod, however, takes
288 precedence.
290 If the ``__ipairs`` field of a value's metatable is set, then it must always
291 refer to a function. When starting iteration over a value with such a
292 metamethod being set, then this function is called with ``self`` (i.e. the
293 value itself) passed as first argument. The return values of the ``__ipairs``
294 metamethod may take one of the following 4 forms:
296 * ``return function_or_callable, static_argument, startindex`` causes the three
297 arguments to be returned by ``ipairs`` without further modification. Using
298 the C macros and functions for iteration, the behavior is according to the
299 generic loop statement in Lua:
300 ``for i, v in function_or_callable, static_argument, startindex do ... end``
301 * ``return "raw", table`` will result in iteration over the table ``table``
302 using ``lua_rawgeti``
303 * ``return "index", table_or_userdata`` will result in iteration over the table
304 or userdata while respecting any ``__index`` metamethod of the table or
305 userdata value
306 * ``return "call", function_or_callable`` will use the callable value as
307 (function) iterator where the function is expected to return a single value
308 without any index (the index is inserted automatically when using the
309 ``ipairs`` function for iteration)
311 These possiblities are demonstrated by the following example code:
313 require "seqlua"
315 do
316 local function ipairsaux(t, i)
317 i = i + 1
318 if i <= 3 then
319 return i, t[i]
320 end
321 end
322 custom = setmetatable(
323 {"one", "two", "three", "four", "five"},
324 {
325 __ipairs = function(self)
326 return ipairsaux, self, 0
327 end
328 }
329 )
330 end
331 print(string.concat(",", custom))
332 -- prints: one,two,three
333 -- (note that "four" and "five" are not printed)
335 tbl = {"alpha", "beta"}
337 proxy1 = setmetatable({}, {__index = tbl})
338 for i, v in ipairs(proxy1) do print(i, v) end
339 -- prints:
340 -- 1 alpha
341 -- 2 beta
343 proxy2 = setmetatable({}, {
344 __ipairs = function(self)
345 return "index", proxy1
346 end
347 })
348 for i, v in ipairs(proxy2) do print(i, v) end
349 -- prints:
350 -- 1 alpha
351 -- 2 beta
352 print(proxy2[1])
353 -- prints: nil
355 cursor = setmetatable({
356 "alice", "bob", "charlie", pos=1
357 }, {
358 __call = function(self)
359 local value = self[self.pos]
360 if value == nil then
361 self.pos = 1
362 else
363 self.pos = self.pos + 1
364 end
365 return value
366 end,
367 __ipairs = function(self)
368 return "call", self
369 end
370 })
371 for i, v in ipairs(cursor) do print(i, v) end
372 -- prints:
373 -- 1 alice
374 -- 2 bob
375 -- 3 charlie
376 print(cursor())
377 -- prints: alice
378 for i, v in ipairs(cursor) do print(i, v) end
379 -- prints:
380 -- 1 bob
381 -- 2 charlie
382 -- (note that "alice" has been returned earlier)
384 coefficients = setmetatable({1.25, 3.14, 17.5}, {
385 __index = function(self) return 1 end,
386 __ipairs = function(self) return "raw", self end
387 })
388 for i, v in ipairs(coefficients) do print(i, v) end
389 -- prints:
390 -- 1 1.25
391 -- 2 3.14
392 -- 3 17.5
393 -- (note that iteration terminates even if coefficients[4] == 1)
394 print(coefficients[4])
395 -- prints: 1