seqlua
view README @ 41:54647a56a47a
Fixed third example of "write_lines" in README
| author | jbe |
|---|---|
| date | Mon Aug 25 03:42:34 2014 +0200 (2014-08-25) |
| parents | 21230b38d858 |
| children | a862f3246b48 |
line source
1 seqlua: Extension for handling sequential data in Lua
2 =====================================================
4 This is an experimental package to extend Lua in the following manner:
6 * allow ``ipairs(seq)`` to accept either tables or functions (i.e function
7 iterators) as an argument,
8 * add a new function ``string.concat(separator, seq)`` that concats either
9 table entries or function return values,
10 * provide auxiliary C functions and macros to simplify iterating over both
11 tables and iterator functions with a generic statement.
13 Existing ``__ipairs`` or ``__index`` (but not ``__len``) metamethods are
14 respected by both the Lua functions and the C functions and macros. The
15 ``__ipairs`` metamethod takes precedence over ``__index``, while the
16 ``__len`` metamethod is never used.
18 Metamethod handling in detail is explained in the last section
19 ("Respected metamethods") at the bottom of this README.
23 Motivation
24 ----------
26 Sequential data (such as arrays or streams) is often represented in two
27 different ways:
29 * as an ordered set of values (usually implemented as an array in other
30 programming languages, or as a sequence in Lua: a table with numeric keys
31 {1..n} associated with a value each),
32 * as some sort of data stream (sometimes implemented as a class of objects
33 providing certain methods, or as an iterator function in Lua: a function that
34 returns the next value with every call, where nil indicates the end of the
35 stream).
37 Quite often, when functions work on sequential data, it shouldn't matter in
38 which form the sequential data is being provided to the function. As an
39 example, consider a function that is writing a sequence of strings to a file.
40 Such function could either be fed with an array of strings (a table with
41 numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
42 function in Lua).
44 A function in Lua that accepts a table, might look like as follows:
46 function write_lines(lines)
47 for i, line in ipairs(lines) do
48 io.stdout:write(line)
49 io.stdout:write("\n")
50 end
51 end
53 In contrast, a function in Lua that accepts an iterator function would have to
54 be implemented differently:
56 function write_lines(get_next_line)
57 for line in get_next_line do
58 io.stdout:write(line)
59 io.stdout:write("\n")
60 end
61 end
63 If one wanted to create a function that accepts either a sequence in form of a
64 table or an iterator function, then one might need to write:
66 do
67 local function write_line(line)
68 io.stdout:write(line)
69 io.stdout:write("\n")
70 end
71 function write_lines(lines)
72 if type(lines) == "function" then
73 for line in lines do
74 write_line(line)
75 end
76 else
77 for i, line in ipairs(lines) do
78 write_line(line)
79 end
80 end
81 end
82 end
84 Obviously, this isn't something we want to do in every function that accepts
85 sequential data. Therefore, we usually decide for one of the two first forms
86 and therefore disallow the other possible representation of sequential data to
87 be passed to the function.
89 This extension, however, modifies Lua's ``ipairs`` statement in such way that
90 it automatically accepts either a table or an iterator function as argument.
91 Thus, the first of the three functions above will accept both (table) sequences
92 and (function) iterators.
94 In addition to the modification of ``ipairs``, it also provides C functions and
95 macros to iterate over values in the same manner as a generic loop statement
96 with ``ipairs`` would do.
98 Note that this extension doesn't aim to supersede Lua's concept of iterator
99 functions. While metamethods (see section "Respected metamethods" below) may be
100 used to customize iteration behavior on values, this extension isn't thought to
101 replace the common practice to use function closures as iterators. Consider the
102 following example:
104 local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
105 write_lines(result:get_column_entries("name"))
107 The ``get_column_entries`` method can return a simple function closure that
108 returns the next entry in the "name" column (returning ``nil`` to indicate the
109 end). Such a closure can then be passed to another function that iterates
110 through a sequence of values by invoking ``ipairs`` with the general for-loop
111 (as previously shown).
113 Where desired, it is also possible to use metamethods to customize iteration
114 behavior. Consider:
116 function process_row(database_row)
117 -- some function
118 end
119 local result = sql_query("SELECT * FROM actor")
120 process_row(result) -- without writing :rows() or similar
122 This extension, however, doesn't respect the ``__len`` metamethod due to the
123 following reasons:
125 * An efficient implementation where ``for i, v in ipairs(tbl) do ... end`` does
126 neither create a closure nor repeatedly evaluate ``#tbl`` seems to be
127 impossible.
128 * Respecting ``__len`` could be used to implement sparse arrays, but this would
129 require iterating functions to expect ``nil`` as a potential value. This may
130 lead to problems because ``nil`` is usually also used to indicate the absence
131 of a value.
133 Though, if such behavior is desired, it can still be implemented through the
134 ``__ipairs`` metamethod.
136 Unless manually done by the user in the ``__ipairs`` metamethod, this extension
137 never creates any closures or other values that need to be garbage collected.
141 Lua part of the library
142 -----------------------
144 The modified ``ipairs(seq)`` and the new ``string.concat(sep, seq)`` functions
145 accept either a table or a function as ``seq``. This is demonstrated in the
146 following examples:
148 require "seqlua"
150 t = {"a", "b", "c"}
152 for i, v in ipairs(t) do
153 print(i, v)
154 end
155 -- prints:
156 -- 1 a
157 -- 2 b
158 -- 3 c
160 print(string.concat(",", t))
161 -- prints: a,b,c
163 function alphabet()
164 local letter = nil
165 return function()
166 if letter == nil then
167 letter = "a"
168 elseif letter == "z" then
169 return nil
170 else
171 letter = string.char(string.byte(letter) + 1)
172 end
173 return letter
174 end
175 end
177 for i, v in ipairs(alphabet()) do
178 print(i, v)
179 end
180 -- prints:
181 -- 1 a
182 -- 2 b
183 -- 3 c
184 -- ...
185 -- 25 y
186 -- 26 z
188 print(string.concat(",", alphabet()))
189 -- prints: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
191 function filter(f)
192 return function(seq)
193 return coroutine.wrap(function()
194 for i, v in ipairs(seq) do f(v) end
195 end)
196 end
197 end
199 alpha_beta_x = filter(function(v)
200 if v == "a" then
201 coroutine.yield("alpha")
202 elseif v == "b" then
203 coroutine.yield("beta")
204 elseif type(v) == "number" then
205 for i = 1, v do
206 coroutine.yield("X")
207 end
208 end
209 end)
211 print((","):concat(alpha_beta_x{"a", 3, "b", "c", "d"}))
212 -- prints: alpha,X,X,X,beta
214 print((","):concat(alpha_beta_x(alphabet())))
215 -- prints: alpha,beta
219 C part of the library
220 ---------------------
222 In ``seqlualib.h``, the following macro is defined:
224 #define seqlua_iterloop(L, iter, idx) \
225 for ( \
226 seqlua_iterinit((L), (iter), (idx)); \
227 seqlua_iternext(iter); \
228 )
230 and
232 #define seqlua_iterloopauto(L, iter, idx) \
233 for ( \
234 seqlua_iterinit((L), (iter), (idx)); \
235 seqlua_iternext(iter); \
236 lua_pop((L), 1) \
237 )
239 This macro allows iteration over either tables or iterator functions as the
240 following example function demonstrates:
242 int printcsv(lua_State *L) {
243 seqlua_Iterator iter;
244 seqlua_iterloop(L, &iter, 1) {
245 if (seqlua_itercount(&iter) > 1) fputs(",", stdout);
246 fputs(luaL_tolstring(L, -1, NULL), stdout);
247 // two values need to be popped (the value pushed by
248 // seqlua_iternext and the value pushed by luaL_tolstring)
249 lua_pop(L, 2);
250 }
251 fputs("\n", stdout);
252 return 0;
253 }
255 printcsv{"a", "b", "c"}
256 -- prints: a,b,c
258 printcsv(assert(io.open("testfile")):lines())
259 -- prints: line1,line2,... of "testfile"
261 NOTE: During iteration using ``seqlua_iterloop``, ``seqlua_iterloopauto``, or
262 ``seqlua_iterinit``, three extra elements are stored on the stack (additionally
263 to the value). These extra elements are removed automatically when the loop ends
264 (i.e. when ``seqlua_iternext`` returns zero). The value pushed onto the stack
265 for every iteration step has to be removed manually from the stack, unless
266 ``seqlua_iterloopauto`` is used.
270 Respected metamethods
271 ---------------------
273 Regarding the behavior of the Lua functions and the C functions and macros
274 provided by this extension, an existing ``__index`` metamethod will be
275 respected automatically. An existing ``__ipairs`` metamethod, however, takes
276 precedence.
278 If the ``__ipairs`` field of a value's metatable is set, then it must always
279 refer to a function. When starting iteration over a value with such a
280 metamethod being set, then this function is called with ``self`` (i.e. the
281 value itself) passed as first argument. The return values of the ``__ipairs``
282 metamethod may take one of the following 4 forms:
284 * ``return function_or_callable, static_argument, startindex`` causes the three
285 arguments to be returned by ``ipairs`` without further modification. Using
286 the C macros and functions for iteration, the behavior is according to the
287 generic loop statement in Lua:
288 ``for i, v in function_or_callable, static_argument, startindex do ... end``
289 * ``return "raw", table`` will result in iteration over the table ``table``
290 using ``lua_rawgeti``
291 * ``return "index", table_or_userdata`` will result in iteration over the table
292 or userdata while respecting any ``__index`` metamethod of the table or
293 userdata value
294 * ``return "call", function_or_callable`` will use the callable value as
295 (function) iterator where the function is expected to return a single value
296 without any index (the index is inserted automatically when using the
297 ``ipairs`` function for iteration)
299 These possiblities are demonstrated by the following example code:
301 require "seqlua"
303 do
304 local function ipairsaux(t, i)
305 i = i + 1
306 if i <= 3 then
307 return i, t[i]
308 end
309 end
310 custom = setmetatable(
311 {"one", "two", "three", "four", "five"},
312 {
313 __ipairs = function(self)
314 return ipairsaux, self, 0
315 end
316 }
317 )
318 end
319 print(string.concat(",", custom))
320 -- prints: one,two,three
321 -- (note that "four" and "five" are not printed)
323 tbl = {"alpha", "beta"}
325 proxy1 = setmetatable({}, {__index = tbl})
326 for i, v in ipairs(proxy1) do print(i, v) end
327 -- prints:
328 -- 1 alpha
329 -- 2 beta
331 proxy2 = setmetatable({}, {
332 __ipairs = function(self)
333 return "index", proxy1
334 end
335 })
336 for i, v in ipairs(proxy2) do print(i, v) end
337 -- prints:
338 -- 1 alpha
339 -- 2 beta
340 print(proxy2[1])
341 -- prints: nil
343 cursor = setmetatable({
344 "alice", "bob", "charlie", pos=1
345 }, {
346 __call = function(self)
347 local value = self[self.pos]
348 if value == nil then
349 self.pos = 1
350 else
351 self.pos = self.pos + 1
352 end
353 return value
354 end,
355 __ipairs = function(self)
356 return "call", self
357 end
358 })
359 for i, v in ipairs(cursor) do print(i, v) end
360 -- prints:
361 -- 1 alice
362 -- 2 bob
363 -- 3 charlie
364 print(cursor())
365 -- prints: alice
366 for i, v in ipairs(cursor) do print(i, v) end
367 -- prints:
368 -- 1 bob
369 -- 2 charlie
370 -- (note that "alice" has been returned earlier)
372 coefficients = setmetatable({1.25, 3.14, 17.5}, {
373 __index = function(self) return 1 end,
374 __ipairs = function(self) return "raw", self end
375 })
376 for i, v in ipairs(coefficients) do print(i, v) end
377 -- prints:
378 -- 1 1.25
379 -- 2 3.14
380 -- 3 17.5
381 -- (note that iteration terminates even if coefficients[4] == 1)
382 print(coefficients[4])
383 -- prints: 1