seqlua
view README @ 43:1a0ce77d9e4b
Removed (wrong) example in README
| author | jbe |
|---|---|
| date | Mon Aug 25 03:51:00 2014 +0200 (2014-08-25) |
| parents | a862f3246b48 |
| children | de5e7dd93863 |
line source
1 seqlua: Extension for handling sequential data in Lua
2 =====================================================
4 This is an experimental package to extend Lua in the following manner:
6 * allow ``ipairs(seq)`` to accept either tables or functions (i.e function
7 iterators) as an argument,
8 * add a new function ``string.concat(separator, seq)`` that concats either
9 table entries or function return values,
10 * provide auxiliary C functions and macros to simplify iterating over both
11 tables and iterator functions with a generic statement.
13 Existing ``__ipairs`` or ``__index`` (but not ``__len``) metamethods are
14 respected by both the Lua functions and the C functions and macros. The
15 ``__ipairs`` metamethod takes precedence over ``__index``, while the
16 ``__len`` metamethod is never used.
18 Metamethod handling in detail is explained in the last section
19 ("Respected metamethods") at the bottom of this README.
23 Motivation
24 ----------
26 Sequential data (such as arrays or streams) is often represented in two
27 different ways:
29 * as an ordered set of values (usually implemented as an array in other
30 programming languages, or as a sequence in Lua: a table with numeric keys
31 {1..n} associated with a value each),
32 * as some sort of data stream (sometimes implemented as a class of objects
33 providing certain methods, or as an iterator function in Lua: a function that
34 returns the next value with every call, where nil indicates the end of the
35 stream).
37 Quite often, when functions work on sequential data, it shouldn't matter in
38 which form the sequential data is being provided to the function. As an
39 example, consider a function that is writing a sequence of strings to a file.
40 Such function could either be fed with an array of strings (a table with
41 numeric keys in Lua) or with a (possibly infinite) stream of data (an iterator
42 function in Lua).
44 A function in Lua that accepts a table, might look like as follows:
46 function write_lines(lines)
47 for i, line in ipairs(lines) do
48 io.stdout:write(line)
49 io.stdout:write("\n")
50 end
51 end
53 In contrast, a function in Lua that accepts an iterator function would have to
54 be implemented differently:
56 function write_lines(get_next_line)
57 for line in get_next_line do
58 io.stdout:write(line)
59 io.stdout:write("\n")
60 end
61 end
63 If one wanted to create a function that accepts either a sequence in form of a
64 table or an iterator function, then one might need to write:
66 do
67 local function write_line(line)
68 io.stdout:write(line)
69 io.stdout:write("\n")
70 end
71 function write_lines(lines)
72 if type(lines) == "function" then
73 for line in lines do
74 write_line(line)
75 end
76 else
77 for i, line in ipairs(lines) do
78 write_line(line)
79 end
80 end
81 end
82 end
84 Obviously, this isn't something we want to do in every function that accepts
85 sequential data. Therefore, we usually decide for one of the two first forms
86 and therefore disallow the other possible representation of sequential data to
87 be passed to the function.
89 This extension, however, modifies Lua's ``ipairs`` statement in such way that
90 it automatically accepts either a table or an iterator function as argument.
91 Thus, the first of the three ``write_lines`` functions above will accept both
92 (table) sequences and (function) iterators.
94 In addition to the modification of ``ipairs``, it also provides C functions and
95 macros to iterate over values in the same manner as a generic loop statement
96 with ``ipairs`` would do.
98 Note that this extension doesn't aim to supersede Lua's concept of iterator
99 functions. While metamethods (see section "Respected metamethods" below) may be
100 used to customize iteration behavior on values, this extension isn't thought to
101 replace the common practice to use function closures as iterators. Consider the
102 following example:
104 local result = sql_query("SELECT * FROM actor ORDER BY birthdate")
105 write_lines(result:get_column_entries("name"))
107 The ``get_column_entries`` method can return a simple function closure that
108 returns the next entry in the "name" column (returning ``nil`` to indicate the
109 end). Such a closure can then be passed to another function that iterates
110 through a sequence of values by invoking ``ipairs`` with the general for-loop
111 (as previously shown).
113 Where desired, it is also possible to use metamethods to customize iteration
114 behavior. This extension, however, doesn't respect the ``__len`` metamethod due
115 to the following reasons:
117 * An efficient implementation where ``for i, v in ipairs(tbl) do ... end`` does
118 neither create a closure nor repeatedly evaluate ``#tbl`` seems to be
119 impossible.
120 * Respecting ``__len`` could be used to implement sparse arrays, but this would
121 require iterating functions to expect ``nil`` as a potential value. This may
122 lead to problems because ``nil`` is usually also used to indicate the absence
123 of a value.
125 Though, if such behavior is desired, it can still be implemented through the
126 ``__ipairs`` metamethod.
128 Unless manually done by the user in the ``__ipairs`` metamethod, this extension
129 never creates any closures or other values that need to be garbage collected.
133 Lua part of the library
134 -----------------------
136 The modified ``ipairs(seq)`` and the new ``string.concat(sep, seq)`` functions
137 accept either a table or a function as ``seq``. This is demonstrated in the
138 following examples:
140 require "seqlua"
142 t = {"a", "b", "c"}
144 for i, v in ipairs(t) do
145 print(i, v)
146 end
147 -- prints:
148 -- 1 a
149 -- 2 b
150 -- 3 c
152 print(string.concat(",", t))
153 -- prints: a,b,c
155 function alphabet()
156 local letter = nil
157 return function()
158 if letter == nil then
159 letter = "a"
160 elseif letter == "z" then
161 return nil
162 else
163 letter = string.char(string.byte(letter) + 1)
164 end
165 return letter
166 end
167 end
169 for i, v in ipairs(alphabet()) do
170 print(i, v)
171 end
172 -- prints:
173 -- 1 a
174 -- 2 b
175 -- 3 c
176 -- ...
177 -- 25 y
178 -- 26 z
180 print(string.concat(",", alphabet()))
181 -- prints: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
183 function filter(f)
184 return function(seq)
185 return coroutine.wrap(function()
186 for i, v in ipairs(seq) do f(v) end
187 end)
188 end
189 end
191 alpha_beta_x = filter(function(v)
192 if v == "a" then
193 coroutine.yield("alpha")
194 elseif v == "b" then
195 coroutine.yield("beta")
196 elseif type(v) == "number" then
197 for i = 1, v do
198 coroutine.yield("X")
199 end
200 end
201 end)
203 print((","):concat(alpha_beta_x{"a", 3, "b", "c", "d"}))
204 -- prints: alpha,X,X,X,beta
206 print((","):concat(alpha_beta_x(alphabet())))
207 -- prints: alpha,beta
211 C part of the library
212 ---------------------
214 In ``seqlualib.h``, the following macro is defined:
216 #define seqlua_iterloop(L, iter, idx) \
217 for ( \
218 seqlua_iterinit((L), (iter), (idx)); \
219 seqlua_iternext(iter); \
220 )
222 and
224 #define seqlua_iterloopauto(L, iter, idx) \
225 for ( \
226 seqlua_iterinit((L), (iter), (idx)); \
227 seqlua_iternext(iter); \
228 lua_pop((L), 1) \
229 )
231 This macro allows iteration over either tables or iterator functions as the
232 following example function demonstrates:
234 int printcsv(lua_State *L) {
235 seqlua_Iterator iter;
236 seqlua_iterloop(L, &iter, 1) {
237 if (seqlua_itercount(&iter) > 1) fputs(",", stdout);
238 fputs(luaL_tolstring(L, -1, NULL), stdout);
239 // two values need to be popped (the value pushed by
240 // seqlua_iternext and the value pushed by luaL_tolstring)
241 lua_pop(L, 2);
242 }
243 fputs("\n", stdout);
244 return 0;
245 }
247 printcsv{"a", "b", "c"}
248 -- prints: a,b,c
250 printcsv(assert(io.open("testfile")):lines())
251 -- prints: line1,line2,... of "testfile"
253 NOTE: During iteration using ``seqlua_iterloop``, ``seqlua_iterloopauto``, or
254 ``seqlua_iterinit``, three extra elements are stored on the stack (additionally
255 to the value). These extra elements are removed automatically when the loop ends
256 (i.e. when ``seqlua_iternext`` returns zero). The value pushed onto the stack
257 for every iteration step has to be removed manually from the stack, unless
258 ``seqlua_iterloopauto`` is used.
262 Respected metamethods
263 ---------------------
265 Regarding the behavior of the Lua functions and the C functions and macros
266 provided by this extension, an existing ``__index`` metamethod will be
267 respected automatically. An existing ``__ipairs`` metamethod, however, takes
268 precedence.
270 If the ``__ipairs`` field of a value's metatable is set, then it must always
271 refer to a function. When starting iteration over a value with such a
272 metamethod being set, then this function is called with ``self`` (i.e. the
273 value itself) passed as first argument. The return values of the ``__ipairs``
274 metamethod may take one of the following 4 forms:
276 * ``return function_or_callable, static_argument, startindex`` causes the three
277 arguments to be returned by ``ipairs`` without further modification. Using
278 the C macros and functions for iteration, the behavior is according to the
279 generic loop statement in Lua:
280 ``for i, v in function_or_callable, static_argument, startindex do ... end``
281 * ``return "raw", table`` will result in iteration over the table ``table``
282 using ``lua_rawgeti``
283 * ``return "index", table_or_userdata`` will result in iteration over the table
284 or userdata while respecting any ``__index`` metamethod of the table or
285 userdata value
286 * ``return "call", function_or_callable`` will use the callable value as
287 (function) iterator where the function is expected to return a single value
288 without any index (the index is inserted automatically when using the
289 ``ipairs`` function for iteration)
291 These possiblities are demonstrated by the following example code:
293 require "seqlua"
295 do
296 local function ipairsaux(t, i)
297 i = i + 1
298 if i <= 3 then
299 return i, t[i]
300 end
301 end
302 custom = setmetatable(
303 {"one", "two", "three", "four", "five"},
304 {
305 __ipairs = function(self)
306 return ipairsaux, self, 0
307 end
308 }
309 )
310 end
311 print(string.concat(",", custom))
312 -- prints: one,two,three
313 -- (note that "four" and "five" are not printed)
315 tbl = {"alpha", "beta"}
317 proxy1 = setmetatable({}, {__index = tbl})
318 for i, v in ipairs(proxy1) do print(i, v) end
319 -- prints:
320 -- 1 alpha
321 -- 2 beta
323 proxy2 = setmetatable({}, {
324 __ipairs = function(self)
325 return "index", proxy1
326 end
327 })
328 for i, v in ipairs(proxy2) do print(i, v) end
329 -- prints:
330 -- 1 alpha
331 -- 2 beta
332 print(proxy2[1])
333 -- prints: nil
335 cursor = setmetatable({
336 "alice", "bob", "charlie", pos=1
337 }, {
338 __call = function(self)
339 local value = self[self.pos]
340 if value == nil then
341 self.pos = 1
342 else
343 self.pos = self.pos + 1
344 end
345 return value
346 end,
347 __ipairs = function(self)
348 return "call", self
349 end
350 })
351 for i, v in ipairs(cursor) do print(i, v) end
352 -- prints:
353 -- 1 alice
354 -- 2 bob
355 -- 3 charlie
356 print(cursor())
357 -- prints: alice
358 for i, v in ipairs(cursor) do print(i, v) end
359 -- prints:
360 -- 1 bob
361 -- 2 charlie
362 -- (note that "alice" has been returned earlier)
364 coefficients = setmetatable({1.25, 3.14, 17.5}, {
365 __index = function(self) return 1 end,
366 __ipairs = function(self) return "raw", self end
367 })
368 for i, v in ipairs(coefficients) do print(i, v) end
369 -- prints:
370 -- 1 1.25
371 -- 2 3.14
372 -- 3 17.5
373 -- (note that iteration terminates even if coefficients[4] == 1)
374 print(coefficients[4])
375 -- prints: 1