Iterator Scoping

Cleanup of async resources is special in that it may require an active event loop. Since asynchronous iterators can hold resources indefinitely, they should be cleaned up deterministically whenever possible (see PEP 533 for discussion). Thus, asyncstdlib defaults to deterministic cleanup but provides tools to explicitly manage the lifetime of iterators.

Cleanup in asyncstdlib

All async iterators of asyncstdlib that work on other iterators assume sole ownership of the iterators passed to them. Passed in async iterators are guaranteed to be aclose()d as soon as the asyncstdlib async iterator itself is cleaned up. This provides a resource-safe default for the most common operation of exhausting iterators.

>>> import asyncio
>>> import asyncstdlib as a
>>>
>>> async def async_squares(i=0):
...     """Provide an infinite stream of squared numbers"""
...     while True:
...         await asyncio.sleep(0.1)
...         yield i**2
...         i += 1
...
>>> async def main():
...     async_iter = async_squares()
...     # loop until we are done
...     async for i, s in a.zip(range(5), async_iter):
...         print(f"{i}: {s}")
...     assert await a.anext(async_iter, "Closed!") == "Closed!"
...
>>> asyncio.run(main())

For consistency, every asyncstdlib async iterator performs such cleanup. This may be unexpected for async variants of iterator utilities that are usually applied multiple times, such as itertools.islice(). Thus, to manage the lifetime of async iterators one can explicitly scope them.

Scoping async iterator lifetime

In order to use a single async iterator across several iterations but guarantee cleanup, the iterator can be scoped to an async with block: using asyncstdlib.scoped_iter() creates an async iterator that is guaranteed to aclose() at the end of the block, but cannot be closed before.

>>> import asyncio
>>> import asyncstdlib as a
>>>
>>> async def async_squares(i=0):
...     """Provide an infinite stream of squared numbers"""
...     while True:
...         await asyncio.sleep(0.1)
...         yield i**2
...         i += 1
...
>>> async def main():
...     # iterator can be re-used in the async with block
...     async with a.scoped_iter(async_squares()) as async_iter:
...         async for s in a.islice(async_iter, 3):
...             print(f"1st Batch: {s}")
...         # async_iter is still open for further iteration
...         async for s in a.islice(async_iter, 3):
...             print(f"2nd Batch: {s}")
...         async for s in a.islice(async_iter, 3):
...             print(f"3rd Batch: {s}")
...     # iterator is closed after the async with block
...     assert await a.anext(async_iter, "Closed!") == "Closed!"
...
>>> asyncio.run(main())

Scoped iterators should be the go-to approach for managing iterator lifetimes. However, not all lifetimes correspond to well-defined lexical scopes; for these cases, one can borrow an iterator instead.