Introduction

Pescador is a library for streaming (numerical) data for use in iterative machine learning applications.

The core concept is the Streamer object, which encapsulates a Python generator to allow for re-use and inter-process communication.

The basic use case is as follows:

  1. Define a generator function g which yields a dictionary of numpy arrays at each step
  2. Construct a Streamer object stream = Streamer(g, args...)
  3. Iterate over examples generated by stream().

On top of this basic functionality, pescador provides the following tools:

For examples of each of these use-cases, refer to the Examples section.