zmq_stream.zmq_stream¶

zmq_stream.zmq_stream(streamer, max_batches=None, min_port=49152, max_port=65535, max_tries=100, copy=False)[source]¶

Parallel data streaming over zeromq sockets.

This allows a data generator to run in a separate process from the consumer.

A typical usage pattern is to construct a Streamer object from a generator (or util.mux of several Streamer`s), and then use `zmq_stream to execute the stream in one process while the other process consumes data, e.g., with a StreamLearner object.

Parameters:

streamer : pescador.Streamer

The streamer object

max_batches : None or int > 0

Maximum number of batches to generate

min_port : int > 0

max_port : int > min_port

The range of TCP ports to use

max_tries : int > 0

The maximum number of connection attempts to make

copy : bool

Set True to enable data copying

Yields:

batch

Data drawn from streamer.generate(max_batches).