pescador.mux.Mux

class pescador.mux.Mux(streamers, k, rate=256.0, weights=None, with_replacement=True, prune_empty_streams=True, revive=False, random_state=None)

Stochastic multiplexor for Streamers

Examples

>>> # Create a collection of streamers
>>> a = pescador.Streamer("a")
>>> b = pescador.Streamer("b")
>>> c = pescador.Streamer("c")
>>> # Multiplex them together into a single streamer
>>> # Use at most 2 streams at once
>>> # Each stream generates 5 examples on average
>>> mux = pescador.Mux([a, b, c], 2, rate=5)
>>> print("".join(mux(max_iter=9)))
'accacbcba'
>>> print("".join(mux(max_iter=30)))
'abaccbbabccbbbccaccbacbccbbbbc'
__init__(streamers, k, rate=256.0, weights=None, with_replacement=True, prune_empty_streams=True, revive=False, random_state=None)

Given an array (pool) of streamer types, do the following:

  1. Select k streams at random to activate
  2. Assign each activated stream a sample count ~ 1 + Poisson(rate)
  3. Yield samples from the streams by randomly multiplexing from the active set.
  4. When a stream is exhausted, select a new one from streamers.
Parameters:
streamers : iterable of streamers

The collection of streamer-type objects

k : int > 0

The number of streams to keep active at any time.

rate : float > 0 or None

Rate parameter for the poisson distribution governing sample counts for individual streams. If None, each active stream is sampled until exhaustion.

weights : np.ndarray or None

Optional weighting for streamers. If None, then weights are assumed to be uniform. Otherwise, weights[i] defines the sampling proportion of streamers[i].

Must have the same length as streamers.

with_replacement : bool

Sample streamers with replacement. This allows a single stream to be used multiple times (even simultaneously). If False, then each streamer is consumed at most once and never revisited.

prune_empty_streams : bool

Disable streamers that produce no data. If True, streamers that previously produced no data are never revisited. Note: 1. This may be undesirable for streams where past emptiness may not imply future emptiness. 2. Failure to prune truly empty streams with revive=True can result in infinite looping behavior. Disable with caution.

revive: bool

If with_replacement is False, setting revive=True will re-insert previously exhausted streams into the candidate set.

This configuration allows a stream to be active at most once at any time.

random_state : None, int, or np.random.RandomState

If int, random_state is the seed used by the random number generator;

If RandomState instance, random_state is the random number generator;

If None, the random number generator is the RandomState instance used by np.random.

Methods

__init__(streamers, k[, rate, weights, …]) Given an array (pool) of streamer types, do the following:
cycle([max_iter]) Iterate from the streamer infinitely.
iterate([max_iter]) Instantiate an iterator.

Attributes

active Returns true if the stream is active (ie there are still open / existing streams)
is_activated_copy is_activated_copy is true if this object is a copy of the original Streamer and has been activated.