pescador.mux.StochasticMux

class pescador.mux.StochasticMux(streamers, n_active, rate, weights=None, mode='with_replacement', prune_empty_streams=True, random_state=None)

Stochastic Mux

Examples

>>> # Create a collection of streamers
>>> a = pescador.Streamer("a")
>>> b = pescador.Streamer("b")
>>> c = pescador.Streamer("c")
>>> # Multiplex them together into a single streamer
>>> # Use at most 2 streams at once
>>> # Each stream generates 5 examples on average
>>> mux = pescador.StochasticMux([a, b, c], 2, rate=5)
>>> print("".join(mux(max_iter=9)))
'accacbcba'
>>> print("".join(mux(max_iter=30)))
'abaccbbabccbbbccaccbacbccbbbbc'
__init__(streamers, n_active, rate, weights=None, mode='with_replacement', prune_empty_streams=True, random_state=None)

Given an array (pool) of streamer types, do the following:

  1. Select k streams at random to iterate from
  2. Assign each activated stream a sample count ~ 1 + Poisson(rate)
  3. Yield samples from the streams by randomly multiplexing from the active set.
  4. When a stream is exhausted, select a new one from streamers.
Parameters:
streamers : iterable of streamers

The collection of streamer-type objects

n_active : int > 0

The number of streams to keep active at any time.

rate : float > 0 or None

Rate parameter for the distribution governing sample counts for individual streams. If None, sample each stream to exhaustion before de-activating.

weights : np.ndarray or None

Optional weighting for streamers. If None, then weights are assumed to be uniform. Otherwise, weights[i] defines the sampling proportion of streamers[i].

Must have the same length as streamers.

mode : [“with_replacement”, “single_active”, “exhaustive”]
with_replacement

Sample streamers with replacement. This allows a single stream to be used multiple times (even simultaneously). Streams are sampled independently and indefinitely.

single_active

This configuration allows a stream to be active at most once at any time.

exhaustive

Each streamer is consumed at most once and never revisited. Run every selected stream once to exhaustion.

prune_empty_streams : bool

Disable streamers that produce no data. See BaseMux

random_state : None, int, or np.random.RandomState

See BaseMux

Methods

__init__(streamers, n_active, rate[, …]) Given an array (pool) of streamer types, do the following:
cycle([max_iter]) Iterate from the streamer infinitely.
iterate([max_iter]) Yields items from the mux, and handles stream exhaustion and replacement.

Attributes

active Returns true if the stream is active (ie there are still open / existing streams)
is_activated_copy is_activated_copy is true if this object is a copy of the original Streamer and has been activated.
n_streams Return the number of streamers.