pescador.mux.StochasticMux

class pescador.mux.StochasticMux(streamers, n_active, rate, weights=None, mode='with_replacement', prune_empty_streams=True, dist='binomial', random_state=None)

Stochastic Mux

Examples

>>> # Create a collection of streamers
>>> a = pescador.Streamer("a")
>>> b = pescador.Streamer("b")
>>> c = pescador.Streamer("c")
>>> # Multiplex them together into a single streamer
>>> # Use at most 2 streams at once
>>> # Each stream generates 5 examples on average
>>> mux = pescador.StochasticMux([a, b, c], 2, rate=5)
>>> print("".join(mux(max_iter=9)))
'accacbcba'
>>> print("".join(mux(max_iter=30)))
'abaccbbabccbbbccaccbacbccbbbbc'

__init__(streamers, n_active, rate, weights=None, mode='with_replacement', prune_empty_streams=True, dist='binomial', random_state=None)

Given an array (pool) of streamer types, do the following:

1. Select k streams at random to iterate from

2. Assign each activated stream a sample count with expected value rate

3. Yield samples from the streams by randomly multiplexing from the active set.

4. When a stream is exhausted, select a new one from streamers.

Parameters:

streamersiterable of streamers

The collection of streamer-type objects

n_activeint > 0

The number of streams to keep active at any time.

ratefloat > 0 or None

Rate parameter for the distribution governing sample counts for individual streams. If None, sample each stream to exhaustion before de-activating.

weightsnp.ndarray or None

Optional weighting for streamers. If None, then weights are assumed to be uniform. Otherwise, weights[i] defines the sampling proportion of streamers[i].

Must have the same length as streamers.

mode[“with_replacement”, “single_active”, “exhaustive”]

with_replacement

Sample streamers with replacement. This allows a single stream to be used multiple times (even simultaneously). Streams are sampled independently and indefinitely.

single_active

This configuration allows a stream to be active at most once at any time.

exhaustive

Each streamer is consumed at most once and never revisited. Run every selected stream once to exhaustion.

prune_empty_streamsbool

Disable streamers that produce no data.

dist[“constant”, “binomial”, “poisson”]

Distribution governing the (maximum) number of samples taken from an active streamer. In each case, the expected number of samples will be rate.

See Analysis of Stochastic Mux for detailed discussion.

random_stateNone, int, or np.random.RandomState

See BaseMux

Methods

__init__(streamers, n_active, rate[, ...])	Given an array (pool) of streamer types, do the following:
cycle([max_iter])	Iterate from the streamer infinitely.
iterate([max_iter])	Yield items from the mux, and handle stream exhaustion and replacement.

Attributes

active	Return true if the stream is active (ie there are still open / existing streams)
is_activated_copy	is_activated_copy is true if this object is a copy of the original Streamer and has been activated.
n_streams	Return the number of streamers.