pescador.mux.Mux¶
-
class
pescador.mux.
Mux
(streamers, k, rate=256.0, weights=None, with_replacement=True, prune_empty_streams=True, revive=False, random_state=None)¶ Stochastic multiplexor for Streamers
Examples
>>> # Create a collection of streamers >>> a = pescador.Streamer("a") >>> b = pescador.Streamer("b") >>> c = pescador.Streamer("c") >>> # Multiplex them together into a single streamer >>> # Use at most 2 streams at once >>> # Each stream generates 5 examples on average >>> mux = pescador.Mux([a, b, c], 2, rate=5) >>> print("".join(mux(max_iter=9))) 'accacbcba' >>> print("".join(mux(max_iter=30))) 'abaccbbabccbbbccaccbacbccbbbbc'
-
__init__
(streamers, k, rate=256.0, weights=None, with_replacement=True, prune_empty_streams=True, revive=False, random_state=None)¶ Given an array (pool) of streamer types, do the following:
- Select
k
streams at random to activate - Assign each activated stream a sample count ~ 1 + Poisson(rate)
- Yield samples from the streams by randomly multiplexing from the active set.
- When a stream is exhausted, select a new one from streamers.
Parameters: streamers : iterable of streamers
The collection of streamer-type objects
k : int > 0
The number of streams to keep active at any time.
rate : float > 0 or None
Rate parameter for the poisson distribution governing sample counts for individual streams. If
None
, each active stream is sampled until exhaustion.weights : np.ndarray or None
Optional weighting for
streamers
. IfNone
, then weights are assumed to be uniform. Otherwise,weights[i]
defines the sampling proportion ofstreamers[i]
.Must have the same length as
streamers
.with_replacement : bool
Sample streamers with replacement. This allows a single stream to be used multiple times (even simultaneously). If
False
, then each streamer is consumed at most once and never revisited.prune_empty_streams : bool
Disable streamers that produce no data. If
True
, streamers that previously produced no data are never revisited. Note: 1. This may be undesirable for streams where past emptiness may not imply future emptiness. 2. Failure to prune truly empty streams with revive=True can result in infinite looping behavior. Disable with caution.revive: bool
If
with_replacement
isFalse
, settingrevive=True
will re-insert previously exhausted streams into the candidate set.This configuration allows a stream to be active at most once at any time.
random_state : None, int, or np.random.RandomState
If int, random_state is the seed used by the random number generator;
If RandomState instance, random_state is the random number generator;
If None, the random number generator is the RandomState instance used by np.random.
- Select
Methods
__init__
(streamers, k[, rate, weights, …])Given an array (pool) of streamer types, do the following: cycle
([max_iter])Iterate from the streamer infinitely. iterate
([max_iter])-