Future of distributions in rand

When working with distributions it is very useful to have access to their density, their distribution function and their quantile function in addition to being able to sample from them. Furthermore, it is nice to be able to calculate their (theoretical, exact) moments. I think it makes sense to have all this functionality in a common interface.

This is implemented in the statrs crate. There is some overlap with rand, sampling is implemented there as well, but for a lot more distributions.

I see the following options:

Only implement sampling in rand. Port the missing distributions from statrs to rand.
Also implement the theoretical properties of the distributions mentioned above in rand, essentially duplicating statrs.
Remove the distributions from rand and suggest to use statrs instead.

What do you think?

@boxtown

Very good question. I don't know; potentially I'm open to big changes (post 0.5).

Statrs focusses on f64 despite using a generic T in the Distribution trait; this probably means at least Range and Uniform should remain in rand.

The Distribution trait in statrs is identical to ours (aside from R: Rng+?Sized addition), so it doesn't make much sense to have both (after 0.5 introduces our Distribution trait). I don't think it makes sense to remove this or basic uniform sampling from rand so statrs could simply re-export our trait later (unfortunately meaning it can't reach 1.0 before rand does).

The other distributions we have (exponential, normal, gamma) do feel out-of-place in rand so potentially we could remove those. I'm embarrassed to say I wasn't aware of statrs until now.

One thing not clear about the statrs distributions is how precise the sampling is — e.g. is the Bernoulli distribution accurate for p < ε (but within the normal range of f64)?

I think we should also ask @boxtown what he thinks?

I see the Bernoulli distribution in statrs simply wraps Binomial, and the latter has a simple O(n) implementation, in contrast to the rejection method used in @fizyk20's PR.

I'm open to the idea of depending on rands Distribution trait. Although to be honest I don't see any issue with the duplication. I am of course open to PRs if there is a desire to move sampling code into Statrs. I'd be willing to tackle it myself if someone is willing to open issues, it might just take a little longer

@fizyk20

Statrs focusses on f64 despite using a generic T in the Distribution trait; this probably means at least Range and Uniform should remain in rand.

I think the main reason for this is simplicity. AFAIK, the sampling of f32 and integers is a desired feature. See statrs-dev/statrs#17.

The other distributions we have (exponential, normal, gamma) do feel out-of-place in rand so potentially we could remove those.

I agree, it makes sense to move them to statrs and advertise the create for generating floats. In the long term, the discrete distributions could be moved there as well.

I see the Bernoulli distribution in statrs simply wraps Binomial, and the latter has a simple O(n) implementation, in contrast to the rejection method used in @fizyk20's PR.

Yes, I think it makes sense to avoid duplicated work implementing the distributions, which is the main reason I brought up the issue.

I labelled this low-priority mostly because it needs more planning and would probably be better to discuss after more distribution code has been implemented / optimised / reviewed, and also because it's beyond the scope of the next release, but this is an important topic to finalise before the 1.0 release.

Just my personal opinion. There are a few distributions I really want to keep/have in Rand:

Some default distribution that all types can implement and that ties in with Rng::gen (Uniform).
The Range distribution.
Some way to get floats with high precision, including a range.
A Bernoulli distribution, making use of the high precision float method to improve upon Rng::gen_bool.
I have really started to like type Exp1 distribution. It is an easy way to improve the performance of code that would otherwise use rng.gen::<f64>().ln().
The Alphanumeric distribution. It does not irreplaceable or perfect, but it is just a nice piece of functionality.

What I am not sure about is generating random numbers that fit some probability distribution. Because if have not used them myself, I can't care strongly whether they are part of Rand or not. Two not-so-strong reasons to have them in Rand (or a clear companion crate to point to):

Generating values in those distributions is a big randomness-related topic. A lot of the literature is about this area, it seems much more to me than there is about RNGs.
Sometimes using some distribution is just the right tool for the job. Simulations / statistical problems are not the only uses. It is nice to have them easily available.

On the other hand, stats-rs is certainly going beyond what rand should provide (in my opinion), and no doubt provides a more complete and useful library for some area's. And it is taking it's inspiration from a good source 😄.

I think it is best to keep generating values according to some distribution in Rand. For simple use cases people can just use Rand. Then it would be nice if stats-rs would use these implementations (work together on improving them), and we would link to stats-rs for the place to find more features.

It's sometimes useful if two projects do the same things from different perspectives, especially if those things involve tricky things like presenting mathematics well.

@vks

It would be good to make some progress on this now. From @vks's suggestions, I like 1 and 3 the best.

1. Only implement sampling in rand. Port the missing distributions from statrs to rand.

This means all the distribution objects (e.g. the Normal struct) exist in Rand, as does the Distribution trait and implementations of it.

Traits like Mean and Variance (see statrs::statistics) remain in statrs and are implemented for the distribution objects in Rand.

3. Remove the distributions from rand and suggest to use statrs instead.

We should keep Standard, Uniform, Alphanumeric and Bernoulli in Rand (these could be re-exported from or duplicated in statrs). For everything else, users should use statrs.

This could be significantly simpler to maintain, and perhaps makes the scope of Rand clearer.

Note that several of the distribution implementations in Rand may be faster than those in statrs but I imagine the statrs implementations can be optimised.

4. New sub-crate

This is just a variant of 1. We move the distribution objects (and their implementations of Distribution) to a sub-crate, e.g. rand_dist, but keep the Distribution trait and a few of the distributions (Standard, Uniform, Alphanumeric, Bernoulli) in Rand.

Functionally, this is not much different than option 3. There main difference is probably that users do not have to import the full statrs to sample from the more complex distributions (though this may not make much difference, especially if we include all the less-commonly-used distributions in rand_dist).

There is some overlap with #494 in that if we use option 3 or 4 then we do not need to worry about most of the distributions should we implement high and low precision sampling variants for e.g. Uniform and Bernoulli.

This is also related to #431 in design concepts; do we prefer few large crates or many small crates? E.g. the statrs::statistics traits can be kept separate from the distributions, and potentially the distributions can be split into multiple sub-crates (though probably not a good idea).

@vks

@vks don't you think we should resolve this issue before trying to duplicate all functionality in statrs?

There are several issues currently:

There is a lot of duplication between rand and statrs
Rand distributions aren't easy to compare (Comparing distributions #560) — having properties like in statrs may be useful
We are now accumulating a lot of distributions into Rand, which is a lot of code that most users are not going to need. On the other hand, some distributions, e.g. Bernoulli, Exp, Normal, may be used more commonly and there is some rationale in not having to import a big stats crate to access them.
All our distribution constructors report errors via panic, which is less useful when input comes from untrusted sources. Possibly we should switch to returning Result types, but this would be a big breaking change.
statrs still depends on Rand 0.3

I'm not sure where we should go from here...

Ignore statrs and implement the bits we want here. Not ideal, but this is what seems to be happening...
Implement sampling within Rand but extended properties in statrs. This seems to be what was suggested above, however I don't know how it can work without making the internal parameters public.
Remove most distributions from Rand and recommend statrs.
Same, but to a sub-crate (e.g. rand_stat)
Split distributions into families and handle each separately; e.g. we roughly have these families:
- uniform and utility distributions (e.g. Alphanumeric)
- Exp, Normal, LogNormal (exp and normal aren't really related, but both use Ziggurat sampling currently)
- Cauchy, Binomial, Poisson
- Gamma and derived (at least, the implementations of Dirichlet and Beta depend on it)
- other: WeightedIndex, Pareto, Triangular, Bernoulli

@dhardy

@dhardy The status quo is that Rand only implements sampling. Most of the work in statrs is calculating the statistics (PDF, CDF, moments etc.). In principle statrs could use the sampling implementations from Rand without introducing a breaking change, so moving them to Rand could actually reduce duplication.

However, as you noted this will result in duplication: A distribution struct has to be created for sampling and another for calculating statistics, unless the internal parameters are made accessible.

I think the main decision is whether we want the code for calculating statistics in Rand or not:

If yes, I think it makes sense to essentially port statrs to rand_stat. We want at least PDF support for testing, and statistics would be nice for Comparing distributions #560.
If not, I think we should remove all non-uniform distributions from Rand. This would be a breaking change but probably not affect most (?) users. statrs would reuse the code for uniform distributions.

Personally, I would prefer the first option, but the disadvantage is that this would add a lot of code that most users will not use. It would also make Rand a bit of a misnomer.

Maybe we should get some community feedback on Reddit/the Rust user forum.

All our distribution constructors report errors via panic, which is less useful when input comes from untrusted sources. Possibly we should switch to returning Result types, but this would be a big breaking change.

This could be done without breaking changes by introducing try_new constructors.

statrs still depends on Rand 0.3

That should be easy to fix.

I think having a rand_stat sub-crate would be a good idea. I'm even open to doing some of the porting myself. I've had less and less time recently to update statrs or even keep it maintained due to work and personal reasons so I think having a more "official" community maintained statistics library would be beneficial.

@vks

@vks yes, this would partially decouple rand and rand_dist versions, but since users would still be dependent on both, I don't see the advantage.

It would make it more difficult to work on the design of rand_dist without changing rand as well.

Is this your real rationale? I wasn't intending on doing much redesign anyway, myself.

I don't see any good rationale for your idea so far.

but since users would still be dependent on both, I don't see the advantage.

Even if we keep the distributions used by Rng in rand and reexport them in rand_dist, we will have this problem, because we need to reexport the moved distributions in rand. So rand will have to depend on rand_dist until we decide to break backwards compatibility. In any case, I don't have a strong preference.

(Ideally, I would like to see rand being a façade for the rand_* crates, similar to num.)

Is this your real rationale?

No, my real rationale is that I think it makes sense to modularize that functionality into a different crate. The decoupling is one of the advantages.

Well, potentially we could push:

the trait plus core distributions (Standard, Uniform, Bernoulli, ...?) into rand_dist, depending only on rand_core
other distributions into rand_stat, depending on rand_dist

then re-export rand_dist as rand::distributions. I'm not sure whether to do the whole, but we can already start on rand_stat (though without bothering with the deprecations).

(Ideally, I would like to see rand being a façade for the rand_* crates, similar to num.)

To fully realise this, we'd also need at least rand_thread (or _entropy or both), _seq and _rng. To me this just seems unnecessary?

Further, we currently use Rng as the "front-end trait for users" and RngCore just for implementation. We'd need to tweak this a bit, e.g. replacing fn sample<R: Rng + ?Sized> with fn sample<R: RngCore + ?Sized>. This is fine, but does mean users get to see more of the complexity of this crate (though that's kind of the case already, e.g. when calling Uniform::new with unsupported types the error message is complex).

Edit: until we've finished dealing with the fall-out from 0.6.2 we should hold off on big PRs.

I think we should start on this now — the goal being to reduce rand itself to the functionality we think it should have by default, parking the rest in another crate like rand_stat for now. This is not the ideal approach, but making this move at least allows us to get the rand crate closer to where we want it.

Eventually this could still merge with statrs or not — this would allow some de-duplication, but I don't like to confuse users with overlap like which Bernoulli or Uniform implementation to use (statrs has both, but we definitely want to keep our versions in Rand for use by Rng::gen_range etc.).

To do items:

create new crate — in a new repo?
move distributions (most which are not depended on by Rng) to this crate
move over relevant issues

No, my real rationale is that I think it makes sense to modularize that functionality into a different crate. The decoupling is one of the advantages.

We've already had complaints about Rand using two many crates. Adding another crate doesn't really gain anything, since if rand depends on rand_dist, then breaking changes in rand_dist still require a new release in rand (and delaying doing so would just be a nuisance to users since the latest version of each crate would have a different API for the same stuff).

So lets do the following:

release rand_distr, versioned in lock-step with rand (so users of both should use the same minor version)
keep some parts of rand::distributions (e.g. Distribution, Uniform) where they are and re-export from rand_distr
move other items to rand_distr

Thus from a user's point of view, they can just use rand for the basics or use rand_distr for the full set of distributions (with rand_distr being a pure superset of rand::distributions).

This change lets us do two things:

slim down Rand for many users
make breaking changes to the less common distributions now (while moving to rand_distr)

Sounds good to me! I'll presumably try to implement this in a week or so.

Keeping the versions in lock-step does have disadvantages:

breaking changes in rand_distr must wait on a version bump in rand
this approach differs from all other Rand crates, thus will surprise users

On the other hand, without this, re-exporting all of rand::distributions makes less sense, because it is difficult to predict whether the re-exported contents are currently equivalent or not (though since the Distribution trait is implemented by external libs we will likely have to use upgrade shims at some point; I believe this is already an issue when using statrs).

However, I think having all distributions available through rand_distr is still going to be more user-friendly than forcing users to remember which distributions are in which package.

This means that rand_distr v0.1 can just be a copy of the current state of distributions, and v0.2 can make breaking changes, while hopefully both versions can be compatible with rand v0.6-0.7.

FYI I've started working on this

rand_distr has been implemented. The duplication between statrs and rand_distr still needs to be addressed.

@boxtown

Does this duplication need to be addressed? @boxtown

Statrs claims to be a port from the C# Math.NET library. It implements a bunch of things (e.g. error function, CDFs) that are quite specialised (not widely used). Its distributions are implemented such that properties like mean etc. can easily be calculated.

rand_distr on the other hand has a narrower focus: only implement sampling algorithms, and optimise for speed.

Because of this I think there is scope for the on-going existance of both libraries, and I have documented this here and here.

(Note: another possibility would be including the extra functionality in rand_distr or another crate in order to make statrs obsolete since @boxtown said above he has less interest in continued maintenance. However I don't have an interest in doing this either, hence it seems more likely we will continue with two crates. This goes beyond the scope of this issue however, so lets close #290.)

We will need some of statrs functionality (namely PDFs) to properly test sampling.

@burdges

I don't really see a need to de-dupe. Like @burdges mentioned above I don't see a harm in having two approaches to sampling and like you mentioned, the bulk of the work Statrs is doing is in calculating attributes of various distributions

We will need some of statrs functionality (namely PDFs) to properly test sampling.

🤣

Last time I tried histogram testing, I think even with 10 million samples I was struggling to get useful results. I think it's probably a dead-end, at least for standard integration testing (though it may be somewhat usable for offline analysis). OTOH "black box testing" (aka testing a few samples produced the same result as last time) is easy to do and somewhat useful.

I was able to get errors smaller than 0.001 with a million samples, but this is for (kind of) uniform distributions: https://github.com/rust-random/rand/blob/master/rand_distr/tests/uniformity.rs

For non-uniform distributions it will be more tricky.

Navigation Menu

Repository navigation

Future of distributions in rand #290

1. Only implement sampling in rand. Port the missing distributions from statrs to rand.

3. Remove the distributions from rand and suggest to use statrs instead.

4. New sub-crate

20 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Future of distributions in rand #290

Description

Activity

dhardy commented on Mar 8, 2018

dhardy commented on Mar 8, 2018

boxtown commented on Mar 9, 2018

vks commented on Mar 9, 2018

dhardy commented on Mar 14, 2018

pitdicker commented on Mar 30, 2018

burdges commented on Mar 30, 2018

dhardy commented on Jun 12, 2018

1. Only implement sampling in rand. Port the missing distributions from statrs to rand.

3. Remove the distributions from rand and suggest to use statrs instead.

4. New sub-crate

dhardy commented on Jul 30, 2018

vks commented on Jul 30, 2018

boxtown commented on Jul 30, 2018

20 remaining items

dhardy commented on Nov 15, 2018

vks commented on Nov 21, 2018

dhardy commented on Jan 6, 2019

dhardy commented on Jan 28, 2019

dhardy commented on Mar 23, 2019

vks commented on Mar 26, 2019

dhardy commented on Mar 27, 2019

vks commented on Jun 27, 2019

dhardy commented on Jun 27, 2019

vks commented on Jun 27, 2019

boxtown commented on Jun 27, 2019

dhardy commented on Jun 27, 2019

vks commented on Jun 27, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions