Skip to content

Tracker: planned changes for 0.5 #232

Closed
Closed
Tracker: planned changes for 0.5#232
Assignees
dhardy
Labels
P-highPriority: highX-trackerType: tracker

Description

@dhardy
dhardy
opened on Jan 12, 2018

To follow up on the release strategy in #205, here is a rough plan for Rand 0.5:

Related issues which need solving:

There are some extra changes which should be relatively easy to add after the above:

Desired additions:

Finally, a list of other planned changes, which probably won't make 0.5:

Activity

  1. dhardy
    self-assigned this
    on Jan 12, 2018

  2. pitdicker commented on Jan 12, 2018

    @pitdicker
    pitdicker
    on Jan 12, 2018
    Contributor

    If we remove reseed from SeedableRng this is going to break RNG implementations. Together with the introduction of rand-core, I think it makes sense to drop the Sized requirement.

  3. dhardy commented on Jan 12, 2018

    @dhardy
    dhardy
    on Jan 12, 2018
    MemberAuthor

    But the Sized thing only concerns Rand, which really doesn't have much to do with the Rng implementations.

  4. dhardy commented on Feb 24, 2018

    @dhardy
    dhardy
    on Feb 24, 2018
    MemberAuthor

    Most of the above is implemented now.

    There are still some changes to Range and possibly also float sampling which I believe @pitdicker is working on; it would be nice to get those in the 0.5 release, and then there is the rand-core sub-crate (#264), which could be done now or could wait until the next release (probably better now so that the impls module can be exported only from rand-core, as in my master branch).

    About the release process: I think the best method might be to publish a 0.5.0-pre.0 pre-release, then wait a minimum of two weeks total for comment and testing (and at least 1 week since the last pre-release) during which no major new features will be merged.

    For rand-core I guess the version number should start at 0.1.0-pre.0 and be released on the same time-schedule; after this release I don't think there will be any need to synchronise versions/releases between the two crates (and hopefully rand-core can hit 1.0 sooner).

    Also during the release process we need to work out what kind of compatibility release to make for 0.4.

  5. dhardy commented on Feb 24, 2018

    @dhardy
    dhardy
    on Feb 24, 2018
    MemberAuthor

    I just added two extra desired features: CryptoRng and switching thread_rng to HC-128.

    Now would be a good time to add any extra requests for this release (but be warned that larger features may get delayed to 0.6 if not ready to merge soon).

  6. pitdicker commented on Feb 24, 2018

    @pitdicker
    pitdicker
    on Feb 24, 2018
    Contributor

    There are still some changes to Range and possibly also float sampling which I believe @pitdicker is working on;

    Yes, copying the Range code with the RangeImp method from your branch was fairly easy (thanks to the huge merge you did 👍). I have used fixed-precision float methods as discussed in dhardy#85 (comment) for Uniform and Range. The maximum precision methods will need a little more thinking, so I'll leave them for another PR. I think it is best to merge on top of #268, because MockAddRng makes fixing the test from WeightedChoice easier (uses Range internally).

    For switching thread_rng to HC-128 I have things ready, waited on #252 before making the PR.

    Hopefully will finish things tomorrow.

  7. pitdicker commented on Mar 7, 2018

    @pitdicker
    pitdicker
    on Mar 7, 2018
    Contributor

    I think we are making pretty good progress towards 0.5 😄.

    From #286 (comment)

    Speaking of alternatives, there's still all your FP code in my experimental branch... have to figure out whether we'll use that at some point.

    I would still like to see it become available. I think it is settled that it should change from pure conversion functions to distributions, that can use arbitrary precision (not a fixed 32 or 64 bits). So I will basically have to rewrite things. We have to come up with names, and figure out how to expose a second kind of uniform floating point ranges.

    That is why I left it out of #274. Shall we leave it until after rand 0.5?

    Do you still want to do the rand-core split?

    One thing I really want to see is

    Change the RNG used by weak_rng: dhardy#60, dhardy#52

    It should not be all that much work to polish the PCG and XoroshiroMT implementations in my small_rngs crate, and also not to move all the PRNGs out to a separate crate.

    @dhardy, did you somehow get a notification from this?

  8. dhardy commented on Mar 7, 2018

    @dhardy
    dhardy
    on Mar 7, 2018
    MemberAuthor

    That is why I left it out of #274. Shall we leave it until after rand 0.5?

    Perhaps; I don't think we should block 0.5 waiting on it at least.

    Do you still want to do the rand-core split?

    I asked, and the answer seems to be yes. This is easy to implement but I thought better to wait until most other stuff is done though; I may get to this soon now. (There's additional motivation: we can make the impls module available only through rand-core.)

    I'm not sure about changing weak_rng though it definitely makes sense. Did we decide which RNG to use yet? Ultimately it would be nice for this to be in a separate crate though it could be in rand for the next release; on the other hand it should be easy to add another crate once rand-core is ready anyway.

    No, I missed the notification on your history article; I'll have a read.

  9. pitdicker commented on Mar 7, 2018

    @pitdicker
    pitdicker
    on Mar 7, 2018
    Contributor

    Did we decide which RNG to use yet?

    The choice was between PCG and my Xoroshiro variant. But to be honest I found a problem with the last one, trying to prove it was correct (a while ago actually). A widening multiply does not guarantee uniformity or equidistribution. It should be a 'real', 128 bit multiply. This can be done by using a widening multiply plus a normal multiply, causing a little slowdown (but still faster than PCG). Just to err on the safe side I would make PCG the default, and also because it is better known.

  10. dhardy commented on Mar 7, 2018

    @dhardy
    dhardy
    on Mar 7, 2018
    MemberAuthor

    That sounds good enough to me; I'd rather err on the side of strong than fast-and-weak personally. But you should probably make a post in dhardy#60 to clarify to the people who commented there what the plan is. Also I think perhaps we should remove weak_rng and instead add a SmallRng wrapper and make users do SmallRng::new(). (We could add FastRng too and make the latter use Xoroshiro*, potentially.)

  11. vks commented on Mar 7, 2018

    @vks
    vks
    on Mar 7, 2018
    Contributor

    I think removing weak_rng is a good idea. I'm not sure about the SmallRng name. Why not just use the name of the RNG? Reproducibility is important, I think changing the implementation of SmallRng should be considered a breaking change. I would rather use the name of the algorithm and recommend RNGs for different purposes in the documentation instead of abstracting them away.

    In my opinion, the following kinds of generators should be considered:

    1. A cryptographic, default one. (E.g. ISAAC or HC128)
    2. A fast one without linear dependencies. (E.g. PCG or truncated xoroshiro.)
    3. A faster one with some linear dependencies. This is what practically almost everyone is currently using if cryptographic properties are not needed. (E.g. xorshift, xoroshiro)
    4. A small generator for easier seeding of larger generators from i.e. a u64. (E.g. splitmix64)
    5. A generator with a large (> 2^128) period for massively parallel computations (xorshift1024*).

    The tradeoffs between 1., 2. and 3. should be clearly documented.

    (Personally, I think 2. and 5. are extremely niche. 4. could be hidden behind some abstraction or replaced with 1. I don't see a convincing reason to use xoroshiro128* over xoroshiro128+.)

  12. dhardy commented on Mar 8, 2018

    @dhardy
    dhardy
    on Mar 8, 2018
    MemberAuthor

    @vks please see dhardy#58. The intention is to provide both names (e.g. StdRng and Hc128Rng), but put the former in rand and the latter in other crates, and to clearly mark the ones in rand as non-reproducible.

    "Practically almost everyone" doesn't fly as an argument; there are still many people using MT19937, and while it might be sufficient for their purposes I won't recommend it just because it's popular.

  13. vks commented on Mar 8, 2018

    @vks
    vks
    on Mar 8, 2018
    Contributor

    Thanks for the link!

    "Practically almost everyone" doesn't fly as an argument; there are still many people using MT19937, and while it might be sufficient for their purposes I won't recommend it just because it's popular.

    In my opinion the argument has some merit, because given the widespread use of generators with linear dependencies, I would expect to hear of practical problems if there were any. Following the same line of thought, I think the statistical quality of MT is ok, the performance seems like a bigger issue in practice.

  14. dhardy commented on Mar 8, 2018

    @dhardy
    dhardy
    on Mar 8, 2018
    MemberAuthor

    @vks I opened #289 regarding your point about dropping weak_rng. Also note:

    • the plan is to provide several good generators, by name, in sub-crates; this is however beyond the scope of the rand 0.5 release
    • we want to have a variety of reproducible generators available somehow, with minimal breakage
    • introducing one or two new generators within rand temporarily may be possible

  15. clarfonthey commented on Mar 8, 2018

    @clarfonthey
    clarfonthey
    on Mar 8, 2018
    Contributor

    I'm kind of leaning toward not having any gen methods on Rng at all and instead making these part of Distribution itself. Also, giving preferential treatment to Uniform may lead to a lot of people using it when they shouldn't, e.g. via habits from other languages, like rng.gen() % n instead of using a proper range.

  16. 25 remaining items

  17. pitdicker commented on Apr 2, 2018

    @pitdicker
    pitdicker
    on Apr 2, 2018
    Contributor

    I have to admit doing a pre-release was useful! Especially thanks to the announcement on Reddit.

    It seems to me we are mostly there taking care of the feedback after the pre-release. At least the part I care about 😄.

    For #359 I have a branch ready. Do we want to try something with #348 now? And maybe hide the ChaChaRng::set_rounds method #349?

  18. dhardy commented on Apr 2, 2018

    @dhardy
    dhardy
    on Apr 2, 2018
    MemberAuthor

    I have some extra docs in progress. Sorry, I've not been able to devote much time to Rand the last week (and I won't be able to next week either; might get a little time this week though).

  19. pitdicker commented on Apr 2, 2018

    @pitdicker
    pitdicker
    on Apr 2, 2018
    Contributor

    We are just volunteers, right?

  20. pitdicker commented on Apr 4, 2018

    @pitdicker
    pitdicker
    on Apr 4, 2018
    Contributor

    Found something nice in the cargo.toml that gets shipped on crates.io:

    # When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g. crates.io) dependencies

    So we don't need to change the rand_core dependency line in cargo.toml to point to the crates.io version, we can just depend on the one in the workspace.

  21. rust-random
    deleted a comment from pitdicker on Apr 5, 2018

  22. dhardy commented on Apr 5, 2018

    @dhardy
    dhardy
    on Apr 5, 2018
    MemberAuthor

    We at least need a version number for the dependency; it complained about that last time.

  23. dhardy commented on Apr 17, 2018

    @dhardy
    dhardy
    on Apr 17, 2018
    MemberAuthor

    rand_core 0.1.0 has been released.

  24. dhardy commented on Apr 27, 2018

    @dhardy
    dhardy
    on Apr 27, 2018
    MemberAuthor

    I'd like to get the following done, then I think we're good for a release:

    Not sure about these:

    We could also look at these, but there's no reason they can't wait IMO:

  25. dhardy commented on May 18, 2018

    @dhardy
    dhardy
    on May 18, 2018
    MemberAuthor

    I posted new pre-releases 3 days ago:

    So far there has been little feedback regarding stuff that needs changing, so it looks likely that these will be the final pre-releases. In this case the releases should be made early next week.

  26. pitdicker commented on May 21, 2018

    @pitdicker
    pitdicker
    on May 21, 2018
    Contributor

    Can you make a PR with the final changes for the release, instead of a direct push (then I can be a second pair of eyes...)?

  27. dhardy commented on May 21, 2018

    @dhardy
    dhardy
    on May 21, 2018
    MemberAuthor

    Looks to me like all the main things are in place for the 0.2 / 0.5 releases. So do you want to make the PR and I'll release?

  28. dhardy commented on May 21, 2018

    @dhardy
    dhardy
    on May 21, 2018
    MemberAuthor

    Done:

  29. dhardy
    closed this as completedon May 21, 2018

  30. pitdicker commented on May 21, 2018

    @pitdicker
    pitdicker
    on May 21, 2018
    Contributor

    🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

  31. dhardy commented on May 22, 2018

    @dhardy
    dhardy
    on May 22, 2018
    MemberAuthor

    Yes, I know! 🎉 🍰 ⛵

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P-highPriority: highX-trackerType: tracker

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions