Skip to content

docs: improve READMEs and add CONTRIBUTING.md - #60

Merged
andygrove merged 8 commits into
apache:mainfrom
andygrove:docs-top-level-readme
Nov 12, 2026
Merged

andygrove merged 8 commits into
apache:mainfrom
andygrove:docs-top-level-readme

Conversation

@andygrove

@andygrove andygrove commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Reworks the documentation entry points:

  • Top-level README: usage pointer, layout, contributing checklist.
  • Crate README: user-facing content for crates.io (overview, quick start, CLI, caveats).
  • New CONTRIBUTING.md: implementation and testing guidelines moved out of the crate README.

Rustdoc/Cargo metadata changes are in #57.

@andygrove
andygrove marked this pull request as ready for review November 12, 2026 03:14
@andygrove andygrove changed the title docs: flesh out top-level README docs: improve READMEs and add CONTRIBUTING.md Nov 12, 2026

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @andygrove this is a good start

@andygrove
andygrove merged commit fc2babe into apache:main Nov 12, 2026
4 checks passed
@andygrove

Copy link
Copy Markdown
Member Author

Thanks @comphead

andygrove added a commit that referenced this pull request Nov 13, 2026
Part of #55

## Rationale

The documentation guidelines added to `CONTRIBUTING.md` in #60 say to
document each function with the `#[user_doc]` macro from
`datafusion-macros`. The crate doesn't depend on that macro yet. Only 1
of the 95 scalar and aggregate UDFs on `main` (`pow`) implements
`documentation()`.

This PR adds the dependencies and applies the guidelines to two
functions, so that they serve as examples to copy when filling in the
rest.

## Changes

- Add `datafusion-macros` and `datafusion-doc` as dependencies.
`datafusion-doc` is needed because the macro expands to code that uses
it. Both come from the same DataFusion branch as the other DataFusion
crates.
- Document `sha2` and `substring` with `#[user_doc]`, and implement
`documentation()` by returning the generated `self.doc()`. Each covers
how the function differs from DataFusion's built-in, its Spark edge
cases, its arguments and SQL examples. `substring` also lists `substr`
as alternative syntax.
- Cut the structs' rustdoc down to a one-line summary and a link to the
Spark docs, so the same facts aren't written twice.

## Testing

- `cargo clippy --all-targets --all-features -- -D warnings`, `cargo
test -p datafusion-spark` and `cargo test -p
datafusion-spark-sqllogictest` pass on the branch, which includes `main`
as of #60.
- I ran every SQL example in the docs through the sqllogictest runner,
and they all match.
- A throwaway unit test checked that `documentation()` returns the
expected section, syntax, alternative syntax and argument names for both
functions.
- The documented behavior is covered by the existing `sha2` and
`substring` SLT tests, or is visible in the code.

## User-facing changes

`documentation()` now returns docs for `sha2` and `substring`, so they
appear in generated function references. Behavior does not change.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
andygrove added a commit that referenced this pull request Nov 16, 2026
Part of #55

## Rationale

The published crate page on crates.io is generic, and docs.rs shows a
bare list of function modules with no descriptions.

## Changes

- Expand the crate-level docs in `lib.rs` (motivation, getting started,
packages, semantics and caveats).
- Add `//!` module docs to every function package. Packages with no
functions yet say so.
- Document `SparkFunctionPlanner`.
- Add `keywords`, `categories` and `documentation` to `Cargo.toml`.

## Testing

Docs only. I could not run `cargo doc` in my checkout (no workspace
root), so please check that CI's doc build is clean. README changes are
in #60.

Per-function documentation is tracked separately in #55.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants