Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 56 additions & 42 deletions plans/events-legacy-backfill.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
status: planned
depends: [events-series-agenda]
depends: [events-storage-layout]
specs:
- specs/behaviors/events.md
- specs/behaviors/legacy-id-mapping.md
Expand All @@ -12,68 +12,82 @@ issues: [194]

## Scope

Laddr held exactly four events (`launchpad-2018`, `launchpad_2019`,
`datahack2020`, `launchpad-2023`), each with a handful of schedule segments.
Carry them into the `events` sheet so `/events/past` has history and the old
URLs keep resolving.

**In:** four hand-authored event records plus one `event-series` record
("Launchpad", grouping the three Launchpads) on the data repo's `published`
branch (via PR, per that repo's conventions), with `legacyId`, original
handles as slugs, `status: scheduled`, and each laddr segment carried as an
`agenda` item (title, times, venue, summary + description). Legacy
`/thumbnail/<id>/…` images referenced in descriptions are captured as
attachments and rewritten, per
Laddr held exactly four "events" (`launchpad-2018`, `launchpad_2019`,
`datahack2020`, `launchpad-2023`), each a multi-week program with a handful
of schedule segments. Under the data model each is a **series** whose days
are events: carry them into the `event-series` and `events` sheets so
`/events/past` has history and the old URLs keep resolving.

**In:** one `event-series` record per laddr event (slug = laddr handle,
title and description from the program), one `events` record per calendar
day of segments (slug `<YYYY-MM-DD>-<series slug>`, venue from the day's
first segment, span from the day's first start to its last end, the day's
segments as `agenda` items), legacy `/thumbnail/<id>/…` images captured as
attachments on the event whose text references them and rewritten per
[legacy-id-mapping.md](../specs/behaviors/legacy-id-mapping.md#legacy-resources-captured-at-import-time-not-proxied-at-runtime).
Committed through gitsheets on a branch of the data repo for a PR into
`published`.

**Out:** per-session events
([deferred.md](../specs/deferred.md#per-session-rsvps-on-multi-day-events));
any Meetup check-in history (dropped).
**Out:** any Meetup check-in history (dropped); RSVPs for historical
events (there were none on the site).

## Implements

- [behaviors/events.md](../specs/behaviors/events.md#legacy-urls).
- [legacy-id-mapping.md](../specs/behaviors/legacy-id-mapping.md#legacy-url-forms-we-accept) —
`/events/<handle>` and `/events/<handle>/segments/<seg>` rows.
- [data-model.md](../specs/data-model.md#event) — `legacyId` on Event and
the naming-map rows.
`/events/<handle>` and `/events/<handle>/segments/<seg>` → the series page
(the redirect itself ships in `events-storage-layout`).
- [data-model.md](../specs/data-model.md#event) — the laddr naming-map rows.

## Approach

1. Source data is the laddr JSON (`/events/*all?format=json` and
`/events/<handle>/segments?format=json`), captured in the research
scratchpad during spec work; re-fetch from
`codeforphilly.live.k8s.phl.io` if needed while it's up.
2. Write a one-off script under `apps/api/scripts/` that translates each
event (epoch → ISO UTC, `Status` → `scheduled`, each segment → an
`agenda` item with `Summary` and `Description` joined by a blank line;
the one segment whose `EndTime` precedes its `StartTime` gets `endsAt`
dropped) and emits the TOML via the gitsheets API against a working clone
of the data repo. The three Launchpads get `seriesId` of the "Launchpad"
series; DataHack stands alone.
3. Open the data-repo PR; merge to `published`; the hot-reload webhook
picks it up.
4. Add the `/events/<handle>/segments/<seg>` → `/events/<handle>#schedule`
rewrite to `plugins/legacy-redirect.ts`.
2. `apps/api/scripts/backfill-laddr-events.ts` (already on the
superseded #213 branch, to be reworked): per laddr event, write the
series (description = the program's markdown, media rewritten, CRLF and
space-less headings normalized); group its segments by Philadelphia
calendar day; per day write an event titled after the day's first
segment (or "<series title> — <weekday>" when the first segment is
generic), `venueName`/`venueAddress` from the first segment that has one,
`startsAt` = first start, `endsAt` = last end (or last start + 1h), the
day's segments as agenda items (a segment whose `EndTime` precedes its
`StartTime` keeps `startsAt` only). Images referenced by a segment go on
that day's event; images referenced by the program description go on
the series (series are content records too) — if series attachments are
not supported by the config, fall back to the first day's event and
rewrite accordingly.
3. Re-runnable: series match by slug, events by `(seriesId, date)`;
an unchanged run is a no-op commit. Branch created from a parent ref
with raw `git update-ref`, then `transact({ parent, branch })`.
4. Regenerate codeforphilly-data #12 (close the old branch) and merge it
into `published` after the release that reads these sheets is live.

## Validation

- [ ] `/events/launchpad-2023`, `/events/launchpad_2019`,
`/events/datahack2020`, `/events/launchpad-2018` render on production
with the Schedule sidebar listing every laddr segment under its day with
time range and venue.
- [ ] `/events/series/launchpad` lists the three Launchpads under Past.
- [ ] `/events/series/launchpad-2023`, `/events/series/launchpad_2019`,
`/events/series/datahack2020`, `/events/series/launchpad-2018` render on
production, each listing its days under Past with venue and date.
- [ ] Each day's event renders its Schedule with every laddr segment of
that day, time range and venue.
- [ ] `/events/launchpad-2023` and
`/events/launchpad-2023/segments/friday_night_kickoff-2023` 301 to the
series page.
- [ ] No description references `codeforphilly.org/thumbnail/…`.
- [ ] `/events/launchpad-2023/segments/friday_night_kickoff-2023` 301s to
the event's schedule anchor.
- [ ] `/events/past` shows the four events in reverse start order.
- [ ] `/events/past` shows the days in reverse start order.

## Risks / unknowns

- **Agenda range rule.** Laddr's `launchpad-2018` had segments with null
`EndTime` and one with end before start; the agenda validator requires
each item's start inside the event range, which these satisfy, but check
the event `endsAt` covers the last segment before importing.
- **Day boundaries.** Segments are grouped by Philadelphia day; a segment
that crosses midnight stays with the day it starts. The 2023 "Weekly
Hack Nights 4/4 – 5/9" was a single laddr segment on one evening and
becomes one event on that date with its description listing the other
dates.
- **Naming the day.** The first segment's title is the event title; for
2018's work day that is "Work Day", for 2023's "Friday Night Kickoff" —
check the four programs by eye before merging #12.

## Notes

Expand Down
105 changes: 105 additions & 0 deletions plans/events-storage-layout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
status: planned
depends: [events-series-agenda]
specs:
- specs/data-model.md
- specs/behaviors/storage.md
- specs/behaviors/events.md
- specs/behaviors/legacy-id-mapping.md
issues: [194]
---

# Plan: year-partitioned, content-oriented event records

## Scope

Change how event records sit in the data repo before any exist in
production: events partitioned by year and stored as markdown-bodied
records, series stored the same way, RSVPs partitioned under the event's
year. Plus the model clarification that a multi-day program is a series of
per-day events, which touches the legacy redirects.

**In:** the three sheet configs on the data repo's `empty`, `fixture` and
`legacy-import` branches (path templates + `[gitsheet.format]`), mirrored
into `apps/api/gitsheets/`; the RSVP write path injecting `year`; the
`/events/<handle>` → `/events/series/<handle>` legacy redirect; tests and
the dev seed. No schema or contract change is expected — `description`
stays a string field; the format block only moves it to the file body.

**Out:** `published` — the other session adopts there in lockstep with the
release (zero event records exist, so the new templates apply cleanly);
the backfill itself ([`events-legacy-backfill`](events-legacy-backfill.md),
which now depends on this plan).

## Implements

- [data-model.md](../specs/data-model.md#event) — Event / EventSeries /
EventRsvp sheet, path template and format lines; the laddr naming-map
rows.
- [behaviors/storage.md](../specs/behaviors/storage.md#sheet-layout) —
the three rows.
- [behaviors/events.md](../specs/behaviors/events.md#series) — the Series
and Agenda semantics (program instance = series of per-day events) and
the Legacy URLs paragraph; the "One venue, one span, one event"
principle.
- [behaviors/legacy-id-mapping.md](../specs/behaviors/legacy-id-mapping.md#legacy-url-forms-we-accept) —
`/events/<handle>` and segment rows now point at the series page.

## Approach

1. **Configs.** `events.toml`: `path = '${{ startsAt.slice(0, 4) }}/${{ slug }}'`
plus `[gitsheet.format] type = 'markdown', body = 'description'`.
`event-series.toml`: same format block, path unchanged.
`event-rsvps.toml`: `path = '${{ year }}/${{ eventSlug }}/${{ personSlug }}'`.
One commit per data branch (empty, fixture, legacy-import), PRs, then
mirror `empty`'s `.gitsheets/` into `apps/api/gitsheets/`. Contract
documents unchanged; `gitsheets contracts verify` before each push.
2. **Write path.** `withRsvpPath` adds `year` (the event's partition year,
`startsAt.slice(0, 4)`) next to `eventSlug` and `personSlug`; the sheet
guard test lists it under `WRITE_INJECTED_PATH_FIELDS['event-rsvps']`.
Event writes need nothing: gitsheets renders the year from `startsAt`
and splits `description` into the body on write, joins on read.
3. **Rename moves.** A slug rename already rewrites each RSVP file; a
`startsAt` change that crosses a year boundary must now move the event
file and every RSVP file too — handle both in `EventWriteService.update`
by deleting the old paths before the upsert whenever the partition year
or slug changes.
4. **Redirects.** `plugins/legacy-redirect.ts`: `/events/<handle>` whose
handle is a live series slug (and not a live event slug) 301s to
`/events/series/<handle>`; segment URLs 301 to the series page.
5. **Tests + seed.** Rename/boundary move cases; the redirect rows; the
events tests keep passing against markdown records (assert a created
event's file lands at `events/<year>/<slug>.md` via `git ls-tree`).

## Validation

- [ ] A created event lands at `events/<YYYY>/<slug>.md` with its
description as the markdown body; reading it back through the API is
byte-for-byte the description that was sent.
- [ ] An RSVP lands at `event-rsvps/<YYYY>/<eventSlug>/<personSlug>.toml`.
- [ ] Moving an event's `startsAt` across a year boundary moves its file
and its RSVP files in the same commit; renaming its slug still does.
- [ ] A series' description is the body of `event-series/<slug>.md`.
- [ ] `/events/launchpad-2023` 301s to `/events/series/launchpad-2023` when
that series exists and no event has that slug.
- [ ] `check:sheet-configs` matches `empty`; `check-contracts` unchanged.
- [ ] `npm run type-check && npm run lint && npm test` clean.

## Risks / unknowns

- **Markdown body + agenda descriptions.** Only the event's own
`description` becomes the body; agenda item descriptions stay as
frontmatter strings. Confirm gitsheets serializes an array of tables
with multi-line strings in frontmatter cleanly (blog posts have no
nested arrays).
- **Year partition is UTC.** `startsAt.slice(0, 4)` is the UTC year, so a
New Year's Eve evening in Philadelphia files under the next year. It is
a storage partition, invisible in URLs; accepted for determinism.

## Notes

(Populated at closeout.)

## Follow-ups

(Populated at closeout.)
Loading
Loading