docs: add running dashboard implementation plan

2026-09-30 20:46:58 +10:00
commit 16e71a69d5
3 changed files with 302 additions and 0 deletions
@@ -0,0 +1,45 @@
+# Credentials and local configuration
+.env
+.env.*
+!.env.example
+garmin_tokens.json
+oauth1_token.json
+oauth2_token.json
+*token*.json
+.garminconnect/
+.garth/
+secrets/
+instance/
+
+# Private activity data
+*.fit
+*.FIT
+*.tcx
+*.gpx
+uploads/
+data/
+backups/
+*.sqlite
+*.sqlite3
+*.db
+
+# Python and tooling
+.venv/
+venv/
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.coverage
+htmlcov/
+*.egg-info/
+build/
+dist/
+
+# Local/editor files
+.idea/
+.vscode/
+.DS_Store
+Thumbs.db
+*.log
@@ -0,0 +1,243 @@
+# Multi-user running dashboard: implementation plan
+
+Status: planning only; application not yet implemented.
+Date: 2026-09-30
+
+## Product and agreed decisions
+
+Build a self-hosted running platform with user accounts, public profiles,
+activity pages, and eventually social features similar to Strava. Initial device
+is a Garmin Forerunner 265, but the data model must support other sources.
+
+- Python, Flask, Jinja templates, PostgreSQL, SQLAlchemy and Flask-Migrate.
+- Leaflet for routes; Chart.js for graphs; Gunicorn for production HTTP.
+- `python-garminconnect` for unofficial Garmin integration; `fitdecode` for FIT parsing.
+- Users authenticate to Garmin on their own computer with a helper script and
+  upload exported session tokens. The website does not collect Garmin passwords or MFA codes.
+- No dependency on approval for the official Garmin developer program.
+- Manual FIT uploads remain available independently of Garmin connectivity.
+- One repository, one Docker image, one Dokku application. Web and background
+  worker are separate processes of that application.
+- Original activity files and provider credentials are private. Publishing is
+  explicitly configured by the activity owner; new accounts default to private imports.
+
+## Architecture
+
+Browser -> Flask web process -> PostgreSQL and private activity file storage.
+Local Garmin helper -> user uploads token bundle -> connection validation job.
+Background worker -> Garmin -> shared FIT import pipeline -> owned activities.
+Public pages -> visibility checks -> sanitized public activity data.
+
+Start with a PostgreSQL-backed job table to avoid adding Redis for the initial
+deployment. The worker claims jobs transactionally, leases them, retries with
+backoff, and recovers expired leases. Keep this small; reconsider RQ/Celery and
+Redis if queue complexity or throughput warrants it. Do not run a scheduler in
+each Gunicorn worker. Schedule due connections from the background worker with
+unique job keys and per-connection locks.
+
+Suggested layout:
+
+```text
+app/
+  __init__.py           # application factory
+  config.py
+  extensions.py
+  models/
+  auth/                # signup, sessions, recovery
+  profiles/
+  activities/
+  connections/         # upload, validation, disconnect
+  admin/               # owner controls and site administration
+  services/
+    garmin_adapter.py
+    credentials.py
+    fit_import.py
+    publishing.py
+    jobs.py
+  templates/
+  static/
+tools/                 # local authentication/export helper
+migrations/
+tests/
+docs/
+Dockerfile
+Procfile
+pyproject.toml
+```
+
+## Phase 0: prove the token workflow
+
+- [ ] Select and pin a compatible Python and `python-garminconnect` version for
+  both helper and server; lock dependencies. Do not assume old garth exports work.
+- [ ] Write a local helper with hidden password input and interactive MFA.
+  Authenticate directly with Garmin and export the supported session JSON.
+- [ ] Wrap exports with an explicit bundle format and library version, or accept
+  the pinned library's raw token JSON with a documented compatibility check.
+- [ ] Test with the owner's account: local login, export, load in a Linux
+  container, identify Garmin account, and download one recorded run.
+- [ ] Repeat from the intended Dokku host. Confirm that session reuse and refresh
+  work from that host; record region/network limitations rather than assuming portability.
+- [ ] Confirm refreshed credentials are captured and persisted; exercise expired
+  or revoked sessions and the reconnect state.
+- [ ] Define whether the exported session remains in local use. Avoid simultaneous
+  helper/server refreshes of a shared credential set.
+- [ ] Document that session tokens grant ongoing Garmin account access and are
+  not an app-specific read-only permission grant.
+
+Exit: a documented end-to-end local-to-host import succeeds without sending a
+Garmin password to the website. If it fails, keep FIT uploads usable and record
+the integration blocker before building token-based onboarding.
+
+## Phase 1: application foundation and accounts
+
+- [ ] Scaffold Flask application factory, environment configuration, migrations,
+  structured redacted logging, and health endpoints.
+- [ ] Add PostgreSQL local development configuration and dependency lockfile.
+- [ ] Implement signup, unique normalized email/username, password hashing,
+  login/logout, email verification, and single-use expiring password reset links.
+- [ ] Use Flask-Login, CSRF protection, secure session cookies, login throttling,
+  and a configured email delivery provider. Keep credentials out of cookies.
+- [ ] Add user time zone, display name, profile slug, and publication defaults.
+- [ ] Separate site administrator permissions from activity ownership.
+- [ ] Include an option to restrict signup to invitations during the pilot.
+
+Exit: two users can register independently; protected pages and mutations enforce
+identity and ownership; recovery and verification work with test email delivery.
+
+## Phase 2: import and activity model
+
+- [ ] Create models and migrations:
+  - User: identity, authentication, profile, time zone, settings.
+  - ExternalConnection: owner, provider/account identity, encrypted credential
+    blob/version, status, sync cursor, last success/error, refresh metadata.
+  - Activity: owner, source ID, timestamps, sport, title, visibility, summary metrics.
+  - ActivityLap and ActivitySample: splits and timestamped measurements.
+  - ActivityFile: private storage key, checksum, size, import provenance.
+  - ImportJob: type, owner/connection, status, attempts, lease, idempotency key.
+  - PrivacyZone: owner and private geographic masking settings.
+- [ ] Implement bounded FIT uploads and parsing through one shared service.
+  Validate contents, use generated storage paths, and bound resource consumption.
+- [ ] Extract GPS, elapsed/timer time, distance, heart rate, cadence, elevation,
+  laps, and available running metrics. Handle missing fields and paused periods.
+- [ ] Keep canonical units and UTC timestamps; render user time zones and units.
+- [ ] Deduplicate provider activities by connection/source ID and repeated files
+  by owner/checksum. Define reconciliation for a manual file later fetched from Garmin.
+- [ ] Preserve original files privately and support controlled reprocessing.
+- [ ] Handle malformed/unsupported files with actionable import errors.
+
+Exit: a manual FIT upload produces correct summary, route, laps, and charts;
+repeated imports do not create duplicates; failed imports do not leave partial activities.
+
+## Phase 3: user-managed Garmin connections and automatic sync
+
+- [ ] Add authenticated token upload with CSRF, strict size/schema/version checks,
+  and rejection of client-supplied paths or arbitrary provider URLs.
+- [ ] Validate the uploaded session against Garmin before enabling sync. Obtain
+  provider account identity from Garmin rather than trusting uploaded metadata.
+- [ ] Bind connections to the current user; enforce one owner per Garmin account
+  initially and require an explicit replacement flow for changing accounts.
+- [ ] Encrypt credentials at application level; supply versioned encryption keys
+  through Dokku config, separately from database backups; document rotation.
+- [ ] Adapt library session loading/saving to per-user encrypted persistence.
+  If temporary files are necessary, restrict access and remove them reliably.
+  Never use the library's default shared home-directory token store for all users.
+- [ ] Provide connection status, last sync, manual sync, reconnect, and disconnect.
+- [ ] Poll connected accounts approximately every 15–30 minutes with jitter,
+  bounded concurrency, rate-limit backoff, and one active refresh/sync per connection.
+- [ ] Persist refreshed credentials atomically and protect against stale writes.
+- [ ] Backfill history in bounded jobs with progress and retry support.
+- [ ] Periodically revisit recent activities for edits; define upstream deletion
+  handling explicitly instead of deleting local data based on an absent result.
+- [ ] Disconnect cancels pending jobs and deletes stored credentials; prevent
+  an in-flight worker from restoring credentials or importing after disconnect.
+- [ ] Explain that deleting the server's token copy may not revoke other copies.
+  Do not claim provider-side revocation unless verified for this integration.
+
+Exit: two accounts sync without crossing credentials or data; restarts preserve
+progress; expired sessions request reconnect; Garmin downtime leaves the site usable.
+
+## Phase 4: public dashboard and privacy
+
+- [ ] Build responsive profile, activity list, run detail, and personal dashboard pages.
+- [ ] Add weekly/monthly mileage, pace, elevation, splits, and heart-rate charts.
+- [ ] Use Leaflet routes with a chosen map tile provider, attribution, and usage policy.
+- [ ] Add owner editing, publish/unpublish, delete, and configurable auto-publishing.
+- [ ] Support private and public visibility first; add followers-only with Phase 5.
+- [ ] Apply authorization to HTML, JSON, chart data, downloads, feeds, and caches.
+- [ ] Mask private route zones on the server before public serialization; do not
+  leak original coordinates through chart payloads or connect lines across masked gaps.
+- [ ] Make originals downloadable only by their owner initially.
+- [ ] Invalidate public caches when visibility or masking settings change.
+- [ ] Define aggregates so private runs cannot leak through public totals or feeds.
+
+Exit: public visitors see only published, sanitized data; another signed-in user
+cannot access private activity details or files by guessing IDs.
+
+## Phase 5: social MVP
+
+- [ ] Add following/unfollowing with private follow requests where applicable.
+- [ ] Add a paginated feed filtered by current visibility and relationship state.
+- [ ] Implement followers-only activities and test unfollow/block effects.
+- [ ] Add account blocking, reporting, and basic site moderation before broad signup.
+- [ ] Add kudos and comments after the core feed, with ownership checks and abuse limits.
+
+Deferred: challenges, segments, leaderboards, live tracking, mobile apps, coaching,
+additional wearable integrations, and a separate frontend application.
+
+## Phase 6: Dokku deployment and operations
+
+- [ ] Build one Linux image with web and worker commands; configure a Procfile.
+- [ ] Provision PostgreSQL and private persistent storage; deploy one Dokku app
+  with one web and one worker process initially.
+- [ ] Configure HTTPS/proxy handling, domain, email, Flask signing key, encryption
+  keys, database URL, and storage paths outside source control.
+- [ ] Run migrations as a controlled release step; avoid racing worker startup.
+- [ ] Add worker heartbeat, import failure visibility, bounded logs, and alerts for
+  stuck jobs or repeated authentication failures without logging secrets.
+- [ ] Back up database and private files; protect encryption keys separately;
+  rehearse restore and document image rollback/migration compatibility.
+- [ ] Implement account deletion covering connections, files, activities, social
+  data, pending jobs, and documented backup retention.
+- [ ] Publish onboarding and operations documentation, including token generation,
+  upload, reconnect, privacy settings, and manual FIT fallback.
+
+## Verification strategy
+
+- Unit tests: FIT metric calculation, malformed files, duplicate detection,
+  privacy masking including repeated zone crossings, and credential serialization.
+- Integration tests against PostgreSQL: per-user authorization, refresh races,
+  leases/retries, disconnect races, migrations, and transactional imports.
+- Browser tests: register -> upload FIT -> inspect private run -> publish ->
+  public view -> unpublish; local-token connection with mocked Garmin responses.
+- Use synthetic or explicitly consented and sanitized FIT fixtures. Never commit
+  real tokens, passwords, home coordinates, or personal raw recordings.
+- Keep live Garmin tests opt-in; run the Phase 0 smoke test with the owner's account.
+- Deployment smoke test: restart web/worker, retry a failed job, verify persistent
+  files, and restore a backup in an isolated environment.
+
+## Delivery milestones
+
+1. Token portability proven and integration constraints recorded.
+2. Accounts plus manual FIT upload and private activity view.
+3. Public profiles, publishing controls, and server-side route privacy.
+4. Per-user token upload, automatic sync, and reconnect flow.
+5. Following/feed and operational readiness for invited users.
+6. Broader signup after privacy, recovery, moderation, and restore checks pass.
+
+## Decisions to resolve during implementation
+
+- Product name and public domain; repository folder is a working name.
+- Initial signup mode, email provider, and map tile provider.
+- History backfill limit and file/account quotas.
+- Public aggregate semantics and handling of activity updates/deletions upstream.
+- Remote Git hosting/provider, visibility, and repository name if remote hosting is desired.
+
+## References
+
+- https://github.com/cyberjunky/python-garminconnect
+- https://github.com/cyberjunky/python-garminconnect/blob/master/example.py
+- https://github.com/cyberjunky/python-garminconnect/blob/master/garminconnect/client.py
+- https://dokku.com/docs/deployment/builders/dockerfiles/
+
+The library's authentication format is version-sensitive. Verify these sources
+against the pinned release during Phase 0 rather than relying on moving master.
@@ -0,0 +1,14 @@
+# Running dashboard
+
+A planned multi-user, self-hosted running dashboard built with Flask and deployed
+as one Dokku application.
+
+Users will upload FIT files or connect Garmin by authenticating locally with
+`python-garminconnect` and uploading their exported session tokens. The server
+will import activities and provide profiles, charts, maps, and publishing controls.
+
+See [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md) for the architecture,
+phased checklist, acceptance criteria, and deployment plan.
+
+Current status: planning only. No application or deployment has been created.
+Do not add Garmin credentials, token exports, or personal activity files to Git.