From 16e71a69d5b33ceca0bb9fc4e199471ba20cc2be Mon Sep 17 00:00:00 2001 From: Peter Stockings Date: Wed, 30 Sep 2026 20:46:58 +1000 Subject: [PATCH] docs: add running dashboard implementation plan --- .gitignore | 45 ++++++++ IMPLEMENTATION_PLAN.md | 243 +++++++++++++++++++++++++++++++++++++++++ README.md | 14 +++ 3 files changed, 302 insertions(+) create mode 100644 .gitignore create mode 100644 IMPLEMENTATION_PLAN.md create mode 100644 README.md diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..c8423d8 --- /dev/null +++ b/.gitignore @@ -0,0 +1,45 @@ +# Credentials and local configuration +.env +.env.* +!.env.example +garmin_tokens.json +oauth1_token.json +oauth2_token.json +*token*.json +.garminconnect/ +.garth/ +secrets/ +instance/ + +# Private activity data +*.fit +*.FIT +*.tcx +*.gpx +uploads/ +data/ +backups/ +*.sqlite +*.sqlite3 +*.db + +# Python and tooling +.venv/ +venv/ +__pycache__/ +*.py[cod] +.pytest_cache/ +.ruff_cache/ +.mypy_cache/ +.coverage +htmlcov/ +*.egg-info/ +build/ +dist/ + +# Local/editor files +.idea/ +.vscode/ +.DS_Store +Thumbs.db +*.log diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md new file mode 100644 index 0000000..11b534c --- /dev/null +++ b/IMPLEMENTATION_PLAN.md @@ -0,0 +1,243 @@ +# Multi-user running dashboard: implementation plan + +Status: planning only; application not yet implemented. +Date: 2026-09-30 + +## Product and agreed decisions + +Build a self-hosted running platform with user accounts, public profiles, +activity pages, and eventually social features similar to Strava. Initial device +is a Garmin Forerunner 265, but the data model must support other sources. + +- Python, Flask, Jinja templates, PostgreSQL, SQLAlchemy and Flask-Migrate. +- Leaflet for routes; Chart.js for graphs; Gunicorn for production HTTP. +- `python-garminconnect` for unofficial Garmin integration; `fitdecode` for FIT parsing. +- Users authenticate to Garmin on their own computer with a helper script and + upload exported session tokens. The website does not collect Garmin passwords or MFA codes. +- No dependency on approval for the official Garmin developer program. +- Manual FIT uploads remain available independently of Garmin connectivity. +- One repository, one Docker image, one Dokku application. Web and background + worker are separate processes of that application. +- Original activity files and provider credentials are private. Publishing is + explicitly configured by the activity owner; new accounts default to private imports. + +## Architecture + +Browser -> Flask web process -> PostgreSQL and private activity file storage. +Local Garmin helper -> user uploads token bundle -> connection validation job. +Background worker -> Garmin -> shared FIT import pipeline -> owned activities. +Public pages -> visibility checks -> sanitized public activity data. + +Start with a PostgreSQL-backed job table to avoid adding Redis for the initial +deployment. The worker claims jobs transactionally, leases them, retries with +backoff, and recovers expired leases. Keep this small; reconsider RQ/Celery and +Redis if queue complexity or throughput warrants it. Do not run a scheduler in +each Gunicorn worker. Schedule due connections from the background worker with +unique job keys and per-connection locks. + +Suggested layout: + +```text +app/ + __init__.py # application factory + config.py + extensions.py + models/ + auth/ # signup, sessions, recovery + profiles/ + activities/ + connections/ # upload, validation, disconnect + admin/ # owner controls and site administration + services/ + garmin_adapter.py + credentials.py + fit_import.py + publishing.py + jobs.py + templates/ + static/ +tools/ # local authentication/export helper +migrations/ +tests/ +docs/ +Dockerfile +Procfile +pyproject.toml +``` + +## Phase 0: prove the token workflow + +- [ ] Select and pin a compatible Python and `python-garminconnect` version for + both helper and server; lock dependencies. Do not assume old garth exports work. +- [ ] Write a local helper with hidden password input and interactive MFA. + Authenticate directly with Garmin and export the supported session JSON. +- [ ] Wrap exports with an explicit bundle format and library version, or accept + the pinned library's raw token JSON with a documented compatibility check. +- [ ] Test with the owner's account: local login, export, load in a Linux + container, identify Garmin account, and download one recorded run. +- [ ] Repeat from the intended Dokku host. Confirm that session reuse and refresh + work from that host; record region/network limitations rather than assuming portability. +- [ ] Confirm refreshed credentials are captured and persisted; exercise expired + or revoked sessions and the reconnect state. +- [ ] Define whether the exported session remains in local use. Avoid simultaneous + helper/server refreshes of a shared credential set. +- [ ] Document that session tokens grant ongoing Garmin account access and are + not an app-specific read-only permission grant. + +Exit: a documented end-to-end local-to-host import succeeds without sending a +Garmin password to the website. If it fails, keep FIT uploads usable and record +the integration blocker before building token-based onboarding. + +## Phase 1: application foundation and accounts + +- [ ] Scaffold Flask application factory, environment configuration, migrations, + structured redacted logging, and health endpoints. +- [ ] Add PostgreSQL local development configuration and dependency lockfile. +- [ ] Implement signup, unique normalized email/username, password hashing, + login/logout, email verification, and single-use expiring password reset links. +- [ ] Use Flask-Login, CSRF protection, secure session cookies, login throttling, + and a configured email delivery provider. Keep credentials out of cookies. +- [ ] Add user time zone, display name, profile slug, and publication defaults. +- [ ] Separate site administrator permissions from activity ownership. +- [ ] Include an option to restrict signup to invitations during the pilot. + +Exit: two users can register independently; protected pages and mutations enforce +identity and ownership; recovery and verification work with test email delivery. + +## Phase 2: import and activity model + +- [ ] Create models and migrations: + - User: identity, authentication, profile, time zone, settings. + - ExternalConnection: owner, provider/account identity, encrypted credential + blob/version, status, sync cursor, last success/error, refresh metadata. + - Activity: owner, source ID, timestamps, sport, title, visibility, summary metrics. + - ActivityLap and ActivitySample: splits and timestamped measurements. + - ActivityFile: private storage key, checksum, size, import provenance. + - ImportJob: type, owner/connection, status, attempts, lease, idempotency key. + - PrivacyZone: owner and private geographic masking settings. +- [ ] Implement bounded FIT uploads and parsing through one shared service. + Validate contents, use generated storage paths, and bound resource consumption. +- [ ] Extract GPS, elapsed/timer time, distance, heart rate, cadence, elevation, + laps, and available running metrics. Handle missing fields and paused periods. +- [ ] Keep canonical units and UTC timestamps; render user time zones and units. +- [ ] Deduplicate provider activities by connection/source ID and repeated files + by owner/checksum. Define reconciliation for a manual file later fetched from Garmin. +- [ ] Preserve original files privately and support controlled reprocessing. +- [ ] Handle malformed/unsupported files with actionable import errors. + +Exit: a manual FIT upload produces correct summary, route, laps, and charts; +repeated imports do not create duplicates; failed imports do not leave partial activities. + +## Phase 3: user-managed Garmin connections and automatic sync + +- [ ] Add authenticated token upload with CSRF, strict size/schema/version checks, + and rejection of client-supplied paths or arbitrary provider URLs. +- [ ] Validate the uploaded session against Garmin before enabling sync. Obtain + provider account identity from Garmin rather than trusting uploaded metadata. +- [ ] Bind connections to the current user; enforce one owner per Garmin account + initially and require an explicit replacement flow for changing accounts. +- [ ] Encrypt credentials at application level; supply versioned encryption keys + through Dokku config, separately from database backups; document rotation. +- [ ] Adapt library session loading/saving to per-user encrypted persistence. + If temporary files are necessary, restrict access and remove them reliably. + Never use the library's default shared home-directory token store for all users. +- [ ] Provide connection status, last sync, manual sync, reconnect, and disconnect. +- [ ] Poll connected accounts approximately every 15–30 minutes with jitter, + bounded concurrency, rate-limit backoff, and one active refresh/sync per connection. +- [ ] Persist refreshed credentials atomically and protect against stale writes. +- [ ] Backfill history in bounded jobs with progress and retry support. +- [ ] Periodically revisit recent activities for edits; define upstream deletion + handling explicitly instead of deleting local data based on an absent result. +- [ ] Disconnect cancels pending jobs and deletes stored credentials; prevent + an in-flight worker from restoring credentials or importing after disconnect. +- [ ] Explain that deleting the server's token copy may not revoke other copies. + Do not claim provider-side revocation unless verified for this integration. + +Exit: two accounts sync without crossing credentials or data; restarts preserve +progress; expired sessions request reconnect; Garmin downtime leaves the site usable. + +## Phase 4: public dashboard and privacy + +- [ ] Build responsive profile, activity list, run detail, and personal dashboard pages. +- [ ] Add weekly/monthly mileage, pace, elevation, splits, and heart-rate charts. +- [ ] Use Leaflet routes with a chosen map tile provider, attribution, and usage policy. +- [ ] Add owner editing, publish/unpublish, delete, and configurable auto-publishing. +- [ ] Support private and public visibility first; add followers-only with Phase 5. +- [ ] Apply authorization to HTML, JSON, chart data, downloads, feeds, and caches. +- [ ] Mask private route zones on the server before public serialization; do not + leak original coordinates through chart payloads or connect lines across masked gaps. +- [ ] Make originals downloadable only by their owner initially. +- [ ] Invalidate public caches when visibility or masking settings change. +- [ ] Define aggregates so private runs cannot leak through public totals or feeds. + +Exit: public visitors see only published, sanitized data; another signed-in user +cannot access private activity details or files by guessing IDs. + +## Phase 5: social MVP + +- [ ] Add following/unfollowing with private follow requests where applicable. +- [ ] Add a paginated feed filtered by current visibility and relationship state. +- [ ] Implement followers-only activities and test unfollow/block effects. +- [ ] Add account blocking, reporting, and basic site moderation before broad signup. +- [ ] Add kudos and comments after the core feed, with ownership checks and abuse limits. + +Deferred: challenges, segments, leaderboards, live tracking, mobile apps, coaching, +additional wearable integrations, and a separate frontend application. + +## Phase 6: Dokku deployment and operations + +- [ ] Build one Linux image with web and worker commands; configure a Procfile. +- [ ] Provision PostgreSQL and private persistent storage; deploy one Dokku app + with one web and one worker process initially. +- [ ] Configure HTTPS/proxy handling, domain, email, Flask signing key, encryption + keys, database URL, and storage paths outside source control. +- [ ] Run migrations as a controlled release step; avoid racing worker startup. +- [ ] Add worker heartbeat, import failure visibility, bounded logs, and alerts for + stuck jobs or repeated authentication failures without logging secrets. +- [ ] Back up database and private files; protect encryption keys separately; + rehearse restore and document image rollback/migration compatibility. +- [ ] Implement account deletion covering connections, files, activities, social + data, pending jobs, and documented backup retention. +- [ ] Publish onboarding and operations documentation, including token generation, + upload, reconnect, privacy settings, and manual FIT fallback. + +## Verification strategy + +- Unit tests: FIT metric calculation, malformed files, duplicate detection, + privacy masking including repeated zone crossings, and credential serialization. +- Integration tests against PostgreSQL: per-user authorization, refresh races, + leases/retries, disconnect races, migrations, and transactional imports. +- Browser tests: register -> upload FIT -> inspect private run -> publish -> + public view -> unpublish; local-token connection with mocked Garmin responses. +- Use synthetic or explicitly consented and sanitized FIT fixtures. Never commit + real tokens, passwords, home coordinates, or personal raw recordings. +- Keep live Garmin tests opt-in; run the Phase 0 smoke test with the owner's account. +- Deployment smoke test: restart web/worker, retry a failed job, verify persistent + files, and restore a backup in an isolated environment. + +## Delivery milestones + +1. Token portability proven and integration constraints recorded. +2. Accounts plus manual FIT upload and private activity view. +3. Public profiles, publishing controls, and server-side route privacy. +4. Per-user token upload, automatic sync, and reconnect flow. +5. Following/feed and operational readiness for invited users. +6. Broader signup after privacy, recovery, moderation, and restore checks pass. + +## Decisions to resolve during implementation + +- Product name and public domain; repository folder is a working name. +- Initial signup mode, email provider, and map tile provider. +- History backfill limit and file/account quotas. +- Public aggregate semantics and handling of activity updates/deletions upstream. +- Remote Git hosting/provider, visibility, and repository name if remote hosting is desired. + +## References + +- https://github.com/cyberjunky/python-garminconnect +- https://github.com/cyberjunky/python-garminconnect/blob/master/example.py +- https://github.com/cyberjunky/python-garminconnect/blob/master/garminconnect/client.py +- https://dokku.com/docs/deployment/builders/dockerfiles/ + +The library's authentication format is version-sensitive. Verify these sources +against the pinned release during Phase 0 rather than relying on moving master. diff --git a/README.md b/README.md new file mode 100644 index 0000000..e05bc56 --- /dev/null +++ b/README.md @@ -0,0 +1,14 @@ +# Running dashboard + +A planned multi-user, self-hosted running dashboard built with Flask and deployed +as one Dokku application. + +Users will upload FIT files or connect Garmin by authenticating locally with +`python-garminconnect` and uploading their exported session tokens. The server +will import activities and provide profiles, charts, maps, and publishing controls. + +See [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md) for the architecture, +phased checklist, acceptance criteria, and deployment plan. + +Current status: planning only. No application or deployment has been created. +Do not add Garmin credentials, token exports, or personal activity files to Git.