PkVision — Roadmap

Infrastructure

Docker + docker-compose — Containerize the full stack (API + worker + Redis + PostgreSQL). Single docker-compose up to run everything. GPU passthrough support for training with NVIDIA Container Toolkit. Separate Dockerfile.api (lightweight, inference only) and Dockerfile.train (full ML deps + CUDA/MPS).
CI/CD with GitHub Actions — Auto-run pytest on every push/PR. Lint with ruff. Type-check with mypy. Block merge if tests fail. Separate workflows for test.yml (fast, no GPU), train-smoke.yml (weekly, runs 5-epoch training on fixture data to catch regressions).
PostgreSQL migration — Switch from SQLite to PostgreSQL for production. Alembic for schema migrations. Keep SQLite as dev default.
Pre-commit hooks — ruff format + ruff check + mypy on staged files.

Live webcam analysis — WebSocket endpoint that streams detections in real-time. YOLO processes frames at ~15fps, detection runs on sliding windows, scores update live.
RTMP/RTSP stream input — Accept live video feeds from competition cameras. Integrate with OBS or professional streaming setups.
Low-latency mode — Optimized pipeline for sub-second detection: skip frames, reduce YOLO input resolution, batch inference.
Live overlay — OpenCV overlay on video feed showing skeleton, detected trick name, confidence bar, running score.

Accuracy benchmark suite — Standard test set of labeled clips with ground truth. Report per-trick precision, recall, F1. Compare angle threshold vs ST-GCN accuracy.
Confusion matrix dashboard — Visual confusion matrix after each training run. Identify which tricks get confused (e.g. gainer vs back flip).
Latency profiling — Measure end-to-end time: video load → pose extraction → detection → scoring. Track per-component timing. Target: < 2x video duration for offline, < 100ms per frame for real-time.
Model versioning — Track model versions with metrics (accuracy, loss, training data size). MLflow or W&B integration for experiment tracking.
Regression tests — Golden test set that must maintain >= X% accuracy. Fail CI if a model change drops below threshold.

3D pose estimation — Integrate MotionBERT or VideoPose3D for monocular 3D pose lifting. Critical for twist detection (rotations along camera axis).
Multi-camera fusion — Combine 2+ camera angles for true 3D keypoints. Triangulation pipeline. Required for competition-grade accuracy.
Execution quality scoring — Rate landing stability, body alignment, height. Separate D-score (difficulty) and E-score (execution) like FIG gymnastics.
Combo detection — Detect trick sequences (back flip → twist → landing). Score combos with flow/transition bonuses.
Trick phase visualization — Show approach/takeoff/execution/landing phases overlaid on the video timeline.

Web dashboard — Next.js frontend for uploading videos, viewing results, browsing the trick catalog. Dark theme, responsive.
Clip submission portal — Web form (not just GitHub Issues) for athletes to submit clips. Upload to S3/Blob, auto-notify maintainers.
Leaderboard — Public leaderboard of highest-scoring runs submitted by the community.
Mobile app — React Native app for filming + instant analysis. Film a trick, get immediate feedback on what was detected.
Multilingual catalog — Expand beyond EN/FR: ES, DE, PT, JP, AR. Community-contributed translations.
Plugin system — Allow third-party detection strategies. Community can train specialized models for niche tricks and share them.

FIG notation export — Export analysis results in FIG-compatible notation format. Align difficulty ratings with official Code of Points.
Judge tablet interface — iPad-optimized UI for competition judges. View AI suggestions, apply overrides, submit final scores.
Multi-athlete tracking — Detect and track multiple athletes in the same frame. Assign tricks to specific athletes.
Competition mode — Locked-down mode with audit logging, no model updates during competition, tamper-evident results.
Replay system — Slow-motion replay with skeleton overlay for judges to review contested detections.