Sistema lúdico orientado a la rehabilitación del miembro superior mediante detección de gestos markerless (MediaPipe) integrado en Unity

Abstract

This project develops a markerless serious-game system for upper-limb rehabilitation using MediaPipe Pose/Hands integrated in Unity. From RGB video, the system estimates body and hand landmarks, applies temporal filtering to stabilize measurements, and computes rehabilitation-oriented metrics (ROM and %ROM, angular velocity, reaction times, hits/misses, and active time). Gestures are mapped to in-game actions across shoulder, elbow, and fine-motor mini-games with immediate HUD feedback. The system logs each session and exports structured outputs (JSON and an automatic PDF report). Real-time tests showed stable operation and promising preliminary interaction and usability results.