Business Challenge

The client approached the Itera Research team to develop a Proof of Concept (PoC) for a mobile application that seamlessly integrates various technologies, including wearable sensors embedded in specialized clothing (trousers and turtlenecks). 

The solution needed to combine mobile and web functionalities while ensuring a robust dashboard for healthcare providers, offering insights into patient progress, training plans, and data management.

A key challenge was to develop a mobile application using Flutter that could incorporate a Unity 3D rendering engine. This integration was critical for rendering a 3D model of the patient’s body and movements in real time while maintaining smooth communication between diverse programming libraries and frameworks.

Solution

The Itera Research team developed an innovative IoT-enabled mobile application tailored to meet the client’s needs. This application allows healthcare providers to create customized exercise plans and monitor patients’ movements in real time using clothing embedded with wearable sensors.

Mobile App Development

The mobile app was built using Flutter, offering a cross-platform solution for both iOS and Android devices. We integrated a Unity 3D component within the Flutter application to facilitate real-time 3D model rendering of the patient’s body, enabling dynamic visualization of movements and exercises.

Sensor Integration and Real-Time Data Exchange

The app communicates with wearable sensors via Bluetooth Low Energy (BLE) using the Nordic UART Service. This setup allows for real-time data exchange without complex pairing processes, simplifying user interaction.

To ensure accurate and reliable communication between the app and the sensors, we developed a custom protocol for data transfer. This protocol efficiently manages the data flow from the sensors, allowing healthcare providers to receive critical information about patients’ movements and performance in real time.

3D Model Rendering

Using Blender technology, our Unity developer created and calibrated 3D models based on sensor data to visualize patient movements accurately. Each sensor – gyroscope, magnetometer, and accelerometer – was calibrated using a Python utility to provide precise measurements based on individual patient metrics. 

This calibration was crucial for ensuring the accuracy of the 3D model rendering and delivering reliable feedback to both patients and healthcare providers.

User-Friendly Dashboard

The application features a comprehensive dashboard for healthcare providers, including progress tracking and statistical analysis of patient data. Healthcare providers can create training plans that incorporate text, photos, and videos from a library or custom-created content. 

Data Security 

To protect sensitive patient information, the application incorporates robust data security measures. All data transmitted between the wearable sensors and the mobile application is encrypted, ensuring patient privacy at all times. Additionally, the application complies with relevant healthcare regulations and standards, such as HIPAA, to safeguard user data.

Improved Patient Engagement

To promote adherence to training regimens and enhance overall patient compliance, the application includes features such as reminders and tutorials. Automated notifications remind patients of their scheduled exercises, ensuring they stay on track with their rehabilitation plans.

Interactive tutorials provide step-by-step guidance on performing exercises correctly, helping patients understand the importance of proper form and technique. 

The immersive 3D visualization feature engages patients, allowing them to see their movements in real time and receive immediate feedback on their performance. This interactive approach fosters a sense of ownership over their rehabilitation journey.

Key Features

• IoT-Enabled Integration: Leverages embedded sensors in clothing for real-time monitoring of patient movements during rehabilitation exercises.
• 3D Visualization: Utilizes Unity 3D to create immersive visual representations of patient movements, enhancing feedback and instruction.
• Personalized Exercise Plans: Empowers healthcare providers to design customized rehabilitation programs tailored to individual patient needs, incorporating various media types (text, photos, videos).
• Bluetooth Low Energy (BLE) Connectivity: Facilitates seamless communication between wearable sensors and the mobile application without complex pairing processes.
• User-Friendly Dashboard: Provides healthcare providers with a comprehensive interface for tracking patient progress, accessing analytics, and managing exercise plans.
• Automated Reminders: Sends notifications to patients to promote adherence to exercise regimens. 
• Pre-recorded Tutorial: Offers step-by-step guidance for correct exercise execution.
• Data Security: Implements encryption and secure authentication methods to protect sensitive patient information, ensuring compliance with healthcare regulations.

Tools & Technologies

• Flutter
• Unity 3D
• Bluetooth Low Energy (BLE)
• Nordic UART Service
• PHP
• Python
• Blender
• Real-time Data Transfer Protocol
• Custom protocol for data exchange

Results

The innovative IoT-enabled mobile application was successfully launched, providing a groundbreaking solution for remote rehabilitation. Key outcomes include:

• Enhanced Rehabilitation Effectiveness: Healthcare providers can track patient movements in real time, significantly improving the effectiveness of personalized rehabilitation plans.
• Improved Engagement and Adherence: Patients have access to tailored exercise programs that improve engagement and adherence with the immersive 3D visualization and interactive feedback features.
• Immediate Guidance: The integration of wearable sensors facilitates accurate monitoring, ensuring patients receive immediate guidance on their performance.

Looking ahead, the team plans to implement several enhancements, including real-time reminders and alerts for patients when their training regimens are not being followed correctly, a dedicated portal for doctors and administrators to access patient protocols, and progress tracking features.