Beyond Accessibility: Unleashing the Potential of Home Automation to Foster Independence and Inclusivity for Individuals with Unique Needs
2024 - EPFL (CS-358) MIT - Group project
Helping Hand is an innovative project designed to enhance the independence of individuals with mobility impairments through adaptable home automation. The core of the project is a modular box equipped with a Wi-Fi enabled microcontroller and a plotter like mechanism that allows remote control of various household appliances. This system can be operated via a smartphone or WEB application, making it accessible for users with limited physical capabilities.
Inspired by the needs of Sean, a person with ALS who has limited mobility, Helping Hand aims to facilitate tasks such as opening doors and operating blinds without assistance. The modular box can accommodate different remote controls, allowing users to control multiple devices through a single, user-friendly interface. The project prioritizes accessibility, ease of use, and adaptability, making it a versatile solution for various remote-controlled appliances.
Note: you can check out the original project's proposal: Helping Hand project proposal.
Below is a brief explanation of this repository's structure to provide context and aid readers' comprehension in the following sections of this document.
codedirectory: contains all the code for the different parts of the project.CVserver: the python server side program responsible for image processing and remote buttons automatic detection and configuration.esp32-cam: the ESP-32 Cam Arduino code to be able to capture an image using an HTTP request.esp32-controller: the ESP-32 C6 code that is responsible to control the motors and press the remote's buttons on the main plate.helping_hand: the Flutter code used to create the application from which we control the device and interact with the different microcontrollers in the system.
designdirectory: contains all the 3D modeling files and exports along with the laser cut sheets.diagramsdirectory: contains all the protocol diagrams specifying how the different parts of the system interact with one another.documentationdirectory: contains all the pictures and schematics to be able to reproduce this project.proposaldirectory: the LaTeX document used to write the original project proposal.reportsdirectory: our first meetings reports to agree on what to do together.
This section is all about providing instructions and material to be able for anyone to reproduce this project by themselves.
This project is mainly composed of electronic components, laser cut 4mm MDF plates and 3D printed parts.
Here are the steps to follow to build the main body (and the camera module) of the system:
- Laser cut the following MDF (4mm) parts
- 3D print the required parts (PETG)
- For the assembly you will also need the following parts:
- One 8mm diameter stainless steel bar, 290mm of length
- One 8mm diameter stainless steel bar, 305mm of length
- Two 8mm diameter stainless steel bar, 160mm of length
- Four 8mm rod bearings to slide on the bars
- Two 5mm diameter stainless steel bars, 150mm of length
- Four M4 screws with nuts (two are nylon insert lock nuts), 35mm of length
- Four M4 screws with nuts, 12mm of length
- Eight M3 screws with nuts, 10mm of length
- One M5 screw with nut, 20mm of length
- Two M5 screws with nuts and spacers, 20mm of length
- Eight M3 screws, 12mm of length
- Two M2.5 screws with nuts, 12mm of length
- Eight neodymium magnets, cubes of 5mm sides
- Two rubber sheets, 30mm by 12mm
- Two plastic belts, 610mm and 320mm
- End gears and axes for the connection between belt and motors
- Assemble everything and glue the MDF parts as indicated in the 3D step file: Full assembly .step file.
For reference, here are some pictures of the full result assembly:
Here are all the required electronic components:
- ESP-32 CAM
- ESP-32 C6
- Stepper mottors Nema14-01 (2 times)
- Motors drivers A4988 (2 times)
- Servo motor MG90S
- DC to DC converters (2 times)
- Two limit switches
- Condensator 100uF (2 times)
- Power Supply 9V
Wire everything together as indicated in the following schematic:
Note: make sure to leave some length on the cables used on the travelling parts.
For reference, here are some pictures of the resulting cable management (done under the board):
We have four system components that need to interact with eachother in order to function properly. The following diagram summarizes the different intractions between the components:
All the system is driven by a Flutter cross-platform application.
You can install the Flutter dev environment by following the installation instructions (on any OS): https://docs.flutter.dev/get-started/install.
Then navigate to the directory of the application, code/helping_hand.
You can now launch the application on your platform of choice, on web for example flutter run -d chrome.
To build for other targets, you can follow the standard Flutter building procedures: https://docs.flutter.dev/deployment.
Here are a few pictures of the application interface:
We use the Arduino IDE to flash the two ESP-32 boards:
- ESP-32 Cam (the automatic configuration module using the camera): flash the
code/esp32-cam/esp32-cam.ino. For more information about this module, please refer to the ESP-32 CAM README. - ESP-32 Plotter (the remote controller): flash the
code/esp32-controller/sketch_webServer/sketch_webServer.ino. For more information about this module, please refer to the ESP-32 C6 README.
We used a Python web server (running on any computer or in the cloud) powered by the Flask framework.
Refer to the associated README to setup the python server: Python CVserver Setup README.
- Yoan Giovannini (@unglazedstamp)
- Nour Guermazi (@nourguermazi)
- Lucas Jung (@gruvw)
- Pinar Oray (@pinar-oray)
- Jonas Sulzer (@violoncelloCH)