Disk Tray Cognitive Assistance

Cognitive assistance application for assembling a disk tray. In collaboration with the company inwinSTACK, we created a Gabriel application for training a new worker in disk tray assembly for a desktop.

Demo

This video demo was shown live at the Computex 2018 show in Taiwan in June 2018. The application was created by Junjue Wang of CMU, and demoed at Computex by inwinSTACK employees. The small size of some of the components (especially the pin) and the precise nature of the assembly were difficult challenges to overcome in creating this application. The wearable device used in this application is an ODG-7.

Installation

Client

An Android client is available on the Google PlayStore

Server

Running the server application using Docker is advised. If you want to install from source, please see Dockerfile for details.

How to Run

Client

From the main activity one can add servers by name and IP/domain. Subtitles for audio feedback can also been toggled. This option is useful for devices that may not have integrated speakers(like ODG R-7). Pressing the 'Play' button next to a server will initiate a connection to the Gabriel server at that address.

Server

Container

nvidia-docker run --rm -it --name disktray \
-p 0.0.0.0:9098:9098 -p 0.0.0.0:9111:9111 -p 0.0.0.0:22222:22222 \
-p 0.0.0.0:7070:7070 -p 0.0.0.0:8080:8080 \
-e "DISKTRAY_VIDEO_SERVER_URL=http://<server-public-ip-or-hostname>:8080"  \
-e "DISKTRAY_DEMO_SHOW_ANNOTATED_IMAGE=True" \
jamesjue/gabriel-disk-tray:latest

Source

1. Set following environment variables
   1. DISKTRAY_FASTER_RCNN_ROOT: root directory of py-faster-rcnn installation.
   2. DISKTRAY_VIDEO_SERVER_URL: video feedback server url.
   3. DISKTRAY_DEMO_SHOW_ANNOTATED_IMAGE: True or False, whether to show annotated image stream on Gabriel Debug Website.
2. Start the Gabriel Control and Ucomm server

docker run --rm --name gabriel -p 0.0.0.0:9098:9098 \
-p 0.0.0.0:9111:9111 -p 0.0.0.0:22222:22222 -p 10120:10120 \
-p 8021:8021 -p 9090:9090 -p 10101:10101 \
jamesjue/gabriel /bin/bash -c \
"gabriel-control -l -d -n eth0 & sleep 5; gabriel-ucomm -s 127.0.0.1:8021"

2. Download the asset file

bash -e scripts/download_asset.sh

2. Start the Video Instruction HTTP Server at the video feedback directory.

cd feedbacks/video
docker run -dit --name my-apache-app \
-p 8080:80 -v "$PWD":/usr/local/apache2/htdocs/ httpd:2.4

4. Run the DiskTray cognitive engine

disktrayapp -s 127.0.0.1:8021