wiki:2012/Projects/ICEMAN

Version 81 (modified by petlin, 4 years ago) (diff)

--

ICEMAN: Intelligent Circuit Equipped Mobile Autonomous Network


Team Members

  • Maxime Delachavonnery
  • Peter Lin
  • Geoffrey Oh
  • John Reed
  • Stephen Shanko

Abstract

The ICEMAN jacket is designed to process information from the environment and to provide the wearer with usable feedback. Through the use of sensors and actuators, the jacket is able to respond to various situations. Programmed in Arduino, the sensors are connected to processor boards, which are embedded in the jacket. The Arduino boards are connected via Bluetooth to an Android device, which processes and logs the data. The ICEMAN jacket has potential for a variety of applications, including driving, video gaming, sports training, rehabilitation, and workplace safety.


Usage

  • Accelerometer, magnetometer, and gyroscope chip sensors recognize hand and arm gestures
  • Heat sensors, light sensors, pressure sensors, alcohol sensors, and microphones used to obtain other miscellaneous information
  • LEDs and buzzers are programmed to respond to sensor inputs
  • Android application is used to output sensor data and to control the jacket

Hand Gestures

  • 9 Degrees of Freedom Stick (Accelerometer/ magnetometer/ gyroscope) recognizes hand and arm gestures
  • 9Dof Sticks are attached to both wrists and gyroscopes are attached to both upper arms
  • Yaw, pitch, and roll (angles of rotation in three dimensions) are recorded and processed
  • Sensor values are averaged over 1 second to minimize noise
  • Current values are compared with past averages to determine if a gesture has been completed
  • Example Gestures: Arm rotating upwards, Upsidedown arm rotating upwards, Wave, Jumping Jack
  • Android application is used to simulate arm movement
  • Direction Cosine Matrix used to find the frame of reference of ground from the frame of reference of the 9DoF Stick's plane

Bluetooth and I2C Connection

  • Having a Bluetooth module attached to each Arduino board ensures easy setup and duplication
  • Bluetooth implementation will require interrupts because smartphones can only read from 1 Bluetooth module at a time
  • SPI wire connection was explored but was too slow
  • SPI tranfer speed was tested to be less than 1 kbps
  • Bluetooth tranfer speed was tested to be around 31 kbps
  • Arduinos are not meant to be connected with each other, only with sensors and actuators (not designed to be the slave, only the master)
  • Arduino boards cannot use I2C to connect between each other
  • I2C is already being used by 9DoF chips, whose addresses cannot be changed, so sharing the I2C line will not allow us to differentate between the sensors
  • Manually created another set of I2C pins to allow connections between multiple Arduino boards and 9DoF sticks
  • Connections between 2 pins makes debugging much easier, but only byte transfer is allowed
  • Used the implementation of the Ports library created by Jeelabs, which provides more control over the function of each pin on the Lilypad but is slower than normal I2C pins because it does not have regular hardware interrupts


System Architecture

  • Consists of 2 programmable Arduino lilypad boards which are sewn into the jacket
  • Sensors and Actuators are attached via conductive thread to the Arduino boards
  • Conductive thread will be sprayed with nonconductive substance to prevent short circuits
  • Conductive thread was attached to wire at the ends, and then soldered to Arduino boards to prevent short circuits
  • Arduino boards are connected via I2C, which are controlled with the Android application via bluetooth

Android Application

  • Android App can pair with up to 8 Bluetooth devices but can only read from 1 of them at a time
  • Interrupts are needed but the data sampling rate between Arduino boards is still fast enough so that outputs are still nearly instantaneous
  • User can decide which type of sensor output will be displayed, either in text or graph form
  • Data for each run will be stored in a file and uploaded to a server

Weekly Progress

Week 1 - 05/21/2012

  • Received 9 Degrees of Freedom Stick (contains accelerometer, gyroscope, magentometer)
  • Used tutorials to start primitive programming of gesture recognition
  • Attempted to record wave files via the microphone

Week 2 - 05/28/2012

  • Programmed the Arduino processors to read sensor values via an I2C connection
  • Applied a direction cosine matrix to find the frame of reference of ground from the frame of reference of the 9DoF Stick's plane
  • Used the Processing Program to visually display the 9DoF stick's orientations

Week 3 - 06/04/2012

  • Created a new version of the Android Application
  • Explored bluetooth chat methodology for data transfer
  • Android App is now able to graphically display various sensor values from Arduino processors

Week 4 - 06/11/2012

  • Fixed sensor calibration errors of the 9DoF stick in order to display its orientation correctly in the Processing program
  • Yaw, pitch, and roll (angles of rotation in three dimensions) of the 9DoF stick can now be recorded and processed
  • Programmed Arduino processors to compare current sensor values with past averages to determine if a gesture has been completed

Week 5 - 06/18/2012

  • Programmed various gestures (Arm rotating upwards, Upsidedown arm rotating upwards, Wave, Jumping Jack, Change in yaw)
  • Researched changing the 9DoF stick address so that the I2C connection can be used with various 9DoF sticks and used to link the Arduino boards
  • Attempted to use multiplexers to deal with address conflicts

Week 6 - 06/25/2012

  • Programmed Arduino to output sensor values based on text commands (i.e. #ot, #oz, #oq, #og) either in text or binary
  • Fixed drift correction method of finding yaw, which can begin at any possible value in gestures and exceed the 0 and 180 degree thresholds
  • Designed and tested multiple architectures for connecting Arduino lilypads (Bluetooth, Analog Pin, SPI)
  • Bluetooth is easy to setup and duplicate but requires interrupts
  • Direct analog pin signals fluctuate and are too unstable
  • SPI wiring is software intensive but contains a library of preprogrammed commands

Week 7 - 07/02/2012

  • More testing of system architectures
  • Android devices cannot connect via bluetooth to other devices listed as Masters because a Master can only connect to a slave device
  • Lilypads connected via bluetooth can other read each other's data by retrieving bytes, so it is better to simply connect all lilypads as slaves to an Android device, which will act as the master
  • Using SPI wiring, the master can only send and receive data bytes from the slaves, so sending data between lilypads seems implausible - switching between master and slave cannot work
  • Arduino lilypad processors are meant to be Masters, not slaves, because they are able to send data flawlessly but can only read 1 data byte at a time

Week 8 - 07/09/2012

  • Improved SPI code so that gestures recognized by the slaves are transferred correctly to the master, but multiple slaves will slow the data transfer rate
  • Improved gesture recognition by adding a gyroscrope to the upper arm, with the 9DoF stick attached to the wrists
  • Android App can now simulate gestures in real time

Week 9 - 07/16/2012

  • Tested data transfer speeds between Bluetooth and SPI
  • Bluetooth tranfer speed was tested to be around 31 kbps
  • SPI tranfer speed was tested to be less than 1 kbps
  • Debugging of SPI code
  • With SPI, sensor values from the slave device are less accurate because they are only able to send bytes to the master

Week 10 - 07/23/2012

  • Soldered test wires connected to Bluetooth modules and sensors, because of connection problems
  • Tested reading serial output via bluetooth from different addresses
  • Researched coding of the Bluetooth piconet, a temporary network allowing communication of up to 8 devices

Week 11 - 07/30/2012

  • Worked on coding the Android App to connect to bluetooth modules, on each of the three lilypads
  • Worked on reading the Input Stream and writing to the Output Stream between the bluetooth modules
  • Began transfering the processing code over to Android from Arduino
  • Started sewing components into the jackets

Week 12 - 08/06/2012

  • Began working with I2C connections to connect the Lilypads
  • Coded to create new I2C pins, which would allow for the connection between multiple Arduino boards and the 9Dof sticks
  • Improved the Android App Interface

Week 13 - 08/13/2012

  • Cleaned up output format for possible future storage on an online server
  • Finished up sewing components onto the jackets
  • Updated the website & poster and created a demo

Attachments (23)

Download all attachments as: .zip