Hairong Jiang

Education

Doctor of Philosophy
ISAT Laboratory, School of Industrial Engineering, Purdue University, West Lafayette (Jan 2011 - Aug 2015)
Institute of Accessible Science, Purdue University (Jan 2011 - Dec 2013)
- Thesis: Effective and Interactive Interpretation of Gestures by Individuals with Mobility Impairments (Confidential)
- Advisors: Juan P. Wachs and Bradley S. Duerstock

Master of Science in Electrical Engineering
Electrical Engineering, Harbin Institute of Technology, Shenzhen, China (Sep 2008 - Dec 2010)
- Thesis: Controller Parameters Tuning for Wire Bonder based on Data-driven Method
- Advisor: Zexiang Li

Bachelor of Science in Electrical Engineering
Electrical Engineering, Harbin Institute of Technology, Harbin, China (Sep 2004 - Aug 2008)
- Thesis: Linear Motor Driven Control System Design
- Advisor: Jie Ma

Relevant Courses

Computer Sciences & Engineering
- CS 590: Statistical Machine Learning (Fall 2011)
- ECE 637: Digital Image Processing (Spring 2012)
- ECE 661: Computer Vision (Fall 2012)
- IE 590: Robotics Machine Vision (Spring 2012)
- Coursera: Machine Learning (Statement) (Summer 2014)
- Coursera: Programming for Everybody (Python) (Statement) (Spring 2015)
- Coursera: Discrete Inference and Learning in Artificial Vision (Statement) (Spring 2014)
- Computer Vision (Fall 2009)

Mathematics & Statistics
- STAT 511: Statistical Methods (Fall 2011)
- STAT 529: Bayesian Decision Theory (Summer 2012)
- IE 537: Optimization Models and Algorithms (Spring 2013)
- IE 633: Dynamic Programming (Spring 2014)
- Real Analysis (Spring 2009)
- Convex Optimization (Fall 2009)

Robotics & Control
- IE 633:Dynamic Programming (Spring 2014)
- Robotic Manipulation (Fall 2009)
- Principles of Automatic Control
- Fundamental Theory of Modern Control System
- System Modeling and Simulation
- Motion Control System Design

Recent Research

1. Effective and Interactive Interpretation of Gestures by Individuals with Mobility Impairments

There has been an increasing attention in the adoption of commodity cameras and gaming based technology to support and complement rehabilitation therapies for users with upper extremity mobility impairments (UEMIs). Innovative applications leveraging on Kinect® and Wii® based technologies have been integrated in rehabilitation programs as part of a new paradigm often referred as exergaming – exercising while playing. These platforms involve the use of physical expressions, such as gestures, as the main form of interaction. Such platforms offer an alternative to traditional rehabilitation sessions, in order to improve rehabilitative outcomes and prevent rehospitalization. The problem is that such platforms rely on gesture interfaces, which are designed primarily for individuals without motor limitations, thus excluding a critical mass of users with some degree of UEMI who could benefit from this technology. The assistive technologies (AT) community has addressed this challenge through customizing hand gesture-based interfaces for specific quadriplegic users, which is tedious, time-consuming, and costly. There is no systematic method to convert existing gesture interfaces (designed for individuals without disabilities) to usable interfaces for persons with UEMIs.

The objective of this research is to solve this hurdle by proposing a framework to establish guidelines, metrics, and procedures for the design of gesture sets (lexicons) suitable for users with UEMIs using fundamentally scientific sound principles. The key idea is to project the existing patterns of gestural behavior from persons without disabilities to match those exhibited by users with quadriplegia due to common cervical spinal cord injuries (SCIs). Two approaches (a user-centered and an analytic approach) have been developed and validated to provide users with quadriplegia with both individualized and universal solutions. The feasibility of the proposed methodology was validated through simulation and user-based experiments. Through these studies, it was found that subjects with UEMIs preferred gestures generated by our approach rather than the standard gestures (thirty-six out of forty-two constrained gestures).Gesture-variability analysis was conducted to further validate the gesture sets, and finally robotic execution was used to mimic gesture trajectories. Based on this, a physical metric (referred as work) was empirically obtained to compare the physical effort of each gesture. An integration method was presented to determine the accessible gesture set based on the stability and empirical robot execution. For all the gesture types, the accessible gestures were found to lie within 34% of the optimality of stability and work. Lastly, the gesture set determined by the proposed methodology was practically evaluated by target users in experiments while solving a spatial navigational problem.

Selected paper: An Analytic Approach to Decipher Usable Gestures for Quadriplegic Users

[top]

2. Integrated Gesture Recognition based Interface for People with Upper Extremity Mobility Impairments

Proposed a 3D particle filter framework based on color and depth information for robust face and hand tracking

Incorporated an interaction model utilizing spatial and motion information to tackle the “false merge” and “false labeling” problem through hands’ interaction and occlusion

Obtained an optimized parameters set using neighborhood search and achieved a tracking accuracy of 98.81%

Encoded acquired trajectories using Dynamic Time Warping and obtained 95.9% on dynamic hand gesture recognition using extended CONDENSATION algorithm

Applied the proposed gesture-based interface to multi-robot control and collaboration

Implemented gesture-based interface using C++ with Windows platform and robotic control using KAREL, and Robot Operation System (ROS) with Linux platform

Selected papers: A Machine Vision-Based Gestural Interface for People with Upper Extremity Physical Impairments; An Optimized Real-time Hands Gesture Recognition based Interface for Individuals with Upper-level Spinal Cord Injuries

[top]

3. Multimodal Vision-based System for Efficient, Automated Control of a Robotic Manipulator

Developed a novel multimodal vision-based robotic system containing two Kinect sensors, a wheelchair mounted commercial robotic arm allowing users with motor impairments to use natural interaction (recognized gesture and speech commands in real time) reliably

Investigate 2D/3D object recognition algorithms to detect and recognize daily living objects

Implemented the system with C++, C#, LibSVM library, OpenCV, and GNU

Selected papers: Integrated Vision-based System for Efficient, Semi-automated Control of a Robotic Manipulator; Autonomous Performance of Multistep Activities with a Wheelchair Mounted Robotic Manipulator Using Body Dependent Positioning

[top]

4. 3D Joystick for Robotic Arm Control by Individuals with High Level Spinal Cord Injuries

Developed an innovative 3D joystick to enable independent and efficient operation of a robotic arm

Compared different input modalities and evaluated the system using the Fitts’s law measurement

Implemented the 3D joystick with C++ and robotic control with C#

Selected paper: 3D Joystick for Robotic Arm Control by Individuals with High Level Spinal Cord Injuries

[top]

Publications

[top]

Teaching & Working Experience

Teaching Assistant, School of Industrial Engineering, Purdue University (Fall 2013, Spring 2015)

IE 474 Industrial Control Systems

System Engineer, Parallax Precision CO. LTD, China (Aug 2009 - Dec 2010)

Designed motion control systems for Wire Bonding Machines (WBMs)

Investigated algorithms for multi-loop precision position and force control for WBMs

Developed algorithms to optimize WBMs’ motion trajectories and reduce vibration

[top]

Patents

Bradley S. Duerstock, Juan P. Wachs, and Hairong Jiang, ''Gesture Translation for Persons with Mobility Impairments'', Pending

Wei Cheng, Xinlu Yu, Jianjun Min, and Hairong Jiang, ''A Four-loop Precision Position/Force Hybrid Control Approach for Golden Wire Bonders'', CN101969034 A, September 3, 2010

[top]

Honors & Awards

Second Place of Poster Competition, Industrial Engineering Research Symposium, April 2015, Purdue University

Second Place of Poster Competition, Industrial Engineering Research Symposium, April 2014, Purdue University

Outstanding Graduate Student Award, 2010, Harbin Institute of Technology, China

National Scholarship, 2008, 2009, 2010, Harbin Institute of Technology, China

People's Scholarship, 2005, 2006, 2007, Harbin Institute of Technology,China

[top]

Other Activities

Volunteer, IEEE Conference on System, Man, and Cybernetics (SMC), 2014

Reviewer, IEEE Transactions on Biomedical Engineering, June, 2015

Reviewer, International Journal of Pattern Recognition and Artificial Intelligence, Aug, 2014

Member, Purdue Robotics and Automation Society, 2014-present

Member, IEEE Computer Society, 2014-present

Member, IEEE Robotics and Automation Society, 2014-present

Member, IEEE System, Man and Cybernetics Society, 2014-present

Member, IEEE Communication Society, 2014-present

Member, ACM Society, 2014-present

Host, 201 Studio, Harbin Institute of Technology, 2005-2006, China

Social Head, Qitian Science and Technology Symposium, Harbin Institute of Technology, 2005-2006, China

[top]

Proejcts & Insights

[top]

Dr. Hairong Jiang