Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors

Comments

Jill

Summary

In Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors, the researchers propose a new form of input for virtual games and other applications using the motion of the hand. While previous forms of input using the hand would use only the accelerometer or the EMG sensors, this project takes both forms of input and combines them together for a more accurate and reproducible input for games.

The accelerometer is used to determine the direction of large scale gestures, and is not accustomed to use with small gestures. It can also sense rotation as well as large movement. The EMG sensor, or electromagnetic sensor, senses the electrical currents that muscles make as they move within the body, thus giving the system the knowledge of when a user has moved. The problem with EMG is that it is hard to determine the exact movement that was made, but it can sense when even the tiniest of movements occurs. The accelerometer (ACC) and the EMG inputs are processed in real-time and converted to recognized gestures and inputs to control the game.

To test this new innovation, the researchers built a virtual Rubik's Cube game, and allowed those in the study to manipulate the cube in real time in 3D space using the gesture input system. After being outfitted with EMG sensors along their wrist, and an accelerometer on the top of their hand, they were able to manipulate the cube, and the system was able to recognize their gestures, 91.7% of the time. Future work would include making the system more robust, extending applications to mobile device control, and the design of smaller sensors for inclusion in a glove, for example.

Discussion

I thought this seemed pretty interesting because unlike newer game control input systems (like Project Natal, by Microsoft, which uses a camera to detect user movement, gestures, and motions, without a controller), they are getting the exact movement from the user. It seemed similar to the accelerometer that Nintendo uses in their Wii controllers, but it would have a higher sensitivity as it actually senses the electrical impulses to the muscles in the body. My question would be if this system was actually more useful than the Wii motion controller system, or was just a novel way to control input using the whole hand and fingers instead of using an external controller.

Parakeet: A Continuous Speech Recognition System for Mobile Touch-Screen Devices

Comments

Nic

Summary

In Parakeet: A Continuous Speech Recognition System for Mobile Touch-Screen Devices, the authors present a new voice-recognition application for mobile touch-screen devices to allow the user to input text into the device. Using the voice recognition, the user can read the text they want to enter, along with using a simple touch-based interface for making corrections and changing words. The features are as follows:

• Voice recognition software converts the spoken word into text.
• When the recognition is done, a beep lets the user know.
• The user can then scan the sentence and easily make changes to words. The words in the sentence are ordered at the top of the screen, and below in columns are examples of other possible words that sound like the word spoken, along with a delete button at the bottom. The user can either replace the word that was recognized, or delete the word altogether.
• If the software is completely unable to recognize the word, it will default to a keyboard entry with predictive text as seen in current mobile devices.

After doing a user study, they found that users were able to enter up to 18 words per minute while sitting still, and 13 words per minute while walking around. While it may seem small, it is easily comparable to the 10 - 20 words per minute that each user had while entering text using the traditional T9 text prediction system (using a keyboard).

Discussion

If implemented correctly and included on all new cell phones, this entry system would revolutionize the text messaging system. I know lots of people that are older than me that don't see the point of text messaging...but if they could just speak what they want to send, just like they're having a conversation over the phone, and send that as a text message, then they could be more comfortable with the system and send messages to users that use traditional typing methods. But, the words per minute would have to be a lot higher before I would use it because I already text almost as fast as I type on the computer. There would be a lot of idiots walking around talking into their phone like the people with bluetooth headsets though....

Emotional Design

Comments

David

Summary

In Norman's first book, The Design of Everyday Things, he stressed the necessity of usability in products of all forms. With a first glance at a device or product, the user should instantly know how the product is supposed to be used, without question; they should not have to read a lengthy manual to understand the inner workings of the product. In Emotional Design, Norman departs from this notion partially and instead focuses on the emotional appeal of objects. His main point (as quoted by William Morris) is as follows:

Have nothing in your houses that you do not know to be useful, or do not believe to be beautiful.

Norman's description of emotional design includes the following points:

• Users report that they are more happy with objects that are aesthetically pleasing, even if they do nothing or do not work as they are supposed to.
• Appeal to the three levels of emotional response:

Visceral: the basic appearance and visual / aural / haptic reponses to the object

Behavioral: the pleasure and effectiveness of use (does it work correctly?)

Reflective: how the object appeals to the user in a high-level, like self-image, personal

reflection, or past memories

• Make the object have its own unique personality.
• The object should be not only beautiful but 'fun' as well. Beauty, fun, and pleasure all work together to form a sense of enjoyment of the product or object.
• Products should go beyond the initial expectations, produce an instinctive reactionary response, and deliver surprising novelty.
• Communication between product and user should make the user feel important and respected.
• Machines or robots that are created should have emotional responses so as to react with humans appropriately.
• Allow the user to have a personal relationship with the product, either through customization, years of use, or personal touches.

In all, Norman wants us to take his past argument (that all products should be usable, even if they're ugly) and change the weights he had given before. He now believes products should be beautiful and emotionally pleasing as well as be usable and work correctly, for successful products have both of these qualities.

Discussion

I thought this book was a nice calm change from the style of The Design of Everyday Things. In that previous book, he seemed to be angry at everyone, continually telling everyone how their products are wrong because they are hard to understand or use, and that they focus too much on aesthetics instead of usability. What was nice about Emotional Design was that he instead focuses on instructing designers on great ways to make their objects and products more emotionally appealing, and gave examples of great emotional designs, instead of criticizing everyone and giving examples of poorly designed products. It was a much more satisfying read than the past.

I completely agreed with Norman on the necessity of designing with emotional and aesthetic appeal. Apple laptops and PCs are not successful because they are more powerful than any other PC (because they're not), but because they are beautiful to look at and to use. I have always wanted an Apple computer (though I can't afford one) even though I know that as a computer programmer, I would not have as many resources for developing applications and doing my daily work as I would on a PC. This shows that the emotional design of the computers can far surpass the functional requirements. I know that I definitely buy products based on aesthetic qualities as much as functional qualities.

Data Driven Exploration of Musical Chord Sequences

Comments

Chris

Summary

In Data Driven Exploration of Musical Chord Sequences, Basu, Nichols, and Morris present a new way of classifying different genres of music based on inherent chord progression styles, and an interface for both music novices and experts alike to modify styles of chord progressions to form their own unique genre of music, or blend two artists or styles together.

The polygon slider component of the interface consists of a polygon in which each point represents a style of music or artist (and all of them are user-defined). The current point in the polygon (inside the polygon) can be defined by the user as well, and can be slid around inside the widget in real-time. Using a distance algorithm, the system finds the current distance of the current point from each point of the polygon (each style), and assigns a percentage to each quality being compared. The system will then take properties of the chord progressions from each of the styles (based on the percentage given) and create a new chord progression style that the user has defined. In this way, users can create their own style of music (as chord progressions are inherent to the differences in musical styles) and put the progression behind a melody to create music. The possible choices for inputs include styles such as Country, Rock, Classical, or Pop, and artists such as AC/DC, the Beatles, and Lenny Kravitz. Chord progression styles for each genre of music are obtained by averaging chord values (and sound frequencies) from a large database of songs in that genre. Progressions for artists are averaged in a similar way from a database of songs by that artist.

Discussion

Being a musician, I thought it was really cool that genres of music could be exactly specified by types of chord progressions. I mean, everyone knows that some types of music have chord styles that are always seen, for example the Blues scale and chords in Jazz and Blues, and minor chords in Alternative Rock. But to think that a specific chord progression such as C->F->G constitutes a Country song is pretty interesting. Now I can tell all my family and friends that I hate Country music because I was born to hate their exact chord progressions! Not my fault, I was just born that way. And the interface to be able to create your own chords (even as a novice) is great. Many people wish they knew how to play instruments, and seeing how it is an extremely hard process, this interface would give many people a way to be creative without taking the time and effort to learn an instrument (or compose music by writing scores of notes).

TrailBlazer: Enabling Blind Users to Blaze Trails Through the Web

Comments

Patrick

Summary

TrailBlazer: Enabling Blind Users to Blaze Trails Through the Web, by Bigham, Lau, and Nichols, details a new way for blind users of computers to navigate the web more quickly. Current accessibility programs for the blind consist of only a linear listing of all the text and options on the screen, whether it is listed through voice or through a refreshable Braille display. Trailblazer allows the user to access scripts written for the website (a script consists of a possible path in the website, and websites with scripts written will include the most important and popular actions on the website) and easily jump to the options they need, significantly reducing the time it takes to navigate a website. Scripts that are already written for websites are listed in a large database called "CoScriptor", but Trailblazer also allows users to create their own scripts. These scripts are a kind of instruction for the system, detailing actions the user wants to make on the website in a pseudocode language. Each time a new user creates a script, that script is added to the CoScriptor database, so as users use TrailBlazer, the database continues to grow and more websites are accessible by the blind.

Additionally, the system saves past user's actions and uses them to predict the next most likely action (picks the script most likely to be used). In this way, the system continues to improve with each time the system is used. After doing an extensive user study with blind users, all of the users said that TrailBlazer significantly reduced the amount of time that it took to navigate a website, but were not as happy with the steep learning curve that they had to overcome to use the software. Another problem with TrailBlazer was its inability to navigate dynamic web content such as Flash. As most of that content is not textual, and is contained in its own application, then it cannot navigate buttons, etc. But in general, most of the users were very happy with the interface.

Discussion

I thought that if this technology made it into all of the current web browsers, and it was accessible to all blind users, then it would be super successful. I can't imagine how long and frustrating it is to do even the most simple task as checking your email when you're blind. So, having a system that records the actions you do most often and predicts (with 75% accuracy) the action that you want to do, then you could check email, etc. much faster. The main problem would be whenever you want to go to a new website, or navigate a flash interface, and the system works much more slowly (or not at all, in the case of the flash interface).

The Inmates are Running the Asylum (Chapters 1 - 7)

Comments

Kerry

John

Summary

The Inmates are Running the Asylum, by Alan Cooper, is a book that promotes "interaction design" for technological devices and software so that the user is not confused or made to feel stupid by the complexity of the technology. He makes a few important points (just Chapters 1 - 7):

1. Any device combined with a computer is still a computer; the complexity and the anti-human behavior of the computer will dominate.
2. Because of this, we need to promote the "partnering of interaction design with programming". Interaction design is different than interface design; it is not just the design of an interface that is used to communicate with the computer, but the way that the user and computer interact with each other.
3. Cognitive Friction is the resistance of humans to understand the complex systems of rules that are always changing that computers and technology follows. Interaction design aims to reduce the cognitive friction so that everyday users feel more comfortable.
4. A clear distinction is made between user and programmer. The programmer is more apt to design the system to make their job of programming easier and in a way that they feel is easy to interact with. What they fail to see is that users do not think the way that they do, and the programmers are incapable of putting themselves in the shoes of the user.
5. Interaction design and programming has to be done by different people; it is nearly impossible to both design for the irrational and emotion world of humans and simultaneously design for the deterministic and rational world of computers.

In these first seven chapters, Cooper presented the problem of software that is too hard to use, and started to explain the solution of interaction design. His main point was that interaction design is necessary before programming and will save lots of money and time in software that is accepted by users and is successful in the real world. Programmers need to give up control so that their ideas and efforts in products will not be wasted.

Discussion

I thought Cooper presented some interesting ideas in the first seven chapters of this book. I agree with his ideas that interaction design should be done before programming; without a clear vision of how the user will react to and interact with the software, its hard to keep low-level implementation details from filtering up into the interface and confusing the user. But, I don't feel that it is mostly the programmer's fault, and that all programmers have their heads stuck staring at their monitors so that they can't understand the interaction of everyday users. I think the problem sits with management, marketing, and economics more. The managers demand a certain functionality as soon as possible, and as the biggest time expense is the base programming, the engineers have to hit the ground running. If given time (and if more management realized that design is not a waste of time or budget but instead a way to make sure your product is received well by the masses), then programmers could easily help make design decisions and produce a program that users like. I just feel that the time constraints put functionality first instead of usability. Interaction designers are important though, even after the main design phase, because they can work concurrently with the programmers to make sure that the project is going along at the correct rate and it is still usable. Now that Cooper has laid down the problem, I am looking forward to the rest of the book to see what his solution is.

Autism Online: A Comparison of Word Usage in Bloggers With and Without Autism Spectrum Disorders

Comments

Subodh

Summary

In Autism Online: A Comparison of Word Usage in Bloggers With and Without Autism Spectrum Disorders, the researchers present a study in which they analyze the language in blogs written by those with Autism Spectrum Disorders and the difference between blogs written by people without mental disorders. People with Autism and other related disorders primarily have problems in social interaction, specifically with communication issues and in face-to-face interaction. The web is a place where those with disorders can feel comfortable and free of the pressures of real-time personal interaction, and spend time thinking how they want to communicate. It gives them a place to be themselves and interact without fear of being judged. The researchers wanted to see if there was a significant communication difference in text of the internet between those with Autism and those without disorders, similar to the physical communication difference in personal interaction. After analyzing 50 blogs from those who have Autism (along with other requirements such as age, etc.) and 50 from those that did not have the disorder, the researchers found that there was not a large difference between the blogs in any categories (social, melancholy, ranty, work, metaphysical). The only difference was that the variability between blogs in the "social" category was much higher, which can probably be explained by the lack of social interest that is common in those with Autism Spectrum Disorders. The conclusion was that because of the lack of difference in communication patterns, blogs and the web in general are an appropriate place for those with disorders to interact socially and feel comfortable, mostly because the requirement for fast mental processing of social cues is taken away.

Discussion

It seemed really great to me that those who suffer with social and communication disorders can have a place to interact with others without feeling pressure of failure. It definitely means that those have a place to vent their feelings of anger and frustration that usually come with autism, and can possibly be a way for those with the disorder to interact with the public and possibly live a normal life, having jobs at home that required internet communication, and working through email and blogging. Hopefully the interaction online that they have can directly help them in their physical social interactions as well.