Sunny Park

#User Interface #Interaction Design #Smart Glass #Leap Motion #Application Development

Smart Glasses & Leap Motion
: Smart TV Interaction

Apr 2013 - May 2014 | UX Researcher, Designer, LG Electronics R&D Center

This capstone project between Sogang University - LG Electronics was focused on developing novel concept ideas to create synergy between rookie devices and LG Electronics products. We developed working prototype and submitted eight user scenarios.

HCI Project

Skills.

Ideation, Personas & Scenarios, UI/UX design

Design Question.

What is the next generation beyond touch screen?

Our team’s goal was to investigate the possibility of Leap Motion and smart glasses as new input devices on LG Smart TV. We developed several concept user scenarios applied on various circumstances.

Research.

Before getting down to design, we carefully looked into the devices: Leap Motion, Google Glasses, and LG Smart TV. For several weeks, we individually collected the instruction the companies provided, the relevant patents, and the credible reviews on media and shared with other members. Simultaneously, we tried using the devices ourselves, in particular Smart TV at the LG R&D Center in Gangnam, South Korea. (At that moment, Google Glass were not released, so we relied on reviews)

Smart Glasses.

User Sceinario

In the first half of the term, we focused on smart glasses interaction. I participated in coming up with ideas and further development with the team. Since Google smart glasses were not on the market at that moment, we designed concept user scenarios only based on paper research. Even though our main task was to design user scenarios for LG smart TV, we further contrived several possible user scenarios on different circumstances in daily life.

TV Display Expansion with Smart Glasses

In the first half of the term, we focused on smart glasses interaction. I participated in coming up with ideas and further development with the team. Since Google smart glasses were not on the market at that moment, we designed concept user scenarios only based on paper research. Even though our main task was to design user scenarios for LG smart TV, we further contrived several possible user scenarios on different circumstances in daily life.
It would be most important to create a reliable augmented reality by quick, accurate calculating the relative distance and direction between the smart glasses and the display.
We developed scenarios based on consideration of 1) ‘expanding’ that utilize expansion of the applications on the screen, 2) ’laying out’ that focused layout of the UI that is difficult to display on the screen.

Information Enhancement on TV

The information on the TV is often instantaneous, limited. Our design goal was to appropriately enhance the limited information on the Smart TV with additional information customized to the user on smart glasses so that encouraging the user to interact more with the product. The necessary technology for this service was to recognise the user’s gazing location on TV with recognition methods such as visual coding or ultrasound technology and to show the proper information on the user’s sight that is relatively matched with the TV location.

Other Applications

On Art Museum: Docent
Smart glasses would provide non-intrusive viewing environment in which the focus should not be dispersed with additional sound aid.


On Gaming: Enhancement on Graphic
It would greatly enhance the overall immersion of gamers based on 3D mapping and projection


On Refrigerator: Ingredient Checker
LG has already provided applications that link smartphones and refrigerators. It can provide a more sensual experience by utilizing AR and notification function features of the smart glasses.


Leap Motion.

User Sceinario

For the interaction design with Leap Motion and LG Electronics products, our team developed user scenario focused on practical use on present product control.

Leap Motion and Smart TV

Since there are a number of basic TV controls that should be considered (such as next/prev, select, play, pause, volume/channel select), we focalized on finding the most appropriate control movements. I continuously iterated examination with trying assumed movement with the device with team members.

Leap Motion and Laptop

We set several situations that Leap Motion can be applicable input device on Laptop: As a control for graphic design applications, for web surfing, and for educational applications.

Smart TV Control Prototype

I participated in making concept video with other Art & Technology students. Moreover, we designed interactive prototype on LG smart TV with independent application. With other team members’ feedback, CS students developed the working prototype on TV with OpenGL. We could examine the prototype on the public in Hanieum Expo, Nov 2013. Students and visitors tried the prototype and gave positive responses for the noble interaction of Leap Motion.

Outcome.

Our team’s goal was to investigate the possibility of Leap Motion and smart glasses as new input devices on LG Smart TV. We developed several concept user scenarios applied on various circumstances.

Achievements.

2014.04

Obtained a patent on the user interface. (Method and apparatus of providing user interface having three-dimensional structure / KR. 10-2014-0043505)

2013.11

Participated 2013 Hanium Expo, by the Ministry of Science, ICT and Future Planning.

2013.11

2013 Smart Application Contest, Sogang Univ. , Top2