Mirror that Talks; A Self-Motivating Personal Vision Assistant (PDF)

Mirror that Talks: A Self-Motivating Personal Vision Assistant Akeem Olowolayemo Faculty of Cognitive Science & Human Development, University Malaysia Sarawak, Malaysia. [email protected] Saleh Alenazi Faculty of Information & Computer Science, University Kebangsaan Malaysia. [email protected] Faten Anis Syairah Seri Faculty of Cognitive Science & Human Development, University Malaysia Sarawak, Malaysia. [email protected] ABSTRACT This work presents a self-motivating personal vision assistant that serves as a self-reflecting and motivating system for people such as those with low self-esteem, lonely or depressed individuals, single mothers, divorced or widowed, who constantly needs to be motivated to remain positive and focused. The system is similar to the rest of the personal digital assistants in the market but with the added functionality of reading the mood of the user and motivating them based on their present states of mind such as happy, sad, bored etc. The system uses facial recognition to determine the users' mood and a database of user's history or past events to adequately motivate the users. Further extensions include creating and adding imagery of beloved ones that have impacts on the lives of the users, such as missed loved ones like mothers, spouse, etc as added stimulation. CCS Concepts • Computing methodologies➝Image processing. Keywords Digital Mirror; Computer Vision; Personal Assistant; Emotion/Mood Aware System; Context Aware system; Motivation/Recommender System. 1. INTRODUCTION There are has been an increase in the number of lonely people in the world. This ranges from widows or widowers, aged, divorced, disabled or even people suffering from low self-esteem or depression to even various individual leaving alone, far away from their families[1], [2]. One of the tools available in every home, used daily for variety of purposes such as when brushing of teeth, shaving, applying cosmetics to the face and other parts of the body, is the mirror. Every day, most people see themselves or rather their reflections in mirror at least a few times. Hence, due to this pervasive use by everyone, the question would be what if the mirror can actually interact with them like a real person or like a digital assistant. The motivation here is that combining the idea of a mirror with technology may actually serve some tremendous purpose of substituting the emotional needs from missed relations or spouses with this kind of personal assistants. Therefore, the intended design is to propose a device that goes beyond an ordinary mirror, where it is possible for people to actually converse with their own reflections in mirrors and receive recommendation or feedbacks based on their present states of mind, the current context as well as specific personal history. With today’s technology, this device can be developed by using facial recognition technology. Facial recognition is a technique that can be used to identify individuals directly from captured images obtained from cameras or from video frames [3], [4]. Most work in facial recognition focuses on individual’s features such as eyes, lips, nose and the shape of their faces[5]. In order to develop the mirror, facial recognition is used to detect emotion in an individual through a mirror as a medium. Besides, for the system to detect emotion, the system has to analyse the facial features of the individual whether the person is smiling, frowning, sad, worried or stressed etc. Current facial recognition systems work with numeric codes to detect a person’s facial features, where the system identify 80 nodal points on a person’s face and measure variables that it can obtain through the identification process [3], [4]. The research purpose is to develop an Eliza-based mirror that can interact with users by detecting their emotions and combining that with their personalized life history. Eliza is a natural language processing computer program that interacts with humans through text-based communications [6]. Eliza was developed to imitate a psychiatrist by interacting with humans in order to ease their psychological problems as well as providing solutions when a human psychiatrist is not available. By embedding a similar system like Eliza program in a mirror, an individual may converse with the mirror with the feeling that they are actually talking to someone. Facial recognition is used in the Eliza-based mirror to detect their emotion and also for security purposes. Whenever an individual uses this device, the mirror will identify their faces first, analyse their emotions and load relevant context from their past history that fit the present context to adequately motivate the user. After the analysis has been done, the mirror will interact with the user regarding what they feel in real time. 2. RELATED WORK Five previous projects were identified to have some functionalities that are related in some ways to what is intended for building a device with one-way mirrors. These five projects include Magic Mirror by Micheal Teuw, Home Mirror by Hannah Mittelstaedt, Smart Mirror by Evan Cohen, Smart Mirror by Max Braun and PANL by Ryan Nelwan. These previous applications will be discussed one after in the next subsections. 2.1 MagicMirror According to [7], Magic Mirror was first proposed by Michael Teuw. The proposed Magic Mirror was in the form of a mirror Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. ICIGP 2018, February 24–26, 2018, Hong Kong, Hong Kong © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-6367-9/18/02…$15.00 https://doi.org/10.1145/3191442.3191476 157