2020 Fiscal Year Annual Research Report
a long-term children's mental health assessment system using sensor-embedded block-shaped tangible user interfaces
| Project/Area Number |
20J14480
|
|---|
| Research Institution | Tohoku University |
|---|
Principal Investigator |
WANG XIYUE 東北大学, 情報科学研究科, 特別研究員(DC2)
|
|---|
| Project Period (FY) |
2020-04-24 – 2022-03-31
|
| Keywords | Toy blocks, / Children, / Mental health, / Tangible user interface, / Play data, / Well being, / Data analysis, / Machine Learning |
|---|
| Outline of Annual Research Achievements |
This research aims to develop block-shaped Tangible User Interfaces (TUIs) that robustly predict children’s mental health. This year, the research achievement can be divided into two parts.
First, using the fundamental statistical methods, we discovered the relationship between block play and children’s stress. The work was published as a journal paper at ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (ACM IMWUT) and gave an oral presentation at UbiComp 2020, which is one of the top conferences in the field of Human-Computer Interaction (HCI), as “AssessBlocks: Exploring Toy Block Play Features for Assessing Stress in Young Children after Natural Disasters.” This work discovered the potentials of our novel approach, received positive feedback at UbiComp, and built a strong foundation for the next steps.
Next, we started to explore the Machine Learning methods to predict a child’s behavior problems from block play data. The work was accepted to ACM CHI 2021 as a conference paper (“Can Playing with Toy Blocks Reflect Behavior Problems in Children?”). CHI is the premier conference in the field of HCI. This paper proposed a toy block approach for predicting a range of behavior problems in young children (4-6 years old) measured by the Child Behavior Checklist (CBCL). We defined and classified a set of quantitative play actions from IMU-embedded toy blocks. Engineering play features, we built Machine Learning models, and demonstrated specific play actions and patterns predicted total problems, internalizing problems, and aggressive behavior in children.
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
Last year, my research was around data processing, and building Machine Learning models to predict a child’s behavior problems, with data extracted from toy block play.
The original plan was: (1) develop TUI; and (2) conduct experiments; and (3) develop Machine Learning models. Due to COVID-19, I was unable to continue children’s data collection around local kindergartens as planned. Furthermore, I could not collaborate with researchers in the U.K. (we checked with each other when we first met at the conference of UIST 2019 in New Orleans, US) to experiment with autistic children as we planned.
Thus, I focused on processing the data in-depth, which my team and I collected in 2016-2017 and 2019-2020. Deeply analyzing the data, the research achieved results that more than I expected. The data processing and analysis from different angles concluded to two achievements: one journal paper and one conference paper, which were accepted in two top conferences in the field of Human-Computer Interaction - Ubicomp 2020 and CHI 2021. They made strong contributions to the field, and build concrete steps towards the final goal. These two research papers also greatly enhanced my skills in many aspects, including data science, machine learning, product design, and paper writing. The current research progress helped me to make clear and higher goals for next year.
|
| Strategy for Future Research Activity |
Last year, we had (1) developed block-shaped TUI for collecting play data; (2) explored data processing techniques to extract meaningful features, such as actions and patterns, from raw data; (3) analyzed the relationship between block play actions and children’s stress; and (4) explored the prediction of a child’s behavior problems from engineered play features using Machine Learning methods. The achievements presented new questions, include robustness and interpretability.
To achieve the goal of predicting children’s mental health, we need to improve the accuracy and interpretability of the system. So far, the data extracted from toy blocks are play actions, which was one aspect of play behavior. Another aspect is the building structure. The specific plan is to extract the building structure, and explore other ML methods to model children’s behavior with high accuracy and interpretability.
Based on the research plan and the current result, this year we will: (1) extract block building structure features from video and IMU data we collected; (2) explore Machine Learning methods that can improve prediction accuracy and interpretability; and (3) revise TUI development and provide design criteria for research and development in similar fields. If the pandemic is over, we will collaborate with researchers from the U.K. to experiment with children with Autism.
|
