Statistical inference in exploratory data analysis and its application

2020 Fiscal Year Final Research Report

• Project/Area Number: 18K18010
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 60030:Statistical science-related
• Research Institution: Nagasaki University (2020); Nagoya Institute of Technology (2018-2019)
• Principal Investigator: UMEZU Yuta 長崎大学, 情報データ科学部, 准教授 (60793049)
• Project Period (FY): 2018-04-01 – 2021-03-31
• Keywords: モデル選択 / selective inference / 高次元漸近理論 / 教師なし学習 / 教師あり学習

Outline of Final Research Achievements

In recent data science, we often observe data without determining hypothesis to be tested. Particularly, severe selection bias could be occur when the same dataset is used both for generating the hypothesis to be tested and for testing it. Here, in order to correct the selection bias, we focus on the selective inference framework, and tried to improve the existing method. Our main results are the application of the idea of selective inference to unsupervised learning and the development of the method that can be applied to more general class of statistical model by relaxing the normality of the data.

Free Research Field

数理統計学

Academic Significance and Societal Importance of the Research Achievements

近年のデータ科学では，検証すべき仮説が定まらないままデータが取得されることが多い．その際，検証すべき仮説の生成と，その仮説の検証を同じデータを用いて行う場合，選択バイアスの問題が生じてしまう．とはいうものの，データの分割や同じ環境での再実験が困難な場合に統計的なエビデンスを提供するためには，同じデータを用いて仮説の生成と検証を行うことが求められる．本研究では，selective inferenceのアイデアに基づき，いろいろな問題に対してこのような統計解析が可能であることを示した．