| Project/Area Number |
22K12160
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61030:Intelligent informatics-related
|
|---|
| Research Institution | Hokkaido University |
|---|
Principal Investigator |
RZEPKA Rafal 北海道大学, 情報科学研究院, 助教 (80396316)
|
|---|
| Project Period (FY) |
2022-04-01 – 2025-03-31
|
| Project Status |
Granted (Fiscal Year 2023)
|
| Budget Amount *help |
¥3,250,000 (Direct Cost: ¥2,500,000、Indirect Cost: ¥750,000)
Fiscal Year 2024: ¥520,000 (Direct Cost: ¥400,000、Indirect Cost: ¥120,000)
Fiscal Year 2023: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2022: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
|
|---|
| Keywords | semantic primes / tacit knowledge / cognition simulation / explainability / large language models / transparency / language acquisition / knowledge acquisition / Semantic Primes / Dataset construction / Tacit knowledge / Perception / 暗黙知 / 知識獲得 / 常識的知識 / コモンセンス |
|---|
| Outline of Research at the Start |
自然言語処理の分野は爆発的な人気を博しているが，現在最も多い知識を含んでいると言われているGPT-3言語モデルも「馬の目は馬の足にある」のような「勘違い」をしていて，子供でもわかる「当たり前」に弱い．本研究は，まずどのように新たな「目に見えない知識」を収集すべきか調査を行う．次ににコモンセンス心理学で用いられている意味の原子要素の概念からヒントを得た「意味の分解」を用いることでどれぐらいの新たな「暗黙知」が収集できるか調査をする．そして，その知識の内容と構造化の変更が言語理解タスクにどのようか影響をもたらすか，という問いへの答えを見つけるための研究を行う．
|
| Outline of Annual Research Achievements |
Knowledge I gained from the struggles of the first year has paid and six publications have been contributed. Idea of basic signals used 20 years ago for recognizing good from wrong acts seemed not be useful in the era of large language models, but combining the human-made signals with language generation showed that they can be mutually beneficial. These findings I described in a book chapter. On the other hand, semantic primes dataset which has been developed last year did not help in solving Winograd Schema Challange for Japanese (the cognitive pseudo-signals made by crowdworkers have been used for fine-tuning). These results confirm my beliefs that wider tests of common-sense capabilities of large language models are required, also in other modalities than text as most of semantic primes are related to the physical world. Simple experiment were run to observe if large language models choose the same words describing five senses as humans when faced with textual input about the real-world objects. The conclusion is that they cannot in most cases, but it is also difficult for human annotators. Another that has been discovered is that if we want to build a robot with a large language model as a source of cognitive pseudo-signals, we need to work on methods causing noise - for example a famous name of an actor or patient can influence action choices. I have described the grant idea in Japanese in a JSAI magazine and finished a story dataset necessary for future experiments with changes of semantic primes in sequential context.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
After a year of difficulties with pre-crowdworking trials and mechanizing texts for manual data creations, the data developed during the first year allowed to conduct new experiments and publish their results. However, I decided to leave the paper describing the main dataset to the last year as without excessive experiments, the contribution will not be appreciated. Therefore, although the output was six time bigger than the first year, I am not fully satisfied with the progress. The biggest step forward was finishing and publishing additional story dataset named DanSto. Based on previous Kakenhi grant achievements, it can be expanded with semantic primes and become a base for novel research not only on cognitive perception changes and its role in learning, language and knowledge acquisition, similarity calculation, analogy learning, categorization or reasoning. From the experiments I have conducted I can see possible contributions for Japanese researchers studying tacit knowledge, time (the data is divided into sentences written in different tenses), and overall story understanding as it is the first data like this made manually in Japanese language.
|
| Strategy for Future Research Activity |
Rapid development of large language models showed a new role for the data that is being developed using this grant. We have started experiments testing several abilities of the large language models, including commercial ones as GPT-4, for example understanding of negation or dangerous items in pictures of kid rooms. Semantic primes can become an original approach to probing latest AIs which lack rich sensory input and for that reason widely differ from human beings. During the last year I will focus on the explainability and cost-saving learning paradigm that could be a remedy for today’s transformer-based large language models which inner workings are hard to analyze. I will experiment with algorithms not using deep learning to see how much transparency can be achieved. The hypothesis is that, similarly to human-beings, even if the cognition is wrong or missing, gathered modalities (experiences) may repair perception outliers (misinterpreted sensory signals). Even if the proposed approach is not computationally feasible today, it could provide a base for future, physical sensory-equipped artificial agents. Another idea to be tested is to change the paradigm of a single pre-training of all available data which mixes human knowledge as it was not a description of particular experiences of individuals. Multi-agency with memory sharing and updating through interaction is the goal of Bacteria Lingualis agents which will be upgraded with semantic primes.
|
