Linking Vision and Language through Computational Modelling
| Project/Area Number |
19K12733
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 90030:Cognitive science-related
|
|---|
| Research Institution | Kobe City University of Foreign Studies |
|---|
Principal Investigator |
CHANG Franklin 神戸市外国語大学, 外国語学部, 教授 (60827343)
|
|---|
| Project Period (FY) |
2019-04-01 – 2024-03-31
|
| Project Status |
Granted (Fiscal Year 2022)
|
| Budget Amount *help |
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2023: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2022: ¥390,000 (Direct Cost: ¥300,000、Indirect Cost: ¥90,000)
Fiscal Year 2021: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2020: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2019: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
|
|---|
| Keywords | Vision / Language / Learning / Event understanding / Computational model / Deep Learning / Priming / Verbs / Syntax / Eyetracking / language / thematic roles / object tracking / connectionist model |
|---|
| Outline of Research at the Start |
The first project will be the development of a computational model which can explain behavioral data from both adults and children within multiple object tracking tasks.
The next step will be to extend this model to address motion understanding.
The next project will link this computational model of action understanding to language.
To test this computational model, we will do a series of eye-tracking studies which test various assumptions of the model.
|
| Outline of Annual Research Achievements |
I have written a paper on a deep learning model of how visual input is used to select verbs. The model is trained on data from human experiments on adults and the model was tested by comparing its verb use to children and adults. This paper has been accepted at Cognitive Science.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
I am continuing to do research on structural priming to test the assumptions about prediction error-based learning in the deep learning model. Recently, I have started to examine the lexical boost as well.
|
| Strategy for Future Research Activity |
Now that the previous paper is almost done, I will start some new deep learning modeling work.
|
Report
(4 results)
Research Products
(4 results)
