Is "2019" Earlier than "2022"? Unsupervised Temporal Representation Learning via Contrastive Learning.

• Project/Area Number: 23K16946
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 61030:Intelligent informatics-related
• Research Institution: Kyoto University
• Principal Investigator: 程 飛 京都大学, 情報学研究科, 特定助教 (70801570)
• Project Period (FY): 2023-04-01 – 2025-03-31
• Project Status: Granted (Fiscal Year 2023)
• Budget Amount: ¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000); Fiscal Year 2024: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000); Fiscal Year 2023: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
• Keywords: 自然言語処理 / 大規模言語モデル / 常識推論 / 時間的常識推論

Outline of Research at the Start

Our method leverages Contrastive Learning to explicitly encourage two time expressions with close numeric distance to have similar embeddings while pulling away the embedding space of two examples with far distance. We then empirically evaluate our models on several temporal-aware downstream tasks (e.g., TempEval-3: temporal information extraction, Clinical TempEval: timeline generation) to verify the improvement of possessing temporal information.