| Project/Area Number |
23K17007
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 62020:Web informatics and service informatics-related
|
|---|
| Research Institution | Nara Institute of Science and Technology |
|---|
Principal Investigator |
SHE WanJou 奈良先端科学技術大学院大学, 先端科学技術研究科, 特任助教 (40967410)
|
|---|
| Project Period (FY) |
2023-04-01 – 2026-03-31
|
| Project Status |
Granted (Fiscal Year 2023)
|
| Budget Amount *help |
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2025: ¥1,950,000 (Direct Cost: ¥1,500,000、Indirect Cost: ¥450,000)
Fiscal Year 2024: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2023: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
|
|---|
| Keywords | Empathy / Large Language Model / Chatbot / Cancer Care / Medical Advice |
|---|
| Outline of Research at the Start |
This project aims to develop EmplifAI, a chatbot to offer medical advice in an emotionally sensitive manner. We fine-tune the chatbot with conversation data from cancer centers and crowdsourcing. Afterwards, EmplifAI would be deployed for patient evaluation in a pilot study at the OICI hospital.
|
| Outline of Annual Research Achievements |
My first-year research aims to develop an empathetic chatbot model for providing emotionally sensitive medical advice to cancer patients. I'm fortunate to utilize emerging Large Language Models (LLMs) like ChatGPT, llama-2, and mixtral, which have significantly enhanced my original study plan with groundbreaking generative AI technology.
In summary, I've collected 280 data points on emotionally challenged medical situations and 4230 data points of empathetic dialogues to address diverse patient emotions through crowdsourcing. I've also conducted interviews with eight cancer professionals to guide empathetic chatbot development. Additionally, I've published three chatbot papers in top HCI venues (CHI and IJHCI) and secured an additional chatbot grant from the JR West-Relief Foundation.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
I'm currently exploring the best methods to fine-tune a language model or construct our Retrieval Augmented Generation (RAG) framework. After receiving predominantly positive yet cautious feedback from our interviews with medical professionals, I've decided to prioritize implementing safety mechanisms in the development of our chatbot prototype. Additionally, I have relocated to Kyoto Institute of Technology and will be submitting the ethical application through the new institution. I aim to publish the two datasets as open-access resources for the Japanese Natural Language Processing research community. The interview data will be analyzed using a thematic analysis approach and submitted to the Journal of Medical Internet Research (JMIR).
|
| Strategy for Future Research Activity |
After successfully developing the language model candidates and integrating them into a chatbot user interface, my next step is to engage doctors in evaluating the safety of the chatbot's responses. The criteria for assessing the safety and performance of the model will include: Scientific Consensus, Potential Harm, Evidence of Miscomprehension, Evidence of Incorrect Retrieval, Evidence of Flawed Reasoning, Inappropriate or Incorrect Content, Helpfulness of Responses, Bedside Manner, and Empathy Level. I aim to enlist up to 5 doctors to evaluate the answers generated by multiple chatbot candidates. These criteria will guide me in selecting the safest and most effective model to advance to the final stage in the third year.
|
