Efficient Data Staging into The Big Data Analytics Platform
| Project/Area Number |
16K21675
|
|---|
| Research Category |
Grant-in-Aid for Young Scientists (B)
|
| Allocation Type | Multi-year Fund |
|---|
| Research Field |
Multimedia database
High performance computing
|
|---|
| Research Institution | National Institute of Advanced Industrial Science and Technology |
|---|
Principal Investigator |
Tanimura Yusuke 国立研究開発法人産業技術総合研究所, 情報・人間工学領域, 主任研究員 (80415710)
|
|---|
| Project Period (FY) |
2016-04-01 – 2018-03-31
|
| Project Status |
Completed (Fiscal Year 2017)
|
| Budget Amount *help |
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2017: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Fiscal Year 2016: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
|
|---|
| Keywords | データストレージ / ビッグデータ解析 / データステージング / 資源管理 / クラウド |
|---|
| Outline of Final Research Achievements |
Data staging between the big data analytics platform (the main computing system) and the backend storage, pre- and post- processing in the backend to achieve efficient data staging, and appropriate scheduling to avoid performance interference by concurrent data staging was studied. The prototype system by using Spark and Alluxio was designed and implemented, and then I/O (staging) performance including storage-side processing was evaluated. The basic results would enable multi-tenant operation of the big data analytics platform and more effective use of the backend storage.
|
Report
(3 results)
Research Products
(3 results)
