Development of Data Management Framework Integrating Stream Processing and Analytical Data Processing
| Project/Area Number |
24700111
|
|---|
| Research Category |
Grant-in-Aid for Young Scientists (B)
|
| Allocation Type | Multi-year Fund |
|---|
| Research Field |
Media informatics/Database
|
|---|
| Research Institution | National Institute of Advanced Industrial Science and Technology |
|---|
Principal Investigator |
YUI Makoto 独立行政法人産業技術総合研究所, 情報技術研究部門, 主任研究員 (10586712)
|
|---|
| Project Period (FY) |
2012-04-01 – 2015-03-31
|
| Project Status |
Completed (Fiscal Year 2014)
|
| Budget Amount *help |
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2014: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2013: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2012: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
|
|---|
| Keywords | 機械学習 / ビッグデータ / データベース / 関係データベース / オンライン学習 / 確率的勾配降下法 / MapReduce / 並列処理 |
|---|
| Outline of Final Research Achievements |
We proposed a database-Hadoop hybrid approach to scalable machine learning where batch-learning is performed on the Hadoop platform, while incremental-learning is performed on PostgreSQL.
We conducted a series of exterimental evaluation using a commercial advertisement dataset provided in the KDD Cup 2012, Track 2. The experimental results show that our scheme has a superior training speed compared with state-of-the-art scalable machine learning frameworks, 5 and 7.65 times faster than Vowpal Wabbit and Bismarck, respectively, for a regression task.
|
Report
(4 results)
Research Products
(11 results)
