Extended MapReduce framework with distributed masters for high-reliablity and high-performance
| Project/Area Number |
23700036
|
|---|
| Research Category |
Grant-in-Aid for Young Scientists (B)
|
| Allocation Type | Multi-year Fund |
|---|
| Research Field |
Software
|
|---|
| Research Institution | Osaka University |
|---|
Principal Investigator |
OKITA Masao 大阪大学, 情報科学研究科, 助教 (50563988)
|
|---|
| Project Period (FY) |
2011 – 2013
|
| Project Status |
Completed (Fiscal Year 2013)
|
| Budget Amount *help |
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2013: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2012: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2011: ¥2,990,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥690,000)
|
|---|
| Keywords | 並列処理・分散処理 / MapReduce / 並列分散処理 / 高性能計算 / PageRank / 耐故障性 |
|---|
| Research Abstract |
To improve MapReduce, a popular large-scale distributed programming framework, we developed a high-reliable framework and techniques for accelerating a MapReduce application.
Firstly, we extend an existing MapReduce framework to improving fault-tolerance. In the existing framework, the master computer is the single point of failure. Our extended framework provides multiple master computers and switches them to avoid a system failure.
Secondly, we developed optimizing techniques for intermediate data processing to improve application performance. One of our technique reduces the amount of intermediate data by compressing data from an alternative viewpoint. Another technique reduces intermediate data transfer by improving data locality in task assignment.
|
Report
(4 results)
Research Products
(27 results)
-
-
-
-
-
-
-
-
-
-
[Presentation] Multilevel modeling of Physiological Systems and Simulation Platform : PhysioDesigner, Flint and Flint K3 service2012
Author(s)
Yoshiyuki Asai, Takeshi Abe, Masao Okita, Tomohiro Okuyama, Nobukazu Yoshioka, Shigetoshi Yokoyama, Masaru Nagaku, Kenichi Hagihara, and Hiroaki Kitano
Organizer
Proceedings of the 12th IEEE/IPSJ International Symposium on Applications and the Internet (SAINT 2012)
Place of Presentation
Izmir, Turkey
Year and Date
2012-07-17
Related Report
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
