2016 Fiscal Year Research-status Report
Structural Recursion on Bulk Synchronous Parallelism for Efficient Large-Graph Querying
| Project/Area Number |
15K12011
|
|---|
| Research Institution | National Institute of Informatics |
|---|
Principal Investigator |
胡 振江 国立情報学研究所, アーキテクチャ科学研究系, 教授 (50292769)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
LI CHONG 国立情報学研究所, アーキテクチャ科学研究系, 特任研究員 (50745312) [Withdrawn]
|
|---|
| Project Period (FY) |
2015-04-01 – 2018-03-31
|
| Keywords | Parallel Programming / Graph Processing / Structural Recursion / Parallelization / Pregel |
|---|
| Outline of Annual Research Achievements |
The objective of this research is to combine the expressive power of structural recursion with Pregel, a popular system based on Bulk Synchronous Parallelism for large scale graph processing.
In this second paper, we have extended our solution to a framework that takes high-level graph queries UnQL as input in order to relax the complexity of designing structural recursive functions. The gap between large graph processing platform and high-level declarative querying language is thus filled by our solution. A high-level graph query written by an end-user is transformed systematically into our internal algebra with a set of structural recursive functions in UnCAL, then a Pregel program will be generated by using our parametrized Pregel algorithms to guarantee the efficiency of the querying evaluation.
More specifically, (1) we identified monadic queries, a useful subclass of UnQL queries that can be translated into parallel-efficient structural recursive functions; (2) we proposed an approach, using pattern trees to describe the relationship between graph variables of queries, to translating all monadic queries into structural recursive functions in a systematic way; and (3) we used real big datasets to validate our graph querying framework. Both correctness and scalability were experimented, and the experimental results show that our solution may outperform an existing industrial solution for complex queries.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
The research project was going smoothly according to our original research plan, except that one of the main researchers moved out from Japan to France early last year, slowing our project a bit (for about half a year). We have done a lot on high-level optimization from UnQL to UnCAL, but could not do much on low-level optimization from UnAL to Pregel.
|
| Strategy for Future Research Activity |
We will focus on low-level optimization of mapping structural recursive functions to more efficient vertex-centric computation. The current Pregel is not really a good model for communication optimization because it has only a single global message type, making it hard to produce optimized low-level codes. We plan to extend Pregel with many-type messages and investigate efficient translate algorithm to reduce message communication cost in vertex-centric computation.
|
| Causes of Carryover |
研究分担者のChong Li氏が退職し、現在は博士学生のYongzhe Zhang氏に協力者として、システムの実現と評価を担当してもらっているが、担当者が交代したことで、当初の研究計画を見直す必要性が出てきた。それにより、国際学会での成果の発表を昨年度中途半端な形でするよりも、今年度完成した成果を発表した方がいいと考え、補助事業期間の延長申請をしたため、未使用額が生じた。
|
| Expenditure Plan for Carryover Budget |
主に研究成果を発表するための旅費として使う予定である。
|
Research Products
(3 results)
