2015 Fiscal Year Research-status Report
Structural Recursion on Bulk Synchronous Parallelism for Efficient Large-Graph Querying
| Project/Area Number |
15K12011
|
|---|
| Research Institution | National Institute of Informatics |
|---|
Principal Investigator |
胡 振江 国立情報学研究所, アーキテクチャ科学研究系, 教授 (50292769)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
LI CHONG 国立情報学研究所, アーキテクチャ科学研究系, 特任研究員 (50745312) [Withdrawn]
|
|---|
| Project Period (FY) |
2015-04-01 – 2017-03-31
|
| Keywords | Parallel Programming / Graph Processing / Structural Recursion / Parallelization, / Pregel |
|---|
| Outline of Annual Research Achievements |
The objective of this research is to combine the expressive power of structural recursion with Pregel, a popular system based on Bulk Synchronous Parallelism for large scale graph processing.
This year, we focused on analysis of structural recursion to find a class of structure recursions that can bridge between high level graph queries and low level Pregel implementation, and gave a prototype implementation of the mapping from graph queries based on regular path expressions to Pregel. As a result, we design and implement a graph transformation framework on top of Pregel.
1. We propose a domain-specific language (DSL) to help users write their scalable graph transformations as structural recursive functions in a declarative way. It is a subset of the UnCAL language but it covers a wide class of structural recursion that can be efficiently evaluated in parallel. Moreover, fusion rules can be freely applied to the composition of structural recursive functions.
2. We improve the bulk semantics in UnCAL in order to efficiently evaluate structural recursive functions written in our DSL. This improvement minimizes the amount of data generated during the evaluation and utilizes the basic Pregel “skeletons” that have been proved to be efficient and scalable.
3. We implement a prototype framework for graphs in which transformations written in our DSL are automatically compiled into Pregel programs. Preliminary results for real-world graphs (citation network, youtube) show that our framework achieves good scalability and speedup compared to the original bulk semantics used in UnCAL.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The research project is going smoothly according to our original research plan. In the first year analyzed structural recursion to identify a class that is not only powerful enough to specify many interesting graph queries, but also easy to be efficiently mapped to Pregel for processing big graphs. In addition, we gave a
proof-of-concept implementation to validate our idea.
|
| Strategy for Future Research Activity |
In the second year, we will extend the prototype implementation to a more practical framework, which is to deal with larger class of parallelizable structural recursions, and to get more optimized Pregel codes. One interesting new idea is to abstract Pregel computation to a low level structural recursion for low level optimization, and find a systematic way to transform the parallelizable structural recursion to this low level one to gain more efficiency.
|
