Fast Information Processing of Large-scale Data Based on a Combination of Compressed Indices and String Compression

• Project/Area Number: 18K18102
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 61030:Intelligent informatics-related
• Research Institution: The University of Tokyo
• Principal Investigator: Denzumi Shuhei 東京大学, 大学院情報理工学系研究科, 助教 (90755729)
• Project Period (FY): 2018-04-01 – 2022-03-31
• Project Status: Completed (Fiscal Year 2021)
• Budget Amount: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000); Fiscal Year 2021: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000); Fiscal Year 2020: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000); Fiscal Year 2019: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000); Fiscal Year 2018: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
• Keywords: データ構造 / 圧縮 / 索引 / 文字列 / 集合族 / 二分決定グラフ / 項分岐決定グラフ / 簡潔データ構造 / 決定グラフ / ゼロサプレス型二分決定グラフ / Top tree / anticover / 系列二分決定グラフ / 文字列集合演算 / オンライン圧縮 / 決定図 / ゼロサプレス型二分決定図 / 項分岐決定図 / 文字列集合

Outline of Final Research Achievements

In this study, I developed a computational technique for compressed representations that dramatically reduces computation time and space by compressing large-scale data in advance. We proposed compression methods that apply not only to sequence binary decision diagrams but also to various types of decision diagrams and evaluated their performance. These results provide a data structure and algorithm that can further compress a sequence binary decision diagrams representing large-scale sets of strings into a compact size and process them at high speed.

Academic Significance and Societal Importance of the Research Achievements

人々の生活のあらゆるところに電子機器が浸透し，それらがネットワークにつながることで日々膨大な量のデータが生み出され続けている．そういったデータを解析処理しようとしても爆発的なデータの生成速度に対し通常のアルゴリズムでは処理が追いつかないという問題が広く顕在化している．そのため，文字列集合のみならず集合族なども圧縮して表現することで効率良く扱えるようにする本研究の成果は計算機科学を利用する広範な分野において共通して重要な基盤技術でありその社会的意義も大きい．また，従来の決定グラフの性能をさらに向上させ，より一層の省領域化や多機能化，理論解析を進展させたことは学術的な側面からも意義深い成果である．

Research Products

• [Int'l Joint Research] Universita di Pisa(イタリア)
• [Int'l Joint Research] King’s College London(英国)
• [Int'l Joint Research] King Saud University(英国)
• [Journal Article] Storing Set Families More Compactly with Top ZDDs (2021)
  • Author(s): Matsuda Kotaro、Denzumi Shuhei、Sadakane Kunihiko
  • Journal Title: Algorithms Volume: 14(6), 172 Issue: 6 Pages: 1-23
  • DOI: 10.3390/a14060172
  • Peer Reviewed / Open Access
• [Journal Article] "DenseZDD: A Compact and Fast Index for Families of Sets" (2018)
  • Author(s): Shuhei Denzumi, Jun Kawahara, Koji Tsuda, Hiroki Arimura, Shin-ichi Minato, and Kunihiko Sadakane
  • Journal Title: Algorithms Volume: Vol.11, No.8 Issue: 8 Pages: 1-23
  • DOI: 10.3390/a11080128
  • NAID: 120006659589
  • Peer Reviewed / Open Access
• [Presentation] ブロックDAGに対する最大k-独立集合問題の二分決定グラフを用いた解法 (2022)
  • Author(s): 伝住 周平、川原 純
  • Organizer: アルゴリズム研究発表会
• [Presentation] Finding the Anticover of a String (2020)
  • Author(s): Mai Alzamel, Alessio Conte, Shuhei Denzumi, Roberto Grossi, Costas S. Iliopoulos, Kazuhiro Kurita and Kunihiro Wasa
  • Organizer: The 31th Annual Symposium on Combinatorial Pattern Matching (CPM 2020), Leibniz International Proceedings in Informatics, Vol. 161, No. 2, pp. 1-11, Copenhagen, Denmark, June 17-19, 2020
  • Int'l Joint Research
• [Presentation] Variable Shift SDD: A More Succinct Sentential Decision Diagram (2020)
  • Author(s): Kengo Nakamura, Shuhei Denzumi and Masaaki Nishino
  • Organizer: The 18th Symposium on Experimental Algorithms (SEA 2020), Leibniz International Proceedings in Informatics, Vol. 160, No. 22, pp. 1-13, Catania, Italy (held online), June 16-18, 2020
  • Int'l Joint Research
• [Presentation] Storing Set Families More Compactly with Top ZDDs (2020)
  • Author(s): Kotaro Matsuda, Shuhei Denzumi and Kunihiko Sadakane
  • Organizer: The 18th Symposium on Experimental Algorithms (SEA 2020), Leibniz International Proceedings in Informatics, Vol. 160, No. 6, pp. 1-13, Catania, Italy (held online), June 16-18, 2020
  • Int'l Joint Research
• [Presentation] New Algorithms for Manipulating Sequence BDDs (2019)
  • Author(s): Shuhei Denzumi
  • Organizer: 24th International Conference on Implementation and Application of Automata (CIAA 2019)
  • Int'l Joint Research
• [Presentation] Approximated ZDD Construction Considering Inclusion Relations of Models (2019)
  • Author(s): Kotaro Matsuda, Shuhei Denzumi, Kengo Nakamura, Masaaki Nishino and Norihito Yasuda
  • Organizer: Special Event on Analysis of Experimental Algorithms (SEA^2 2019)
  • Int'l Joint Research
• [Presentation] Sequence Sentential Decision Diagrams (2018)
  • Author(s): Shuhei Denzumi
  • Organizer: The 12th Annual International Conference on Combinatorial Optimization and Applications (COCOA'18)
  • Int'l Joint Research
• [Presentation] 非同型な2端子直並列グラフの列挙とランダムサンプリング (2018)
  • Author(s): 伝住周平, 堀山貴史, 栗田和宏, 中畑 裕, 鈴木浩史, ○和佐州洋, 山崎一明
  • Organizer: コンピュテーション研究会 （2018年9月18日開催，会場：九州工業大学（飯塚キャンパス））