BW変換技術の深化による大規模データ処理基盤技術の開発

• Project/Area Number: 19K20213
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 60010:Theory of informatics-related
• Research Institution: Kyushu Institute of Technology
• Principal Investigator: 井 智弘 九州工業大学, 大学院情報工学研究院, 准教授 (20773360)
• Project Period (FY): 2019-04-01 – 2024-03-31
• Project Status: Completed (Fiscal Year 2023)
• Budget Amount: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000); Fiscal Year 2021: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000); Fiscal Year 2020: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000); Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
• Keywords: BW変換 / 圧縮索引 / 一般化文字列照合 / 圧縮変換 / 圧縮文字列処理 / データ圧縮 / 圧縮情報処理 / 文法圧縮 / Burrows-Wheeler変換

Outline of Research at the Start

Burrows-Wheeler変換（BW変換）は1994年に圧縮のためのデータ変換手法として提案された．BW変換は，後にデータ処理において様々な利点を有していることが判明し，近年も多くの重要な発見がされている．本研究では，BW変換をデータ処理に適した表現への変換技術と位置付け，その根底にあるアイデアを徹底的に追求することで，大規模データ解析の基盤技術を開発する．

Outline of Annual Research Achievements

BW変換は文字列中の各文字をその後ろに続く文字列（文脈）によってソートして得られる可逆変換である．それ自体では，文字を並べ替えただけなので圧縮になっていないが，文脈が似通っている文字を隣接する位置に配置することで同じ文字が連続しやすくなるため，連長圧縮などの単純な手法で圧縮することが可能となる．特に，高度反復文字列に対して，BW変換文字列の連長圧縮サイズは非常に小さくなり，データによっては数百分の一のサイズまで大幅に圧縮可能であることが知られている．BW変換は単に圧縮の前処理に止まらず，様々な文字列処理に応用されている．例えば，FerraginaとManziniはBW変換文字列を索引構造のように使えることを示し，文字列を簡潔な領域で保持しながら効率的なパターン照合機能を提供する簡潔データ構造FM-indexを提案した．FM-indexはテキスト中でパタン文字列と完全に一致する部分文字列の位置を検索できるが，完全一致の照合問題を一般化した照合に対してはそのまま使うことはできない．近年，パラメタ化文字列照合に対するFM-indexをはじめとして，様々な一般化文字列照合に対するFM-indexが盛んに研究されている．本年度は，パラメタ化文字列照合に対するFM-indexを効率的に省スペースで構築する問題に取り組んだ．パラメタ化文字列照合に対する索引はFM-indexベースのもの以外にも様々提案されており，それらを効率的に構築する手法も広く研究されているが，最も省スペースな索引であるFM-indexベースの索引を省スペースで構築する手法はこれまで提案されていなかった．本研究では，この問題に内在する技術的な課題を解決し，パラメタ化文字列照合に対するFM-indexベースの索引を省スペースで構築する初の手法を提案した．

Research Products

• [Journal Article] Computing Longest Lyndon Subsequences and Longest Common Lyndon Subsequences (2024)
  • Author(s): Hideo Bannai and Tomohiro I and Tomasz Kociumaka and Dominik Koeppl and Simon J. Puglisi
  • Journal Title: Algorithmica Volume: 86 Issue: 3 Pages: 735-756
  • DOI: 10.1007/s00453-023-01125-z
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] On the Hardness of Smallest RLSLPs and Collage Systems (2024)
  • Author(s): Akiyoshi Kawamoto, Tomohiro I, Dominik Koeppl, Hideo Bannai
  • Journal Title: Proc. Data Compression Conference (DCC) 2024 Volume: - Pages: 243-252
  • Peer Reviewed
• [Journal Article] Breaking a Barrier in Constructing Compact Indexes for Parameterized Pattern Matching (2024)
  • Author(s): Kento Iseri, Tomohiro I, Diptarama Hendrian, Dominik Koeppl, Ryo Yoshinaka, Ayumi Shinohara
  • Journal Title: Proc. 51st International Colloquium on Automata, Languages, and Programming (ICALP) 2024 Volume: -
  • Peer Reviewed / Open Access
• [Journal Article] Longest bordered and periodic subsequences (2023)
  • Author(s): Hideo Bannai and Tomohiro I and Dominik Koeppl
  • Journal Title: Inf. Process. Lett. Volume: 182 Pages: 1-6
  • DOI: 10.1016/j.ipl.2023.106398
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] PalFM-index: FM-index for Palindrome Pattern Matching (2023)
  • Author(s): Shinya Nagashita, Tomohiro I
  • Journal Title: Proc. 34th Annual Symposium on Combinatorial Pattern Matching (CPM) 2023 Volume: -
  • Peer Reviewed / Open Access
• [Journal Article] Space-Efficient B Trees via Load-Balancing (2022)
  • Author(s): Tomohiro I, Dominik Koeppl
  • Journal Title: Proc. 33rd International Workshop on Combinatorial Algorithms (IWOCA) 2022 Volume: － Pages: 327-340
  • DOI: 10.1007/978-3-031-06678-8_24
  • ISBN: 9783031066771, 9783031066788
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Computing Longest (Common) Lyndon Subsequences (2022)
  • Author(s): Hideo Bannai, Tomohiro I, Tomasz Kociumaka, Dominik Koeppl, Simon J. Puglisi
  • Journal Title: Proc. 33rd International Workshop on Combinatorial Algorithms (IWOCA) 2022 Volume: － Pages: 128-142
  • DOI: 10.1007/978-3-031-06678-8_10
  • ISBN: 9783031066771, 9783031066788
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Converting RLBWT to LZ77 in smaller space (2022)
  • Author(s): Masaki Shigekuni, Tomohiro I
  • Journal Title: IEEE Computer Society Press CPS Online Volume: － Pages: 242-251
  • Peer Reviewed
• [Journal Article] PHONI: Streamed Matching Statistics with Multi-genome References (2021)
  • Author(s): Christina Boucher, Travis Gagie, Tomohiro I, Dominik Koeppl, Ben Langmead, Giovanni Manzini, Gonzalo Navarro, Alejandro Pacheco, Massimiliano Rossi
  • Journal Title: Proc. Data Compression Conference (DCC) 2021 Volume: - Pages: 193-202
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Deterministic Sparse Suffix Sorting in the Restore Model (2020)
  • Author(s): Johannes Fischer, Tomohiro I, Dominik Koeppl
  • Journal Title: ACM Transactions on Algorithms Volume: 16(4) Issue: 4 Pages: 1-53
  • DOI: 10.1145/3398681
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Re-Pair in Small Space (2020)
  • Author(s): Dominik Koeppl, Tomohiro I, Isamu Furuya, Yoshimasa Takabatake, Kensuke Sakai, Keisuke Goto,
  • Journal Title: Algorithms Volume: 14(1) Issue: 1 Pages: 1-20
  • DOI: 10.3390/a14010005
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Practical Random Access to SLP-Compressed Texts (2020)
  • Author(s): Travis Gagie, Tomohiro I, Giovanni Manzini, Gonzalo Navarro, Hiroshi Sakamoto, Louisa Seelbach Benkner, Yoshimasa Takabatake,
  • Journal Title: Proc. 27th International Symposium on String Processing and Information Retrieval (SPIRE) 2020 Volume: - Pages: 221-231
  • DOI: 10.1007/978-3-030-59212-7_16
  • ISBN: 9783030592110, 9783030592127
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Dynamic index and LZ factorization in compressed space (2020)
  • Author(s): Takaaki Nishimoto, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, Masayuki Takeda
  • Journal Title: Discrete Applied Mathematics Volume: 274 Pages: 116-129
  • DOI: 10.1016/j.dam.2019.01.014
  • Peer Reviewed
• [Journal Article] Faster Privacy-Preserving Computation of Edit Distance with Moves (2020)
  • Author(s): Yohei Yoshimoto, Masaharu Kataoka, Yoshimasa Takabatake, Tomohiro I, Kilho Shin, Hiroshi Sakamoto
  • Journal Title: Proc. International Workshop on Algorithms and Computation (WALCOM) 2020 Volume: - Pages: 308-320
  • DOI: 10.1007/978-3-030-39881-1_26
  • ISBN: 9783030398804, 9783030398811
  • Peer Reviewed
• [Journal Article] Rpair: Rescaling RePair with Rsync (2019)
  • Author(s): Travis Gagie, Tomohiro I, Giovanni Manzini, Gonzalo Navarro, Hiroshi Sakamoto, Yoshimasa Takabatake
  • Journal Title: Proc. 26th International Symposium on String Processing and Information Retrieval (SPIRE) 2019 Volume: - Pages: 35-44
  • DOI: 10.1007/978-3-030-32686-9_3
  • ISBN: 9783030326852, 9783030326869
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] k-Abelian Pattern Matching: Revisited, Corrected, and Extended (2019)
  • Author(s): Golnaz Badkobeh, Hideo Bannai, Maxime Crochemore, Tomohiro I, Shunsuke Inenaga, Shiho Sugimoto
  • Journal Title: Proc. Prague Stringology Conference 2019 Volume: - Pages: 29-40
  • Peer Reviewed
• [Presentation] On the Hardness of Smallest RLSLPs and Collage Systems (2024)
  • Author(s): Akiyoshi Kawamoto, Tomohiro I, Dominik Koeppl, Hideo Bannai
  • Organizer: Data Compression Conference (DCC) 2024
  • Int'l Joint Research
• [Presentation] PalFM-index: FM-index for Palindrome Pattern Matching (2023)
  • Author(s): Shinya Nagashita and Tomohiro I
  • Organizer: 34th Annual Symposium on Combinatorial Pattern Matching (CPM) 2023
  • Int'l Joint Research
• [Presentation] PalFM-index: FM-index for Palindrome Pattern Matching (2023)
  • Author(s): Shinya Nagashita, Tomohiro I
  • Organizer: 34th Annual Symposium on Combinatorial Pattern Matching (CPM) 2023
  • Int'l Joint Research
• [Presentation] Space-Efficient B Trees via Load-Balancing (2022)
  • Author(s): Tomohiro I, Dominik Koeppl
  • Organizer: 33rd International Workshop on Combinatorial Algorithms (IWOCA) 2022
  • Int'l Joint Research
• [Presentation] Computing Longest (Common) Lyndon Subsequences (2022)
  • Author(s): Hideo Bannai, Tomohiro I, Tomasz Kociumaka, Dominik Koeppl, Simon J. Puglisi
  • Organizer: 33rd International Workshop on Combinatorial Algorithms (IWOCA) 2022
  • Int'l Joint Research
• [Presentation] Converting RLBWT to LZ77 in smaller space (2022)
  • Author(s): Masaki Shigekuni, Tomohiro I
  • Organizer: Data Compression Conference 2022
  • Int'l Joint Research
• [Presentation] PHONI: Streamed Matching Statistics with Multi-genome References (2021)
  • Author(s): Christina Boucher, Travis Gagie, Tomohiro I, Dominik Koeppl, Ben Langmead, Giovanni Manzini, Gonzalo Navarro, Alejandro Pacheco, Massimiliano Rossi
  • Organizer: Data Compression Conference (DCC) 2021
  • Int'l Joint Research
• [Presentation] Re-Pair in Small Space (2020)
  • Author(s): Dominik Koeppl, Tomohiro I, Isamu Furuya, Yoshimasa Takabatake, Kensuke Sakai, Keisuke Goto
  • Organizer: Prague Stringology Conference (PSC) 2020
  • Int'l Joint Research
• [Presentation] Practical Random Access to SLP-Compressed Texts (2020)
  • Author(s): Travis Gagie, Tomohiro I, Giovanni Manzini, Gonzalo Navarro, Hiroshi Sakamoto, Louisa Seelbach Benkner, Yoshimasa Takabatake
  • Organizer: 27th International Symposium on String Processing and Information Retrieval (SPIRE) 2020
  • Int'l Joint Research
• [Presentation] Rpair: Rescaling RePair with Rsync (2019)
  • Author(s): Tomohiro I
  • Organizer: SPIRE
  • Int'l Joint Research