Pattern Discovery and Data Classification Based on String Compression

• Project/Area Number: 22680014
• Research Category: Grant-in-Aid for Young Scientists (A)
• Allocation Type: Single-year Grants
• Research Field: Intelligent informatics
• Research Institution: Kyushu University
• Principal Investigator: BANNAI Hideo 九州大学, システム情報研究院, 准教授 (20323644)
• Project Period (FY): 2010 – 2012
• Project Status: Completed (Fiscal Year 2012)
• Budget Amount: ¥7,540,000 (Direct Cost: ¥5,800,000、Indirect Cost: ¥1,740,000); Fiscal Year 2012: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000); Fiscal Year 2011: ¥2,470,000 (Direct Cost: ¥1,900,000、Indirect Cost: ¥570,000); Fiscal Year 2010: ¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000)
• Keywords: 圧縮文字列処理 / パターン発見 / 直線的プログラム / 文字n-グラム / 文字列パターン発見 / 文字列データ分類 / q-gram

Research Abstract

Compressed string processing is an approach that aims to process a compressed representation of the string without explicitly decompressing it. In this study, we investigated the application of this approach to the problem of string pattern discovery and string data classification, and developed various efficient algorithms. Especially for the q-gram frequencies problem, we succeeded in developing a practically efficient algorithm that can be faster than directly processing the uncompressed text, showing, the effectiveness of the approach to the string pattern discovery and string classification problems

Research Products

• [Journal Article] Simpler and Faster Lempel Ziv Factorization, Proc. (2013)
  • Author(s): Keisuke Goto and Hideo Bannai
  • Journal Title: Data Compression Conference 2013 (DCC 2013) Pages: 133-142
• [Journal Article] From Run Length Encoding to LZ78 and Back Again (2013)
  • Author(s): Yuya Tamakoshi, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda
  • Journal Title: Proc. Data Compression Conference 2013 (DCC 2013) Pages: 143-152
• [Journal Article] Computing convolution on grammar-compressed text (2013)
  • Author(s): Toshiya Tanaka, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda
  • Journal Title: Proc. Data Compression Conference 2013 (DCC 2013) Pages: 451-460
• [Journal Article] Fast q-gram mining on SLP compressed strings (2013)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, and Masayuki Takeda
  • Journal Title: Journal of Discrete Algorithms Volume: 18 Pages: 89-99
  • NAID: 120006654954
• [Journal Article] Simpler and Faster Lempel Ziv Factorization (2013)
  • Author(s): Keisuke Goto and Hideo Bannai
  • Journal Title: Proc. Data Compression Conference 2013 (DCC 2013) Volume: DCC 2013 Pages: 133-142
  • DOI: 10.1109/dcc.2013.21
  • Peer Reviewed
• [Journal Article] Efficient LZ78 factorization of grammar compressed text, Proc (2012)
  • Author(s): Hideo Bannai, Shunsuke Inenaga, and Masayuki Takeda
  • Journal Title: 19th International Symposium on String Processing and Information Retrieval (SPIRE 2012) Volume: 7608 Pages: 86-98
• [Journal Article] An Efficient Algorithm to Test Square-Freeness of Strings Compressed by Straight-Line Programs (2012)
  • Author(s): Hideo Bannai, Travis Gagie, Tomohiro I, Shunsuke Inenaga, Gad M. Landau, and Moshe Lewenstein
  • Journal Title: Information Processing Letters Volume: 112(19) Pages: 711-714
• [Journal Article] Speeding up q-gram mining on grammar-based compressed texts (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proc. 23rd Annual Symposium on Combinatorial Pattern Matching (CPM 2012) Volume: 7354 Pages: 220-231
• [Journal Article] Computing q-gram Non-overlapping Frequencies on SLP Compressed Texts (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proc. 38th International Conference on Current Trends in Theory and Practice of Computer Science (SOFSEM 2012) Volume: 7147 Pages: 301-312
• [Journal Article] Finding Characteristic Substrings from Compressed Texts (2012)
  • Author(s): Shunsuke Inenaga, Hideo Bannai
  • Journal Title: International Journal of Foundations of Computer Science Volume: 23(2) Pages: 261-280
• [Journal Article] Efficient LZ78 Factorization of Grammar Compressed Text (2012)
  • Author(s): Hideo Bannai
  • Journal Title: SPIRE 2012 Volume: - Pages: 86-98
  • DOI: 10.1007/978-3-642-34109-0_10
  • ISBN: 9783642341083, 9783642341090
  • Peer Reviewed
• [Journal Article] Computing q-gram Non-overlapping Frequencies on SLP Compressed Texts (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proceedings of the 38th International Conference on Current Trends in Theory and Practice of Computer Science (SOFSEM 2012) Volume: LNOS7147 Pages: 301-312
  • DOI: 10.1007/978-3-642-27660-6_25
  • ISBN: 9783642276590, 9783642276606
  • Peer Reviewed
• [Journal Article] Speeding up q-gram mining on grammar-based compressed texts (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proceedings of the 23rd Annual Symposium on Combinatorial Pattern Matching (CPM 2012) Volume: (掲載決定)
  • Peer Reviewed
• [Journal Article] Fast q-gram Mining on SLP Compressed Strings (2011)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proc. 18th International Symposium on String Processing and Information Retrieval (SPIRE 2011) Volume: 7024 Pages: 278-289
  • NAID: 120006654954
• [Journal Article] Faster Subsequence and Don't-Care Pattern Matching on Compressed Texts (2011)
  • Author(s): Takanori Yamamoto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proc. 22nd Annual Symposium on Combinatorial Pattern Matching (CPM 2011) Volume: 6661 Pages: 309-322
  • NAID: 120006654962
• [Journal Article] Fast q-gram Mining on SLP Compressed Strings (2011)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proceedings of the 18th International Symposium on String Processing and Information Retrieval (SPIRE 2011) Volume: LNCS7024 Pages: 278-289
  • DOI: 10.1007/978-3-642-24583-1_27
  • NAID: 120006654954
  • ISBN: 9783642245824, 9783642245831
  • Peer Reviewed
• [Journal Article] Faster Subsequence and Don't-Care Pattern Matching on Compressed Texts (2011)
  • Author(s): Takanori Yamamoto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Journal Title: Proceedings of the 22nd Annual Symposium on Combinatorial Pattern Matching (CPM 2011) Volume: (掲載確定)
  • NAID: 120006654962
  • Peer Reviewed
• [Journal Article] Sparse Substring Pattern Set Discovery using Linear Programming Boosting (2010)
  • Author(s): Kazuaki Kashihara, Kohei Hatano, Hideo Bannai, Masayuki Takeda
  • Journal Title: Proc. 13th International Conference on Discovery Science (DS 2010) Volume: 6332 Pages: 132-143
• [Journal Article] Sparse Substring Pattern Set Discovery using Linear Programming Boosting (2010)
  • Author(s): Kazuaki Kashihara, Kohei Hatano, Hideo Bannai, Masayuki Takeda
  • Journal Title: Proceedings of the 13th International Conference on Discovery Science (DS 2010) Volume: LNAI 6332 Pages: 132-143
  • Peer Reviewed
• [Presentation] Simpler and Faster Lempel Ziv Factorization (2013)
  • Author(s): Keisuke Goto and Hideo Bannai
  • Organizer: Data Compression Conference 2013 (DCC 2013)
  • Place of Presentation: Snowbird, USA.
• [Presentation] From Run Length Encoding to LZ78 and Back Again (2013)
  • Author(s): Yuya Tamakoshi, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda
  • Organizer: Data Compression Conference 2013 (DCC 2013)
  • Place of Presentation: Snowbird, USA.
• [Presentation] Computing convolution on grammar-compressed text (2013)
  • Author(s): Toshiya Tanaka, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda
  • Organizer: Data Compression Conference 2013 (DCC 2013)
  • Place of Presentation: Snowbird, USA.
• [Presentation] Computing convolution on grammar-compressed text (2013)
  • Author(s): Toshiya Tanaka, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda
  • Organizer: Data Compression Conference 2013 (DCC 2013)
  • Place of Presentation: Snowbird, Utah, USA
• [Presentation] From Run Length Encoding to LZ78 and Back Again (2013)
  • Author(s): Yuya Tamakoshi, Tomohiro I, Shunsuke Inenaga, Hideo Bannai, and Masayuki Takeda
  • Organizer: Data Compression Conference 2013 (DCC 2013)
  • Place of Presentation: Snowbird, Utah, USA
• [Presentation] Simpler and Faster Lempel Ziv Factorization (2013)
  • Author(s): Keisuke Goto and Hideo Bannai
  • Organizer: Data Compression Conference 2013 (DCC 2013)
  • Place of Presentation: Snowbird, Utah, USA
• [Presentation] Improved q-gram Mining on SLP Compressed Strings (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Organizer: London Stringology Days/London Algorithmic Workshop 2012 (LSD & LAW 2012)
  • Place of Presentation: King's College London, London, United Kingdom
  • Year and Date: 2012-02-09
• [Presentation] Efficient LZ78 factorization of grammar compressed text (2012)
  • Author(s): Hideo Bannai, Shunsuke Inenaga, and Masayuki Takeda
  • Organizer: 19th International Symposium on String Processing and Information Retrieval (SPIRE 2012)
  • Place of Presentation: Cartagena, Colombia
• [Presentation] Speeding up q-gram mining on grammar-based compressed texts (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Organizer: 23rd Annual Symposium on Combinatorial Pattern Matching (CPM 2012)
  • Place of Presentation: Helsinki, Finland
• [Presentation] Computing q-gram Non-overlapping Frequencies on SLP Compressed Texts (2012)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Organizer: 38th International Conference on Current Trends in Theory and Practice of Computer Science (SOFSEM 2012)
  • Place of Presentation: spindleruv Mlyn, Czech Republic.
• [Presentation] Efficient LZ78 factorization of grammar compressed text, (2012)
  • Author(s): Hideo Bannai, Shunsuke Inenaga, and Masayuki Takeda
  • Organizer: 19th International Symposium on String Processing and Information Retrieval (SPIRE 2012)
  • Place of Presentation: Cartagena, Colombia
• [Presentation] Fast q-gram Mining on SLP Compressed Strings (2011)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Organizer: Second Workshop on Algorithms for Large-Scale Information Processing in Knowledge Discovery (ALSIP 2011)
  • Place of Presentation: サンポートホール高松(高松市)
  • Year and Date: 2011-12-01
• [Presentation] 圧縮テキスト上でのn-gram非重複頻度の効率的な計算とその応用 (2011)
  • Author(s): 後藤啓介, 坂内英夫, 稲永俊介, 竹田正幸
  • Organizer: 第134回アルゴリズム研究発表会
  • Place of Presentation: 琉球大学
  • Year and Date: 2011-03-07
• [Presentation] 圧縮テキスト上でのVLDCパターン照合問題 (2011)
  • Author(s): 山本卓典, 坂内英夫, 稲永俊介, 竹田正幸
  • Organizer: 第134回アルゴリズム研究発表会
  • Place of Presentation: 琉球大学
  • Year and Date: 2011-03-07
• [Presentation] 圧縮文字列上でのn-gram頻度の高速な計算方法 (2011)
  • Author(s): 後藤啓介, 坂内英夫, 稲永俊介, 竹田正幸
  • Organizer: 冬のLAシンポジウム2010
  • Place of Presentation: 京都大学
  • Year and Date: 2011-02-02
• [Presentation] 圧縮テキスト上での高速エピソードパターン照合 (2011)
  • Author(s): 山本卓典, 坂内英夫, 稲永俊介, 竹田正幸
  • Organizer: 冬のLAシンポジウム2010
  • Place of Presentation: 京都大学
  • Year and Date: 2011-02-02
• [Presentation] Fast q-gram Mining on SLP Compressed Strings (2011)
  • Author(s): Keisuke Goto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Organizer: 18th International Symposium on String Processing and Information Retrieval (SPIRE 2011)
  • Place of Presentation: Pisa, Italy.
• [Presentation] Faster Subsequence and Don't-Care Pattern Matching on Compressed Texts (2011)
  • Author(s): Takanori Yamamoto, Hideo Bannai, Shunsuke Inenaga, Masayuki Takeda
  • Organizer: 22nd Annual Symposium on Combinatorial Pattern Matching (CPM 2011)
  • Place of Presentation: Palermo, Italy.
• [Presentation] Sparse Substring Pattern Set Discovery using Linear Programming Boosting (2010)
  • Author(s): Kazuaki Kashihara, Kohei Hatano, Hideo Bannai, Masayuki Takeda
  • Organizer: 13th International Conference on Discovery Science (DS 2010)
  • Place of Presentation: Canberra, Australia.