• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Research on Augmented Real Big Data Processing Frameworks with High-level Virtualization Facilities

Research Project

Project/Area Number 19H04114
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 60080:Database-related
Research InstitutionUniversity of Tsukuba

Principal Investigator

Kitagawa Hiroyuki  筑波大学, 国際統合睡眠医科学研究機構, 教授 (00204876)

Co-Investigator(Kenkyū-buntansha) 天笠 俊之  筑波大学, 計算科学研究センター, 教授 (70314531)
塩川 浩昭  筑波大学, 計算科学研究センター, 准教授 (90775248)
早瀬 康裕  筑波大学, システム情報系, 助教 (40423090)
堀江 和正  筑波大学, 計算科学研究センター, 助教 (60817112)
Project Period (FY) 2019-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥17,160,000 (Direct Cost: ¥13,200,000、Indirect Cost: ¥3,960,000)
Fiscal Year 2022: ¥3,770,000 (Direct Cost: ¥2,900,000、Indirect Cost: ¥870,000)
Fiscal Year 2021: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2020: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2019: ¥4,940,000 (Direct Cost: ¥3,800,000、Indirect Cost: ¥1,140,000)
Keywordsビッグデータ / Augmentedデータ / 仮想化 / トレーサビリティ
Outline of Research at the Start

ビッグデータ処理では,蓄積データやストリーム等の様々な大規模リアルデータに対して,結合・集約処理等に加え,機械学習等を用いた補完・推定処理等の多様なデータ処理を有機的に適用することが求められている.このような複合的データ処理を支援する上で,データ構造や処理の詳細を隠ぺいする仮想化技術が極めて重要である.本研究では,実世界から直接観測・取得されるリアルデータと,機械学習,メタデータ推論,シミュレーション等を適用することにより元データを大幅に拡充・補填するAugmentedデータをシームレスに統合する仮想化技術を確立し,それに立脚したAugmentedリアルビッグデータ利活用基盤の構築を目指す.

Outline of Final Research Achievements

This research was conducted with the aim of building a technology for realizing an augmented real big data infrastructure that can seamlessly integrate augmented data obtained by AI, machine learning, etc. with real data accumulated in a database as fact data. As a result, we devised new concepts and methods from the viewpoint of data description, consistency management, and processing efficiency related to augmented data on such topics as time-series pattern processing for sequence data, complex data analysis in databases, complex stream analysis, boundary point detection, integration of external information sources with knowledge bases, aggregate calculation in stream processing, and spatial stream processing infrastructure.

Academic Significance and Societal Importance of the Research Achievements

本研究の多くの研究成果は,査読付きの国内外ジャーナル論文,国際会議論文等で発表済である.特に,「リレーショナルデータベースにおけるAI・ML等による Augmented データ生成を含む複合分析処理」の成果はVLDB Journal,「ストリーム処理における集約値 Augmented データ計算」の成果はIEEE TKDEという,当該分野を代表するトップレベル国際ジャーナルに採択され,学術的に高い評価を得ている.本研究で対象としたAI・ML等によるAugmentedデータ生成・利用は,今後急速に社会へ普及することが予想され,本研究の成果は社会的意義も大きいものと確信する.

Report

(5 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • 2019 Annual Research Report
  • Research Products

    (35 results)

All 2023 2022 2021 2020 2019 Other

All Journal Article (20 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 20 results,  Open Access: 6 results) Presentation (14 results) (of which Int'l Joint Research: 2 results,  Invited: 4 results) Remarks (1 results)

  • [Journal Article] BPF: A Novel Cluster Boundary Points Detection Method for Static and Streaming Data2023

    • Author(s)
      Vijdan Khalique, Hiroyuki Kitagawa, and Toshiyuki Amagasa
    • Journal Title

      Knowledge and Information Systems

      Volume: - Issue: 7 Pages: 1-32

    • DOI

      10.1007/s10115-023-01854-1

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] GeoFlink: An Efficient and Scalable Spatial Data Stream Management System2022

    • Author(s)
      Salman Ahmed Shaikh, Hiroyuki Kitagawa, Akiyoshi Matono, Komal Mariam, and Kyoung-Sook Kim
    • Journal Title

      IEEE Access

      Volume: 10 Pages: 24909-24935

    • DOI

      10.1109/access.2022.3154063

    • Related Report
      2022 Annual Research Report 2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Augmented Lineage: Traceability of Data Analysis Including Complex UDF Processing2022

    • Author(s)
      Masaya Yamada, Hiroyuki Kitagawa, Toshiyuki Amagasa, Akiyoshi Maton
    • Journal Title

      The VLDB Journal

      Volume: - Issue: 5 Pages: 963-983

    • DOI

      10.1007/s00778-022-00769-7

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] BPF: An Effective Cluster Boundary Points Detection Technique2022

    • Author(s)
      Vijdan Khalique and Hiroyuki Kitagawa
    • Journal Title

      Proc. 33rd International Conference on Database and Expert Systems Applications (DEXA 2022)

      Volume: 1 Pages: 404-416

    • DOI

      10.1007/978-3-031-12423-5_31

    • ISBN
      9783031124228, 9783031124235
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] InTrans: Fast Incremental Transformer for Time Series Data Prediction2022

    • Author(s)
      Savong Bou, Toshiyuki Amagasa, Hiroyuki Kitagawa
    • Journal Title

      Proc. 33rd International Conference on Database and Expert Systems Applications (DEXA 2022)

      Volume: 2 Pages: 47-61

    • DOI

      10.1007/978-3-031-12426-6_4

    • ISBN
      9783031124259, 9783031124266
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] TStream: A Framework for Real-time and Scalable Trajectory Stream Processing and Analysis2022

    • Author(s)
      Salman Ahmed Shaikh, Hiroyuki Kitagawa, Akiyoshi Matono, Kyoung-Sook Kim
    • Journal Title

      Proc. 30th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems 2022 (ACM SIGSPATIAL 2022)

      Volume: - Pages: 1-4

    • DOI

      10.1145/3557915.3560964

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Streaming Augmented Lineage: Traceability of Complex Stream Data Analysis2022

    • Author(s)
      Masaya Yamada, Hiroyuki Kitagawa, Salman Ahmed Shaikh, Toshiyuki Amagasa, Akiyoshi Matono
    • Journal Title

      Proc. 24th International Conference on Information Integration and Web Intelligence (iiWAS2022)

      Volume: - Pages: 224-236

    • DOI

      10.1007/978-3-031-21047-1_20

    • ISBN
      9783031210464, 9783031210471
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] PR-MVI: Efficient Missing Value Imputation over Data Streams by Distance Likelihood2022

    • Author(s)
      Savong Bou, Toshiyuki Amagasa, Hiroyuki Kitagawa, Salman Ahmed Shaikh, Akiyoshi Matono
    • Journal Title

      Proc. 24th International Conference on Information Integration and Web Intelligence (iiWAS2022)

      Volume: - Pages: 338-351

    • DOI

      10.1007/978-3-031-21047-1_28

    • ISBN
      9783031210464, 9783031210471
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] An FPGA-based Accelerator for Regular Path Queries over Edge-labeled Graphs2022

    • Author(s)
      Kento Miura, Ryohei Kobayashi, Toshiyuki Amagasa, Hiroyuki Kitagawa, Norihisa Fujita, and Taisuke Boku
    • Journal Title

      Proceedings of 2022 IEEE International Conference on Big Data (IEEE BigData2022)

      Volume: - Pages: 415-422

    • DOI

      10.1109/bigdata55660.2022.10020406

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Region-based Sub-Snapshot (RegSnap): Enhanced Fault Tolerance in Distributed Stream Processing with Partial Snapshot2022

    • Author(s)
      Takdir, Hiroyuki Kitagawa, and Toshiyuki Amagasa
    • Journal Title

      Proceedings of 2022 IEEE International Conference on Big Data (IEEE BigData2022)

      Volume: - Pages: 3374-3382

    • DOI

      10.1109/bigdata55660.2022.10020607

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] CPiX: Real-Time Analytics Over Out-of-Order Data Streams By Incremental Sliding-Window Aggregation2021

    • Author(s)
      Bou Savong、Kitagawa Hiroyuki、Amagasa Toshiyuki
    • Journal Title

      IEEE Transactions on Knowledge and Data Engineering

      Volume: - Issue: 11 Pages: 1-1

    • DOI

      10.1109/tkde.2021.3054898

    • Related Report
      2022 Annual Research Report 2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] VOA*: Fast Angle-Based Outlier Detection over High-Dimensional Data Streams2021

    • Author(s)
      Khalique Vijdan、Kitagawa Hiroyuki
    • Journal Title

      Proc. 25th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD2021

      Volume: - Pages: 40-52

    • DOI

      10.1007/978-3-030-75762-5_4

    • ISBN
      9783030757618, 9783030757625
    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Augmented Lineage: Traceability of Data Analysis Including Complex UDFs2021

    • Author(s)
      Yamada Masaya、Kitagawa Hiroyuki、Amagasa Toshiyuki、Matono Akiyoshi
    • Journal Title

      Proc. 32nd International Conference on Database and Expert Systems Applications (DEXA2021)

      Volume: - Pages: 65-77

    • DOI

      10.1007/978-3-030-86472-9_6

    • ISBN
      9783030864712, 9783030864729
    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Continuous Top-k Spatial-Keyword Search on Dynamic Objects2021

    • Author(s)
      Yuyang Dong, Chuan Xiao, Hanxiong Chen, Jeffrey Xu Yu, Kunihiro Takeoka, Masafumi Oyamada, and Hiroyuki Kitagawa
    • Journal Title

      The VLDB Journal

      Volume: 30 Issue: 2 Pages: 141-161

    • DOI

      10.1007/s00778-020-00627-4

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] シーケンスデータに対する行パターンマッチングの効率化2021

    • Author(s)
      中挾晃介,北川博之
    • Journal Title

      情報処理学会論文誌

      Volume: 62 Pages: 302-320

    • NAID

      170000184263

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] StreamingCube: Seamless Integration of Stream Processing and OLAP Analysis2020

    • Author(s)
      Shaikh Salman Ahmed、Kitagawa Hiroyuki
    • Journal Title

      IEEE Access

      Volume: 8 Pages: 104632-104649

    • DOI

      10.1109/access.2020.2999572

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] GeoFlink: A Distributed and Scalable Framework for the Real-time Processing of Spatial Streams2020

    • Author(s)
      Shaikh Salman Ahmed、Mariam Komal、Kitagawa Hiroyuki、Kim Kyoung-Sook
    • Journal Title

      Proc. 29th ACM International Conference on Information and Knowledge Management (CIKM2020)

      Volume: - Pages: 3149-3156

    • DOI

      10.1145/3340531.3412761

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] L-BiX: incremental sliding-window aggregation over data streams using linear bidirectional aggregating indexes2020

    • Author(s)
      Bou Savong、Kitagawa Hiroyuki、Amagasa Toshiyuki
    • Journal Title

      Knowledge and Information Systems

      Volume: - Issue: 8 Pages: 3107-3131

    • DOI

      10.1007/s10115-020-01444-5

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Efficient Row Pattern Matching Using Pattern Hierarchies for Sequence OLAP2019

    • Author(s)
      Nasu Yuya、Kitagawa Hiroyuki、Nakabasami Kosuke
    • Journal Title

      Proc. 21st International Conference on Big Data Analytics and Knowledge Discovery (DaWak2019)

      Volume: 11708 Pages: 89-104

    • DOI

      10.1007/978-3-030-27520-4_7

    • ISBN
      9783030275198, 9783030275204
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Optimization of Row Pattern Matching over Sequence Data in Spark SQL2019

    • Author(s)
      Nakabasami Kosuke、Kitagawa Hiroyuki、Nasu Yuya
    • Journal Title

      Proc. 30th International Conference on Database and Expert Systems Applications (DEXA2019)

      Volume: 11706 Pages: 3-17

    • DOI

      10.1007/978-3-030-27615-7_1

    • ISBN
      9783030276140, 9783030276157
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Presentation] エンティティリンキング機能を有する知識ベースと外部情報源の統合利用手法2023

    • Author(s)
      大森雄基, 北川博之, 天笠俊之
    • Organizer
      第15回データ工学と情報マネジメントに関するフォーラム (DEIM 2023)
    • Related Report
      2022 Annual Research Report
  • [Presentation] 複合的ストリーム処理に対するトレーサビリティの研究2023

    • Author(s)
      山田真也, 北川博之, Salman Ahmed Shaikh, 天笠俊之, 的野晃
    • Organizer
      第15回データ工学と情報マネジメントに関するフォーラム (DEIM 2023)
    • Related Report
      2022 Annual Research Report
  • [Presentation] リアルタイム性を考慮した自動睡眠ステージ判定システムの設計2023

    • Author(s)
      国生泰資, 山田空, 堀江和正, 阿部高志, 北川博之
    • Organizer
      第15回データ工学と情報マネジメントに関するフォーラム (DEIM 2023)
    • Related Report
      2022 Annual Research Report
  • [Presentation] Big Sequence Data Analysis: From Stream Processing Technology to Applications in Sleep Medicine2022

    • Author(s)
      Hiroyuki Kitagawa
    • Organizer
      IRI2022
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] ユーザ定義関数を利用した知識ベースと外部情報源の統合利用手法2022

    • Author(s)
      大森雄基, 北川博之, 天笠俊之
    • Organizer
      第14回データ工学と情報マネジメントに関するフォーラム (DEIM 2022)
    • Related Report
      2021 Annual Research Report
  • [Presentation] 複合的データ分析処理に対する拡張来歴導出手法と性能評価2022

    • Author(s)
      山田真也, 北川博之, 天笠俊之, 的野晃整
    • Organizer
      第14回データ工学と情報マネジメントに関するフォーラム (DEIM 2022)
    • Related Report
      2021 Annual Research Report
  • [Presentation] 知識ベースと外部情報源の統合利用環境2022

    • Author(s)
      大森雄基,北川博之,天笠俊之
    • Organizer
      情報処理学会第84回全国大会 (IPSJ全国大会 2022)
    • Related Report
      2021 Annual Research Report
  • [Presentation] ストリーム処理の基礎:Velocityへのたゆまざる挑戦2021

    • Author(s)
      北川博之
    • Organizer
      最強データベース講義シリーズ#9,日本データベース学会
    • Related Report
      2021 Annual Research Report
    • Invited
  • [Presentation] 複合的データ解析を伴う分析処理に対するトレーサビリティの研究2021

    • Author(s)
      山田真也, 北川博之, 天笠俊之
    • Organizer
      第13回データ工学と情報マネジメントに関するフォーラム (DEIM 2021)
    • Related Report
      2020 Annual Research Report
  • [Presentation] バンディットアルゴリズムとメンション関係を利用した特定トピックに関する特定の地域のツイートの収集2021

    • Author(s)
      大森雄基,北川博之,天笠俊之
    • Organizer
      情報処理学会第83回全国大会 (IPSJ全国大会 2021)
    • Related Report
      2020 Annual Research Report
  • [Presentation] Computing as a Scienceを担うデータベース研究2020

    • Author(s)
      北川博之
    • Organizer
      情報処理学会コンピュータサイエンス領域功績賞受賞記念講演,情報処理学会第171回データベースシステム研究会・情報処理学会第140回情報基礎とアクセス技術研究会・電子情報通信学会データ工学研究会合同研究会
    • Related Report
      2020 Annual Research Report
    • Invited
  • [Presentation] Topic-aware Scheme for Collecting Local Tweets2020

    • Author(s)
      Carina Miwa Yoshimura, Hiroyuki Kitagawa
    • Organizer
      第12回データ工学と情報マネジメントに関するフォーラム (DEIM 2020)
    • Related Report
      2019 Annual Research Report
  • [Presentation] コンテンツ解析を含む大規模データ分析処理に対するトレーサビリティ2020

    • Author(s)
      山田真也,天笠俊之,北川博之
    • Organizer
      情報処理学会第82回全国大会 (IPSJ全国大会 2020)
    • Related Report
      2019 Annual Research Report
  • [Presentation] Big Data Analytics and Management: Perspectives from Big Sequence Data Analysis and Research Projects in Japan2019

    • Author(s)
      Hiroyuki Kitagawa
    • Organizer
      The 36th CCF National Database Conference (NDBC2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Remarks] 筑波大学 知識・データ工学研究室

    • URL

      https://www.kde.cs.tsukuba.ac.jp/

    • Related Report
      2021 Annual Research Report 2020 Annual Research Report

URL: 

Published: 2019-04-18   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi