2021 Fiscal Year Annual Research Report
Explore how to facilitate the human creative process from the perspective of Computer G raphics
| Project/Area Number |
21F20075
|
|---|
| Research Institution | The University of Tokyo |
|---|
Principal Investigator |
五十嵐 健夫 東京大学, 大学院情報理工学系研究科, 教授 (80345123)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
SHEN I-CHAO 東京大学, 情報理工学(系)研究科, 外国人特別研究員
|
|---|
| Project Period (FY) |
2021-04-28 – 2023-03-31
|
| Keywords | コンピュータグラフィックス / 設計支援アルゴリズム |
|---|
| Outline of Annual Research Achievements |
In the past one year, I have finished 6 projects, and have 3 papers accepted, while the remaining 3 projects are still under submission. The central theme of these projects are to bridge this gap by designing data-driven algorithms and user interfaces using human preferences of visual content and knowledge. These projects are relevant to sketch beautification, real-time virtual try-on, support system for neural network architecture editing, image-based shape part manipulation, icon usability design, and automatic 2D polygonal construction sequence prediction.
In more detail, I co-developed a virtual try-on system based on a per-garment capture and synthesis workflow to handle rich interactions by training the model with many systematically captured images. Using a custom actuated mannequin, our system captures detailed deformations of the target garments under diverse body sizes and poses. I also propose a unique human-in-the-loop framework that allows users to improve the usabilities of interface icons efficiently. We formulate several usability criteria into a perceptual usability function and enable users to iteratively revise an icon set with an interactive design tool, EvIcon. Finally, I proposed a hybrid method that generates a polygonal mesh construction sequence from a silhouette image. The key idea of our method is the use of the Monte Carlo tree search (MCTS) algorithm and differentiable rendering to separately predict sequential topological actions and geometric actions.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
Currently, I am working on how to make the proposed new technologies available for more users, more specifically, working on prototypes that other users can access. For EvIcon, I am working on a web-based prototype so the designers can use a browser to use our technology. Our interface augments existing vector graphics design tools with additional usability feedback panels. The interface contains three main panels: (i) the main canvas panel which includes a vector graphics editor for icon revision and a list to present the uploaded icon set, (ii) perceptual feedback panel (box with blue borderline), and (iii) distinguishability visualization panel (box with orange borderline). I plan to publish a version of the system after I submitted EvIcon project after the deadline in mid-May. Meanwhile, I am exploring new method for improving the quality of my virtual try-on project. The direction of improving this project including improve the temporal coherence, and the video quality, and support multiple new functions, such as layering, and remove the measure garment requirement. I already started to experiment several improving directions at this moment
|
| Strategy for Future Research Activity |
I plan to keep developing novel assistive algorithms and user interfaces for supporting visual content design, emphasizing the construction sequences of 3D polygonal modeling. The complexity of 3D polygonal modeling, in terms of the number and the order of operations and the time required to execute them, makes it challenging to learn and execute. Automatic 3D polygonal modeling My previous work AutoPoly derives the construction sequence of a 2D target shape, and it opens up exciting directions for general and efficient construction sequence prediction.
Following are several directions I plan to investigate: Artist-like 3D construction sequence prediction. Currently, the derived construction sequence from AutoPoly did not match how humans would construct a polygonal model, not to mention a professional artist. Another limitation of AutoPoly is that it can only predict construction sequences for 2D shapes because of the inefficient search due to the enormous searching space. In order to predict a 3D modeling construction sequence that is more artist-like, we have to overcome several challenges.
As a starting point, I plan to construct a new dataset by collecting 3D modeling construction sequences from professional 3D modeling artists. The construction sequences collected from 3D modeling artists can be used to guide the searching process and optimization in AutoPoly so that the algorithm derives construction sequences that match the properties in the collected construction sequences.
|
Research Products
(6 results)
