2014 Fiscal Year Research-status Report
Improving High-Performance Neural Network-based Sequence Classifiers
| Project/Area Number |
26730093
|
|---|
| Research Institution | Kyushu University |
|---|
Principal Investigator |
FRINKEN Volkmar 九州大学, システム情報科学研究科(研究院, 学術研究員 (70724417)
|
|---|
| Project Period (FY) |
2014-04-01 – 2016-03-31
|
| Keywords | Machine Learning / Sequence Classification / Deep Learning / Pattern Recognition / LSTM Neural Networks |
|---|
| Outline of Annual Research Achievements |
The work on high performance-based sequence classifiers resulted in valuable insight and one publication. The biggest problem in analyzing sequences are long-term dependencies and contextual information. A study on many different network architectures revealed underlying mechanism of successful sequence processing. By using deep recurrent neural networks, the classification of a sequence element depends upon two processing dimensions: the layers of the neural network and previous sequence elements. This adds another dimension to the training and new algorithms are needed. Currently, efficient training algorithms can deal either with sequence dependencies or multiple layers, but not both at the same time.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
Gaining insights into the behavior of the recurrent neural network sequence classifier turned out to be extremely challenging. Initial estimations to model the internal states of a NN with a hidden Markov model - as mentioned in the "Purpose of the Research" section in the initial submission - had been unsuccessful. To still reach the goal of improving training and performance of these recognizers, a different approach was chosen. A detailed study on different, deep, architectures, nevertheless, revealed interesting insights that can be used in the future.
|
| Strategy for Future Research Activity |
Currently, the internal behavior and processes of large neural network-based recognition system for sequence processing are still not fully understood. More research has to be done. In addition, there a need for new training algorithms became obvious during my research. Deep neural networks for sequences have to deal with sequence dependencies as well as multiple layers. These are two related, yet different dimensions of dependencies that need to be taken into account. Existing training algorithms currently can only handle either very deep NN for static data, or sequential data with shallow networks. A combined approach, however, is very promising for future research directions.
|
| Causes of Carryover |
The estimation of the travel expenses cannot always be done with great certainty. In the end, the flight was cheaper than anticipated. The money needed was a bit less than I expected. Since there was no usage for that money in the last fiscal year, it is better to transfer it to the next year.
|
| Expenditure Plan for Carryover Budget |
There is no current plan for the extra 10,186円. It will be kept as a buffer, in case some items turn out to be more expensive.
|
