| 研究実績の概要 |
In the project, Improving High-Performance Neural Network-based Sequence Recognizers, I focused my effort to understand the internal dynamics of LSTM recurrent neural networks. During the course of the year, I have done my investigations along two major directions. The first research direction considered the node activations of the neural networks during the recognition of a sequence. For simple tasks, such as counting peaks in a sequence, the activation levels of the LSTM nodes can be interpreted fairly easily. Unfortunately, the same cannot be said for more complex networks and tasks, such as handwriting recognition. We could make an observation that only a subset of the nodes in the hidden layer takes on meaningful values, whereas the remaining nodes constantly increase their stored value, regardless of the input. Our hope was to detect the minimal number or meaningful hidden nodes for different tasks and establish a form of intrinsic complexity dimensionality for different databases. Unfortunately, the results were inconclusive.
The second research direction was to increase the layers for LSTM based handwriting recognition in a systematic way. We carefully analyzed various topologies with different number of hidden layers and different node activation functions. The results indicate that these meta-parameters have a significant influence. Our findings have been published and can help to improve future systems.
|
