Guided Research Jiax Zhao

Literature review of the state of art Transfer Learning and Multi-task Learning applications

Deep Learning is popular over the past ten years. It can be used in many fields such as computer vision, speech recognition, natural language processing, machine translation, etc. The tremendous potential value behind deep learning lies in massive labeled data. The deeper the network is, the more data we need to train the models. However, data sparsity should be given careful attention to, since deep neural network can not be trained sufficiently when there is no enough data. How to solve data sparsity problem is an important topic now.

Transfer Learning and Multi-Task Learning can both be used to solve data sparsity problem, now there is tremendous excellent paper about diversiform applications of these two learning methods. As a result, it’s hard for beginners to choose suitable methods when dealing with data sparsity problems.

Although both TL and MTL can tackle data sparsity problem, there are differences between them. In a nutshell, Transfer Learning is more like sequential learning, it learns new task after task learned before, and only new task matters. On the other hand, instead of using multiple models, Multi-Task Learning learns several tasks which share common features at the same time in one model, and all tasks are important. 

My goal of this guided research is: First, learn more about Transfer Learning and Multi-task learning theoretically and applicationally. Then, come up with my own conclusion of which learning method suits which type of applications. Last but not least, write a research paper containing both theory part and application part with more focus on application part. 

The process of the research can be divided into 4 parts: 1.Keywords definition 2.Paper finding 3.Paper reading and reviewing 4.Paper feedback and conclusion

Overall, this research paper focuses on data sparsity problem and has an overview of two different learning methods Transfer Learning and Multi-task Learning to solve data sparsity in different use cases. With the guidance of this paper, people can choose their models properly.