Transfer Learning Through Embedding Spaces

Recent progress in artificial intelligence (AI) has revolutionized our everyday life. Many AI algorithms have reached human-level performance and AI agents are replacing humans in most professions. It is predicted that this trend will continue and 30% of work activities in 60% of current occupations will be automated.

This success, however, is conditioned on availability of huge annotated datasets to training AI models. Data annotation is a time-consuming and expensive task which still is being performed by human workers. Learning efficiently from less data is a next step for making AI more similar to natural intelligence. Transfer learning has been suggested a remedy to relax the need for data annotation. The core idea in transfer learning is to transfer knowledge across similar tasks and use similarities and previously learned knowledge to learn more efficiently.

In this book, we provide a brief background on transfer learning and then focus on the idea of transferring knowledge through intermediate embedding spaces. The idea is to couple and relate different learning through embedding spaces that encode task-level relations and similarities. We cover various machine learning scenarios and demonstrate that this idea can be used to overcome challenges of zero-shot learning, few-shot learning, domain adaptation, continual learning, lifelong learning, and collaborative learning.

近年來，人工智慧（AI）的進展已經徹底改變了我們的日常生活。許多AI演算法已經達到了人類水平的表現，AI代理人正在取代大部分職業中的人類。預測這一趨勢將繼續下去，目前職業中有60%的工作活動將有30%被自動化。

然而，這一成功是建立在大量標註數據集用於訓練AI模型的基礎上的。數據標註是一項耗時且昂貴的任務，目前仍然由人工工作者執行。從少量數據中高效學習是使AI更接近自然智能的下一步。轉移學習被提出作為減輕數據標註需求的解決方案。轉移學習的核心思想是在相似任務之間轉移知識，利用相似性和先前學到的知識來更高效地學習。

在本書中，我們提供了轉移學習的簡要背景，然後專注於通過中間嵌入空間傳遞知識的想法。這個想法是通過編碼任務級別的關係和相似性的嵌入空間來耦合和關聯不同的學習。我們涵蓋了各種機器學習場景，並展示了這個想法可以用於克服零樣本學習、少樣本學習、領域適應、持續學習、終身學習和協作學習的挑戰。

Mohammad Rostami is a computer scientist at USC Information Sciences Institute. He is a graduate of the University of Pennsylvania, University of Waterloo, and Sharif University of Technology. His research area includes continual machine learning and learning in data scarce regimes.

Mohammad Rostami 是南加州大學資訊科學研究所的電腦科學家。他畢業於賓夕法尼亞大學、滑鐵盧大學和莎里夫科技大學。他的研究領域包括持續機器學習和在資料稀缺環境中的學習。