Deep Generative Modeling

This textbook tackles the problem of formulating AI systems by combining probabilistic modeling and deep learning. Moreover, it goes beyond typical predictive modeling and brings together supervised learning and unsupervised learning. The resulting paradigm, called deep generative modeling, utilizes the generative perspective on perceiving the surrounding world. It assumes that each phenomenon is driven by an underlying generative process that defines a joint distribution over random variables and their stochastic interactions, i.e., how events occur and in what order. The adjective deep comes from the fact that the distribution is parameterized using deep neural networks. There are two distinct traits of deep generative modeling. First, the application of deep neural networks allows rich and flexible parameterization of distributions. Second, the principled manner of modeling stochastic dependencies using probability theory ensures rigorous formulation and prevents potential flaws in reasoning. Moreover, probability theory provides a unified framework where the likelihood function plays a crucial role in quantifying uncertainty and defining objective functions.

Deep Generative Modeling is designed to appeal to curious students, engineers, and researchers with a modest mathematical background in undergraduate calculus, linear algebra, probability theory, and the basics in machine learning, deep learning, and programming in Python and PyTorch (or other deep learning libraries). It will appeal to students and researchers from a variety of backgrounds, including computer science, engineering, data science, physics, and bioinformatics, who wish to become familiar with deep generative modeling. To engage the reader, the book introduces fundamental concepts with specific examples and code snippets. The full code accompanying the book is available on github.

The ultimate aim of the book is to outline the most important techniques in deep generative modeling and, eventually, enable readers to formulate new models and implement them.

這本教科書探討了結合概率建模和深度學習來制定人工智慧系統的問題。此外，它超越了典型的預測建模，將監督學習和非監督學習結合在一起。所得到的範式被稱為深度生成建模，利用生成觀點來感知周圍世界。它假設每個現象都由一個潛在的生成過程驅動，該過程定義了隨機變量及其隨機交互的聯合分佈，即事件如何發生以及以什麼順序發生。形容詞深度來自於使用深度神經網絡對分佈進行參數化。深度生成建模有兩個明顯的特點。首先，應用深度神經網絡可以對分佈進行豐富而靈活的參數化。其次，使用概率理論以原則性的方式建模隨機依賴關係，確保了嚴謹的表述並防止了推理中的潛在缺陷。此外，概率理論提供了一個統一的框架，其中似然函數在量化不確定性和定義目標函數方面起著關鍵作用。

《深度生成建模》旨在吸引對深度生成建模感興趣的學生、工程師和研究人員，他們具有本科微積分、線性代數、概率論以及機器學習、深度學習和Python和PyTorch（或其他深度學習庫）的基礎知識。這本書將吸引來自各種背景的學生和研究人員，包括計算機科學、工程學、數據科學、物理學和生物信息學，他們希望熟悉深度生成建模。為了吸引讀者，本書通過具體的例子和代碼片段介紹基本概念。附帶的完整代碼可在github上找到。

本書的最終目標是概述深度生成建模中最重要的技術，並最終使讀者能夠制定新模型並實施它們。