Normalization Techniques in Deep Learning

This book presents and surveys normalization techniques with a deep analysis in training deep neural networks. In addition, the author provides technical details in designing new normalization methods and network architectures tailored to specific tasks. Normalization methods can improve the training stability, optimization efficiency, and generalization ability of deep neural networks (DNNs) and have become basic components in most state-of-the-art DNN architectures. The author provides guidelines for elaborating, understanding, and applying normalization methods. This book is ideal for readers working on the development of novel deep learning algorithms and/or their applications to solve practical problems in computer vision and machine learning tasks. The book also serves as a resource researchers, engineers, and students who are new to the field and need to understand and train DNNs.

本書介紹並探討了深度神經網絡訓練中的正規化技術。此外，作者提供了設計新的正規化方法和針對特定任務定制網絡架構的技術細節。正規化方法可以提高深度神經網絡（DNN）的訓練穩定性、優化效率和泛化能力，已成為大多數最先進的DNN架構的基本組件。作者提供了關於詳細解釋、理解和應用正規化方法的指南。本書非常適合從事新型深度學習算法開發和/或將其應用於解決計算機視覺和機器學習任務實際問題的讀者。本書還可作為新手研究人員、工程師和學生了解和訓練DNN的資源。

Lei Huang, Ph.D., is an Associate Professor at Beihang University. His current research interests include normalization techniques involving methods, theories, and applications in training deep neural networks (DNNs). He also has wide interests in representation and optimization of deep learning theory and computer vision tasks. Dr. Huang serves as a reviewer for top-tier conferences and journals in machine learning and computer vision.

黃磊博士是北京航空航天大學的副教授。他目前的研究興趣包括在訓練深度神經網絡（DNN）中涉及方法、理論和應用的正規化技術。他還對深度學習理論和計算機視覺任務的表示和優化有廣泛的興趣。黃博士擔任機器學習和計算機視覺領域頂級會議和期刊的審稿人。