統計計算與智能分析理論及其Python實踐

本書介紹統計計算與智能分析基礎理論以及基於Python的模型算法實現。全書由13章組成，主要內容有：隨機數生成技術；數據探索性分析；特徵提取與選擇方法；最大期望算法；Markov鏈蒙特卡羅方法；重採樣技術；重要採樣技術；序貫重要性採樣；非參數概率密度估計；非參數回歸分析；樹模型理論；概率圖模型；模型性能評價技術。

第 1章 隨機數生成技術..............................................................1 1.1標準分佈的隨機數生成 ....................................................... 1 1.1.1連續型隨機變量模擬生成 ...................................................2 1.1.2離散型隨機變量模擬生成 ..................................................10 1.2非標準分佈的隨機數生成 ................................................... 14 1.2.1逆變換法 ............................................................... 14 1.2.2接受-拒絕法與自適應拒絕法 ............................................... 16 1.2.3組合法 ................................................................. 22 1.3隨機過程的隨機數生成 ......................................................26 1.3.1馬爾可夫過程模擬生成....................................................27 1.3.2泊松過程模擬生成 ....................................................... 31 1.3.3維納過程模擬生成 ....................................................... 35 1.4基於變分自編碼器模型的數據生成 .......................................... 36 1.4.1 VAE模型基本思想.......................................................37 1.4.2變分自編碼器模型 ....................................................... 37 1.5基於生成式對抗網絡的數據生成 .............................................46 1.5.1 GANs的基本原理 ....................................................... 46 1.5.2 GANs理論推導 ......................................................... 48 1.5.3 GANs算法的近優算法 ................................................... 53 1.6習題 ........................................................................ 57 第 2章 探索性數據分析 ............................................................ 61 2.1一維探索性數據分析 ........................................................ 61 2.1.1匯總統計量 ............................................................. 62 2.1.2直方圖 ................................................................. 65 2.1.3莖葉圖 ................................................................. 66 2.1.4箱線圖 ................................................................. 68 2.1.5正態概率圖 ............................................................. 70 2.1.6 Q-Q圖 .................................................................72 2.2多維探索性數據分析 ........................................................ 75 2.2.1多屬性統計量 ........................................................... 75 2.2.2散點圖 ................................................................. 77 2.2.3邊緣直方圖 ............................................................. 83 2.2.4邊緣箱形圖 ............................................................. 84 2.2.5成對圖 ................................................................. 86 2.2.6 Box-Cox線性變換圖 ..................................................... 87 2.2.7自相關圖和偏自相關圖....................................................90 2.2.8交叉相關圖 .............................................................94 2.2.9滯後圖 ................................................................. 95 2.3習題 ........................................................................ 97 第 3章 特徵提取與選擇方法 .......................................................100 3.1特徵提取方法 ..............................................................100 3.1.1主成分分析 ............................................................ 100 3.1.2因子分析 .............................................................. 109 3.1.3獨立分量分析 .......................................................... 115 3.1.4線性判別分析 .......................................................... 125 3.2時間序列的特徵提取方法 .................................................. 130 3.2.1 STL分解算法 ..........................................................130 3.2.2經驗模態分解 .......................................................... 132 3.2.3奇異譜分析方法 ........................................................ 139 3.2.4小波變換 .............................................................. 143 3.3特徵選擇方法 ..............................................................160 3.3.1過濾特徵選擇 .......................................................... 161 3.3.2 Wrapper法 ............................................................163 3.3.3 Embedded法 .......................................................... 166 3.3.4貝葉斯統計和正則化 .................................................... 168 3.4習題 .......................................................................173 第 4章 最大期望算法..............................................................176 4.1從極大似然估計到 EM算法 ............................................... 176 4.2 EM算法原理與實現 ....................................................... 178 4.2.1 EM算法原理 .......................................................... 178 4.2.2 EM算法 .............................................................. 180 4.3 EM算法應用 .............................................................. 184 4.3.1 K-Means聚類算法 ......................................................184 4.3.2高斯混合模型聚類算法 .................................................. 187 4.3.3 K-Means和 GMM的關系 ............................................... 195 4.4習題 .......................................................................195 第 5章 馬爾可夫鏈蒙特卡羅方法 .................................................. 197 5.1蒙特卡羅方法引入 ......................................................... 197 5.2馬爾可夫鏈蒙特卡羅方法 .................................................. 198 5.3 Metropolis-Hastings採樣 .................................................. 200 5.3.1 Metropolis採樣算法 .................................................... 200 5.3.2 Metropolis-Hastings採樣算法 ............................................ 204 5.3.3多維 Metropolis-Hastings採樣算法 ....................................... 207 5.4 Gibbs採樣 ................................................................ 209 5.5馬爾可夫鏈蒙特卡羅方法應用 ..............................................213 5.5.1基於 MCMC的貝葉斯統計推斷...........................................213 5.5.2可逆跳轉 MCMC方法 .................................................. 215 5.6習題 .......................................................................220 第 6章 重採樣技術 ................................................................222 6.1刀切法.....................................................................222 6.1.1刀切法基本原理 ........................................................ 222 6.1.2刀切法算法與實現 ...................................................... 225 6.2自助法.....................................................................225 6.2.1自助法基本原理 ........................................................ 225 6.2.2 Rn的統計特性 ......................................................... 229 6.3重採樣技術的應用 ......................................................... 230 6.3.1 Bagging算法 .......................................................... 230 6.3.2 Boosting算法 ..........................................................237 6.3.3總結 .................................................................. 244 6.4習題 .......................................................................244 第 7章 重要抽樣技術..............................................................247 7.1重要抽樣基本原理 ......................................................... 247 7.2分層重要抽樣方法 ......................................................... 253 7.3重要抽樣在深度學習中的應用 ..............................................257 7.4習題 .......................................................................260 第 8章 序貫重要抽樣..............................................................263 8.1貝葉斯重要抽樣方法 ....................................................... 264 8.2序貫重要抽樣算法 ......................................................... 265 8.3重要函數的選擇 ........................................................... 267 8.4重採樣方法 ................................................................ 270 8.5習題 .......................................................................274 第 9章 非參數概率密度估計 .......................................................276 9.1直方圖法 .................................................................. 276 9.2 Parzen窗估計法 ...........................................................279 9.3 K.近鄰法 .................................................................281 9.4核密度估計法 ..............................................................283 9.5 B樣條密度估計 ........................................................... 291 9.6習題 .......................................................................296 第 10章 非參數回歸分析 .......................................................... 298 10.1非參數回歸概念 .......................................................... 298 10.2權函數方法 ...............................................................299 10.2.1核權函數法 ........................................................... 299 10.2.2局部多項式回歸 ....................................................... 302 10.2.3局部多項式加權散點圖平滑估計 ..........................................304 10.3最近鄰函數法.............................................................306 10.4習題 ......................................................................309 第 11章 樹模型理論...............................................................311 11.1決策樹模型 ...............................................................311 11.1.1決策樹分類算法 ....................................................... 311 11.1.2特徵選擇 ............................................................. 313 11.1.3決策樹的生成 ......................................................... 318 11.1.4剪枝過程 ............................................................. 319 11.2分類回歸樹模型 .......................................................... 321 11.3提升樹模型 ...............................................................328 11.3.1 GBDT模型 .......................................................... 328 11.3.2 XGBoost模型 ........................................................ 340 11.3.3 LightGBM模型 ....................................................... 344 11.4習題 ......................................................................350 第 12章 概率圖模型...............................................................353 12.1貝葉斯網絡 ...............................................................353 12.1.1貝葉斯方法與貝葉斯定理 ............................................... 353 12.1.2貝葉斯網絡 ........................................................... 357 12.1.3貝葉斯網絡結構學習算法 ............................................... 364 12.1.4貝葉斯網絡推理 ....................................................... 366 12.1.5動態貝葉斯網絡 ....................................................... 368 12.2馬爾可夫網絡.............................................................369 12.2.1馬爾可夫網絡定義 ..................................................... 370 12.2.2條件獨立性質 ......................................................... 371 12.2.3馬爾可夫網絡分解 ..................................................... 373 12.3因子圖 ................................................................... 375 12.3.1因子圖定義與描述 ..................................................... 375 12.3.2因子圖的提取——和積算法 ............................................. 377 12.4習題 ......................................................................381 第 13章 模型性能評價技術 ........................................................382 13.1模型評價方法.............................................................382 13.1.1交叉驗證過程 ......................................................... 383 13.1.2簡單交叉驗證 ......................................................... 384 13.1.3 k-折交叉驗證 ......................................................... 384 13.1.4留一交叉驗證 ......................................................... 385 13.1.5 Bootstrap交叉驗證 ....................................................385 13.2模型性能評價指標 ........................................................ 386 13.2.1分類模型評價指標 ..................................................... 386 13.2.2回歸模型評價指標 ..................................................... 393 13.3習題 ......................................................................395 參考文獻 ............................................................................ 396