4. 簡(jiǎn)單線性回歸的python實(shí)現(xiàn)

點(diǎn)擊標(biāo)題即可獲取源代碼和筆記

4.1 導(dǎo)入相關(guān)包

import numpy as np import pandas as pd import random import matplotlib as mpl import matplotlib.pyplot as pltplt.rcParams['font.sans-serif'] = ['simhei'] # 顯示中文 plt.rcParams['axes.unicode_minus'] = False # 用來(lái)正常顯示負(fù)號(hào)%matplotlib inline # 將圖片嵌套在輸出框中顯示，而不是單獨(dú)跳出一張圖片

4.2 導(dǎo)入數(shù)據(jù)集并探索數(shù)據(jù)

ex0 = pd.read_table("./datas/ex0.txt",header=None) ex0.head() 01201234

1.0	0.067732	3.176513
1.0	0.427810	3.816464
1.0	0.995731	4.550095
1.0	0.738336	4.256571
1.0	0.981083	4.560815

ex0.shape (200, 3) ex0.describe() 012countmeanstdmin25%50%75%max

200.0	200.000000	200.000000
1.0	0.488319	3.835601
0.0	0.292943	0.503443
1.0	0.014855	3.078132
1.0	0.234368	3.452775
1.0	0.466573	3.839350
1.0	0.730712	4.247613
1.0	0.995731	4.692514

4.3 構(gòu)建輔助函數(shù)

ex0.iloc[:,-1].values array([3.176513, 3.816464, 4.550095, 4.256571, 4.560815, 3.929515,3.52617 , 3.156393, 3.110301, 3.149813, 3.476346, 4.119688,4.282233, 3.486582, 4.655492, 3.965162, 3.5149 , 3.125947,4.094115, 3.476039, 3.21061 , 3.190612, 4.631504, 4.29589 ,3.085028, 3.44808 , 3.16744 , 3.364266, 3.993482, 3.891471,3.143259, 3.114204, 3.851484, 4.621899, 4.580768, 3.620992,3.580501, 4.618706, 3.676867, 4.641845, 3.175939, 4.26498 ,3.558448, 3.436632, 3.831052, 3.182853, 3.498906, 3.946833,3.900583, 4.238522, 4.23308 , 3.521557, 3.203344, 4.278105,3.555705, 3.502661, 3.859776, 4.275956, 3.916191, 3.587961,3.183004, 4.225236, 4.231083, 4.240544, 3.222372, 4.021445,3.567479, 3.56258 , 4.262059, 3.208813, 3.169825, 4.193949,3.491678, 4.533306, 3.550108, 4.636427, 3.557078, 3.552874,3.494159, 3.206828, 3.195266, 4.221292, 4.413372, 4.184347,3.742878, 3.201878, 4.648964, 3.510117, 3.274434, 3.579622,3.489244, 4.237386, 3.913749, 3.22899 , 4.286286, 4.628614,3.239536, 4.457997, 3.513384, 3.729674, 3.834274, 3.811155,3.598316, 4.692514, 4.604859, 3.864912, 3.184236, 3.500796,3.743365, 3.622905, 4.310796, 3.583357, 3.901852, 3.233521,3.105266, 3.865544, 4.628625, 4.231213, 3.791149, 3.968271,4.25391 , 3.19471 , 3.996503, 3.904358, 3.503976, 4.557545,3.699876, 4.613614, 3.140401, 4.206717, 3.969524, 4.476096,3.136528, 4.279071, 3.200603, 3.299012, 3.209873, 3.632942,3.248361, 3.995783, 3.563262, 3.649712, 3.951845, 3.145031,3.181577, 4.637087, 3.404964, 3.873188, 4.633648, 3.154768,4.623637, 3.078132, 3.913596, 3.221817, 3.938071, 3.880822,4.176436, 4.648161, 3.332312, 4.240614, 4.532224, 4.557105,4.610072, 4.636569, 4.229813, 3.50086 , 4.245514, 4.605182,3.45434 , 3.180775, 3.38082 , 4.56502 , 3.279973, 4.554241,4.63352 , 4.281037, 3.844426, 3.891601, 3.849728, 3.492215,4.592374, 4.632025, 3.75675 , 3.133555, 3.567919, 4.363382,3.560165, 4.564305, 4.215055, 4.174999, 4.58664 , 3.960008,3.529963, 4.213412, 3.908685, 3.585821, 4.374394, 3.213817,3.952681, 3.129283]) ex0.iloc[:,-1].values.shape (200,) (ex0.iloc[:,-1].values).T array([3.176513, 3.816464, 4.550095, 4.256571, 4.560815, 3.929515,3.52617 , 3.156393, 3.110301, 3.149813, 3.476346, 4.119688,4.282233, 3.486582, 4.655492, 3.965162, 3.5149 , 3.125947,4.094115, 3.476039, 3.21061 , 3.190612, 4.631504, 4.29589 ,3.085028, 3.44808 , 3.16744 , 3.364266, 3.993482, 3.891471,3.143259, 3.114204, 3.851484, 4.621899, 4.580768, 3.620992,3.580501, 4.618706, 3.676867, 4.641845, 3.175939, 4.26498 ,3.558448, 3.436632, 3.831052, 3.182853, 3.498906, 3.946833,3.900583, 4.238522, 4.23308 , 3.521557, 3.203344, 4.278105,3.555705, 3.502661, 3.859776, 4.275956, 3.916191, 3.587961,3.183004, 4.225236, 4.231083, 4.240544, 3.222372, 4.021445,3.567479, 3.56258 , 4.262059, 3.208813, 3.169825, 4.193949,3.491678, 4.533306, 3.550108, 4.636427, 3.557078, 3.552874,3.494159, 3.206828, 3.195266, 4.221292, 4.413372, 4.184347,3.742878, 3.201878, 4.648964, 3.510117, 3.274434, 3.579622,3.489244, 4.237386, 3.913749, 3.22899 , 4.286286, 4.628614,3.239536, 4.457997, 3.513384, 3.729674, 3.834274, 3.811155,3.598316, 4.692514, 4.604859, 3.864912, 3.184236, 3.500796,3.743365, 3.622905, 4.310796, 3.583357, 3.901852, 3.233521,3.105266, 3.865544, 4.628625, 4.231213, 3.791149, 3.968271,4.25391 , 3.19471 , 3.996503, 3.904358, 3.503976, 4.557545,3.699876, 4.613614, 3.140401, 4.206717, 3.969524, 4.476096,3.136528, 4.279071, 3.200603, 3.299012, 3.209873, 3.632942,3.248361, 3.995783, 3.563262, 3.649712, 3.951845, 3.145031,3.181577, 4.637087, 3.404964, 3.873188, 4.633648, 3.154768,4.623637, 3.078132, 3.913596, 3.221817, 3.938071, 3.880822,4.176436, 4.648161, 3.332312, 4.240614, 4.532224, 4.557105,4.610072, 4.636569, 4.229813, 3.50086 , 4.245514, 4.605182,3.45434 , 3.180775, 3.38082 , 4.56502 , 3.279973, 4.554241,4.63352 , 4.281037, 3.844426, 3.891601, 3.849728, 3.492215,4.592374, 4.632025, 3.75675 , 3.133555, 3.567919, 4.363382,3.560165, 4.564305, 4.215055, 4.174999, 4.58664 , 3.960008,3.529963, 4.213412, 3.908685, 3.585821, 4.374394, 3.213817,3.952681, 3.129283]) (ex0.iloc[:,-1].values).T.shape (200,) ''' 函數(shù)功能：輸入DF數(shù)據(jù)集（最后一列為標(biāo)簽），返回特征矩陣和標(biāo)簽矩陣 ''' def get_Mat(dataSet):xMat = np.mat(dataSet.iloc[:,:-1].values)yMat = np.mat(dataSet.iloc[:,-1].values).Treturn xMat,yMat # 查看函數(shù)運(yùn)行結(jié)果 xMat,yMat = get_Mat(ex0) xMat.shape (200, 2) xMat matrix([[1. , 0.067732],[1. , 0.42781 ],[1. , 0.995731],[1. , 0.738336],[1. , 0.981083],[1. , 0.526171],[1. , 0.378887],[1. , 0.033859],[1. , 0.132791],[1. , 0.138306],[1. , 0.247809],[1. , 0.64827 ],[1. , 0.731209],[1. , 0.236833],[1. , 0.969788],[1. , 0.607492],[1. , 0.358622],[1. , 0.147846],[1. , 0.63782 ],[1. , 0.230372],[1. , 0.070237],[1. , 0.067154],[1. , 0.925577],[1. , 0.717733],[1. , 0.015371],[1. , 0.33507 ],[1. , 0.040486],[1. , 0.212575],[1. , 0.617218],[1. , 0.541196],[1. , 0.045353],[1. , 0.126762],[1. , 0.556486],[1. , 0.901144],[1. , 0.958476],[1. , 0.274561],[1. , 0.394396],[1. , 0.87248 ],[1. , 0.409932],[1. , 0.908969],[1. , 0.166819],[1. , 0.665016],[1. , 0.263727],[1. , 0.231214],[1. , 0.552928],[1. , 0.047744],[1. , 0.365746],[1. , 0.495002],[1. , 0.493466],[1. , 0.792101],[1. , 0.76966 ],[1. , 0.251821],[1. , 0.181951],[1. , 0.808177],[1. , 0.334116],[1. , 0.33863 ],[1. , 0.452584],[1. , 0.69477 ],[1. , 0.590902],[1. , 0.307928],[1. , 0.148364],[1. , 0.70218 ],[1. , 0.721544],[1. , 0.666886],[1. , 0.124931],[1. , 0.618286],[1. , 0.381086],[1. , 0.385643],[1. , 0.777175],[1. , 0.116089],[1. , 0.115487],[1. , 0.66351 ],[1. , 0.254884],[1. , 0.993888],[1. , 0.295434],[1. , 0.952523],[1. , 0.307047],[1. , 0.277261],[1. , 0.279101],[1. , 0.175724],[1. , 0.156383],[1. , 0.733165],[1. , 0.848142],[1. , 0.771184],[1. , 0.429492],[1. , 0.162176],[1. , 0.917064],[1. , 0.315044],[1. , 0.201473],[1. , 0.297038],[1. , 0.336647],[1. , 0.666109],[1. , 0.583888],[1. , 0.085031],[1. , 0.687006],[1. , 0.949655],[1. , 0.189912],[1. , 0.844027],[1. , 0.333288],[1. , 0.427035],[1. , 0.466369],[1. , 0.550659],[1. , 0.278213],[1. , 0.918769],[1. , 0.886555],[1. , 0.569488],[1. , 0.066379],[1. , 0.335751],[1. , 0.426863],[1. , 0.395746],[1. , 0.694221],[1. , 0.27276 ],[1. , 0.503495],[1. , 0.067119],[1. , 0.038326],[1. , 0.599122],[1. , 0.947054],[1. , 0.671279],[1. , 0.434811],[1. , 0.509381],[1. , 0.749442],[1. , 0.058014],[1. , 0.482978],[1. , 0.466776],[1. , 0.357767],[1. , 0.949123],[1. , 0.41732 ],[1. , 0.920461],[1. , 0.156433],[1. , 0.656662],[1. , 0.616418],[1. , 0.853428],[1. , 0.133295],[1. , 0.693007],[1. , 0.178449],[1. , 0.199526],[1. , 0.073224],[1. , 0.286515],[1. , 0.182026],[1. , 0.621523],[1. , 0.344584],[1. , 0.398556],[1. , 0.480369],[1. , 0.15335 ],[1. , 0.171846],[1. , 0.867082],[1. , 0.223855],[1. , 0.528301],[1. , 0.890192],[1. , 0.106352],[1. , 0.917886],[1. , 0.014855],[1. , 0.567682],[1. , 0.068854],[1. , 0.603535],[1. , 0.53205 ],[1. , 0.651362],[1. , 0.901225],[1. , 0.204337],[1. , 0.696081],[1. , 0.963924],[1. , 0.98139 ],[1. , 0.987911],[1. , 0.990947],[1. , 0.736021],[1. , 0.253574],[1. , 0.674722],[1. , 0.939368],[1. , 0.235419],[1. , 0.110521],[1. , 0.218023],[1. , 0.869778],[1. , 0.19683 ],[1. , 0.958178],[1. , 0.972673],[1. , 0.745797],[1. , 0.445674],[1. , 0.470557],[1. , 0.549236],[1. , 0.335691],[1. , 0.884739],[1. , 0.918916],[1. , 0.441815],[1. , 0.116598],[1. , 0.359274],[1. , 0.814811],[1. , 0.387125],[1. , 0.982243],[1. , 0.78088 ],[1. , 0.652565],[1. , 0.87003 ],[1. , 0.604755],[1. , 0.255212],[1. , 0.730546],[1. , 0.493829],[1. , 0.257017],[1. , 0.833735],[1. , 0.070095],[1. , 0.52707 ],[1. , 0.116163]]) # xMat.A ,把matrix變?yōu)閍rray類型 xMat.A[:,1] array([0.067732, 0.42781 , 0.995731, 0.738336, 0.981083, 0.526171,0.378887, 0.033859, 0.132791, 0.138306, 0.247809, 0.64827 ,0.731209, 0.236833, 0.969788, 0.607492, 0.358622, 0.147846,0.63782 , 0.230372, 0.070237, 0.067154, 0.925577, 0.717733,0.015371, 0.33507 , 0.040486, 0.212575, 0.617218, 0.541196,0.045353, 0.126762, 0.556486, 0.901144, 0.958476, 0.274561,0.394396, 0.87248 , 0.409932, 0.908969, 0.166819, 0.665016,0.263727, 0.231214, 0.552928, 0.047744, 0.365746, 0.495002,0.493466, 0.792101, 0.76966 , 0.251821, 0.181951, 0.808177,0.334116, 0.33863 , 0.452584, 0.69477 , 0.590902, 0.307928,0.148364, 0.70218 , 0.721544, 0.666886, 0.124931, 0.618286,0.381086, 0.385643, 0.777175, 0.116089, 0.115487, 0.66351 ,0.254884, 0.993888, 0.295434, 0.952523, 0.307047, 0.277261,0.279101, 0.175724, 0.156383, 0.733165, 0.848142, 0.771184,0.429492, 0.162176, 0.917064, 0.315044, 0.201473, 0.297038,0.336647, 0.666109, 0.583888, 0.085031, 0.687006, 0.949655,0.189912, 0.844027, 0.333288, 0.427035, 0.466369, 0.550659,0.278213, 0.918769, 0.886555, 0.569488, 0.066379, 0.335751,0.426863, 0.395746, 0.694221, 0.27276 , 0.503495, 0.067119,0.038326, 0.599122, 0.947054, 0.671279, 0.434811, 0.509381,0.749442, 0.058014, 0.482978, 0.466776, 0.357767, 0.949123,0.41732 , 0.920461, 0.156433, 0.656662, 0.616418, 0.853428,0.133295, 0.693007, 0.178449, 0.199526, 0.073224, 0.286515,0.182026, 0.621523, 0.344584, 0.398556, 0.480369, 0.15335 ,0.171846, 0.867082, 0.223855, 0.528301, 0.890192, 0.106352,0.917886, 0.014855, 0.567682, 0.068854, 0.603535, 0.53205 ,0.651362, 0.901225, 0.204337, 0.696081, 0.963924, 0.98139 ,0.987911, 0.990947, 0.736021, 0.253574, 0.674722, 0.939368,0.235419, 0.110521, 0.218023, 0.869778, 0.19683 , 0.958178,0.972673, 0.745797, 0.445674, 0.470557, 0.549236, 0.335691,0.884739, 0.918916, 0.441815, 0.116598, 0.359274, 0.814811,0.387125, 0.982243, 0.78088 , 0.652565, 0.87003 , 0.604755,0.255212, 0.730546, 0.493829, 0.257017, 0.833735, 0.070095,0.52707 , 0.116163]) xMat.A[:,1].shape (200,) yMat matrix([[3.176513],[3.816464],[4.550095],[4.256571],[4.560815],[3.929515],[3.52617 ],[3.156393],[3.110301],[3.149813],[3.476346],[4.119688],[4.282233],[3.486582],[4.655492],[3.965162],[3.5149 ],[3.125947],[4.094115],[3.476039],[3.21061 ],[3.190612],[4.631504],[4.29589 ],[3.085028],[3.44808 ],[3.16744 ],[3.364266],[3.993482],[3.891471],[3.143259],[3.114204],[3.851484],[4.621899],[4.580768],[3.620992],[3.580501],[4.618706],[3.676867],[4.641845],[3.175939],[4.26498 ],[3.558448],[3.436632],[3.831052],[3.182853],[3.498906],[3.946833],[3.900583],[4.238522],[4.23308 ],[3.521557],[3.203344],[4.278105],[3.555705],[3.502661],[3.859776],[4.275956],[3.916191],[3.587961],[3.183004],[4.225236],[4.231083],[4.240544],[3.222372],[4.021445],[3.567479],[3.56258 ],[4.262059],[3.208813],[3.169825],[4.193949],[3.491678],[4.533306],[3.550108],[4.636427],[3.557078],[3.552874],[3.494159],[3.206828],[3.195266],[4.221292],[4.413372],[4.184347],[3.742878],[3.201878],[4.648964],[3.510117],[3.274434],[3.579622],[3.489244],[4.237386],[3.913749],[3.22899 ],[4.286286],[4.628614],[3.239536],[4.457997],[3.513384],[3.729674],[3.834274],[3.811155],[3.598316],[4.692514],[4.604859],[3.864912],[3.184236],[3.500796],[3.743365],[3.622905],[4.310796],[3.583357],[3.901852],[3.233521],[3.105266],[3.865544],[4.628625],[4.231213],[3.791149],[3.968271],[4.25391 ],[3.19471 ],[3.996503],[3.904358],[3.503976],[4.557545],[3.699876],[4.613614],[3.140401],[4.206717],[3.969524],[4.476096],[3.136528],[4.279071],[3.200603],[3.299012],[3.209873],[3.632942],[3.248361],[3.995783],[3.563262],[3.649712],[3.951845],[3.145031],[3.181577],[4.637087],[3.404964],[3.873188],[4.633648],[3.154768],[4.623637],[3.078132],[3.913596],[3.221817],[3.938071],[3.880822],[4.176436],[4.648161],[3.332312],[4.240614],[4.532224],[4.557105],[4.610072],[4.636569],[4.229813],[3.50086 ],[4.245514],[4.605182],[3.45434 ],[3.180775],[3.38082 ],[4.56502 ],[3.279973],[4.554241],[4.63352 ],[4.281037],[3.844426],[3.891601],[3.849728],[3.492215],[4.592374],[4.632025],[3.75675 ],[3.133555],[3.567919],[4.363382],[3.560165],[4.564305],[4.215055],[4.174999],[4.58664 ],[3.960008],[3.529963],[4.213412],[3.908685],[3.585821],[4.374394],[3.213817],[3.952681],[3.129283]]) ''' 函數(shù)功能：數(shù)據(jù)集可視化 ''' def plotShow(dataSet):xMat,yMat = get_Mat(dataSet)plt.scatter(xMat.A[:,1],yMat.A,c='b',s=5)plt.show() plotShow(ex0)

4.5 計(jì)算回歸系數(shù)

''' 函數(shù)功能：計(jì)算回歸系數(shù) 參數(shù)說(shuō)明：dataSet:原始數(shù)據(jù)集 返回：ws:回歸系數(shù) ''' def standRegres(dataSet):xMat,yMat = get_Mat(dataSet)xTx = xMat.T * xMatif np.linalg.det(xTx) == 0:print('矩陣為奇異矩陣，無(wú)法求逆！')returnws = xTx.I*(xMat.T*yMat) # xTx.I ，用來(lái)求逆矩陣return ws

說(shuō)明：det(A)指的是矩陣A的行列式（determinant），如果det(A)=0，則說(shuō)明矩陣A是奇異矩陣，不可逆。

ws = standRegres(ex0) ws matrix([[3.00774324],[1.69532264]])

4.6 繪制最佳擬合直線

''' 函數(shù)功能：繪制散點(diǎn)圖和最佳擬合直線 '''def plotReg(dataSet):xMat,yMat = get_Mat(dataSet)plt.scatter(xMat.A[:,1],yMat.A,c='b',s=5)ws = standRegres(dataSet)yHat = xMat*wsplt.plot(xMat[:,1],yHat,c='r')plt.xlabel("第2列特征的數(shù)值：xMat[:,1]")plt.ylabel("預(yù)測(cè)值：yHat")plt.title('簡(jiǎn)單線性回歸')plt.show() # 繪制ex0數(shù)據(jù)集的散點(diǎn)圖和最佳擬合直線 plotReg(ex0)

4.7 計(jì)算相關(guān)系數(shù)

xMat,yMat = get_Mat(ex0) ws = standRegres(ex0) yHat = xMat*ws np.corrcoef(yHat.T,yMat.T) # 參數(shù)需要保證兩個(gè)都是行向量 array([[1. , 0.98647356],[0.98647356, 1. ]])

該矩陣包含所有兩兩組合的相關(guān)系數(shù)。可以看到，對(duì)角線上全部為1.0，因?yàn)樽陨砥ヅ淇隙ㄊ亲钔昝赖?#xff0c;而yHat和yMat的相關(guān)系數(shù)為0.98。看起來(lái)似乎是一個(gè)不錯(cuò)的結(jié)果。但是仔細(xì)觀察數(shù)據(jù)集，會(huì)發(fā)現(xiàn)數(shù)據(jù)呈現(xiàn)有規(guī)律的波動(dòng)，但是直線似乎沒(méi)有很好的捕捉到這些波動(dòng)。

局部加權(quán)線性回歸

#此段代碼供大家參考 xMat,yMat = get_Mat(ex0) x=0.5 xi = np.arange(0,1.0,0.01) k1,k2,k3=0.5,0.1,0.01 w1 = np.exp((xi-x)**2/(-2*k1**2)) w2 = np.exp((xi-x)**2/(-2*k2**2)) w3 = np.exp((xi-x)**2/(-2*k3**2))#創(chuàng)建畫布 fig = plt.figure(figsize=(6,8),dpi=100) #子畫布1，原始數(shù)據(jù)集 fig1 = fig.add_subplot(411) plt.scatter(xMat.A[:,1],yMat.A,c='b',s=5) #子畫布2，k=0.5 fig2 = fig.add_subplot(412) plt.plot(xi,w1,color='r') plt.legend(['k = 0.5'])#子畫布3，k=0.1 fig3 = fig.add_subplot(413) plt.plot(xi,w2,color='g') plt.legend(['k = 0.1'])#子畫布4，k=0.01 fig4 = fig.add_subplot(414) plt.plot(xi,w3,color='orange') plt.legend(['k = 0.01']) plt.show()

這里假定我們預(yù)測(cè)的點(diǎn)是x=0.5，最上面的圖是原始數(shù)據(jù)集，從下面三張圖可以看出隨著k的減小，被用于訓(xùn)練模型的數(shù)據(jù)點(diǎn)越來(lái)越少。

1. 構(gòu)建LWLR函數(shù)

這個(gè)過(guò)程與簡(jiǎn)單線性函數(shù)的基本一致，唯一不同的是加入了權(quán)重weights，這里我將權(quán)重參數(shù)求解和預(yù)測(cè)yHat放在了一個(gè)函數(shù)里面。

# np.eye(5) 單位矩陣 a_eye = np.eye(5) a_eye[0,2]=55 a_eye array([[ 1., 0., 55., 0., 0.],[ 0., 1., 0., 0., 0.],[ 0., 0., 1., 0., 0.],[ 0., 0., 0., 1., 0.],[ 0., 0., 0., 0., 1.]]) a_eye[0] array([ 1., 0., 55., 0., 0.]) a_eye.T array([[ 1., 0., 0., 0., 0.],[ 0., 1., 0., 0., 0.],[55., 0., 1., 0., 0.],[ 0., 0., 0., 1., 0.],[ 0., 0., 0., 0., 1.]]) a_eye.T[0] array([1., 0., 0., 0., 0.]) ''' 函數(shù)功能：計(jì)算局部加權(quán)線性回歸的預(yù)測(cè)值 參數(shù)說(shuō)明：testMat:測(cè)試集xMat:訓(xùn)練集的特征矩陣yMat:訓(xùn)練集的標(biāo)簽矩陣返回：yHat：函數(shù)預(yù)測(cè)值 ''' def LWLR(testMat,xMat,yMat,k=1.0):n = testMat.shape[0] # 測(cè)試數(shù)據(jù)集行數(shù)m = xMat.shape[0] # 訓(xùn)練集特征矩陣行數(shù)weights = np.mat(np.eye(m)) # 用單位矩陣來(lái)初始化權(quán)重矩陣，yHat = np.zeros(n) # 用0矩陣來(lái)初始化預(yù)測(cè)值矩陣for i in range(n):for j in range(m):diffMat = testMat[i] - xMat[j]weights[j,j] = np.exp(diffMat*diffMat.T / (-2*k**2))xTx = xMat.T*(weights*xMat)if np.linalg.det(xTx) == 0:print('矩陣為奇異矩陣，無(wú)法求逆')returnws = xTx.I*(xMat.T*(weights*yMat))yHat[i] = testMat[i] * wsreturn ws,yHat xMat matrix([[1. , 0.067732],[1. , 0.42781 ],[1. , 0.995731],[1. , 0.738336],[1. , 0.981083],[1. , 0.526171],[1. , 0.378887],[1. , 0.033859],[1. , 0.132791],[1. , 0.138306],[1. , 0.247809],[1. , 0.64827 ],[1. , 0.731209],[1. , 0.236833],[1. , 0.969788],[1. , 0.607492],[1. , 0.358622],[1. , 0.147846],[1. , 0.63782 ],[1. , 0.230372],[1. , 0.070237],[1. , 0.067154],[1. , 0.925577],[1. , 0.717733],[1. , 0.015371],[1. , 0.33507 ],[1. , 0.040486],[1. , 0.212575],[1. , 0.617218],[1. , 0.541196],[1. , 0.045353],[1. , 0.126762],[1. , 0.556486],[1. , 0.901144],[1. , 0.958476],[1. , 0.274561],[1. , 0.394396],[1. , 0.87248 ],[1. , 0.409932],[1. , 0.908969],[1. , 0.166819],[1. , 0.665016],[1. , 0.263727],[1. , 0.231214],[1. , 0.552928],[1. , 0.047744],[1. , 0.365746],[1. , 0.495002],[1. , 0.493466],[1. , 0.792101],[1. , 0.76966 ],[1. , 0.251821],[1. , 0.181951],[1. , 0.808177],[1. , 0.334116],[1. , 0.33863 ],[1. , 0.452584],[1. , 0.69477 ],[1. , 0.590902],[1. , 0.307928],[1. , 0.148364],[1. , 0.70218 ],[1. , 0.721544],[1. , 0.666886],[1. , 0.124931],[1. , 0.618286],[1. , 0.381086],[1. , 0.385643],[1. , 0.777175],[1. , 0.116089],[1. , 0.115487],[1. , 0.66351 ],[1. , 0.254884],[1. , 0.993888],[1. , 0.295434],[1. , 0.952523],[1. , 0.307047],[1. , 0.277261],[1. , 0.279101],[1. , 0.175724],[1. , 0.156383],[1. , 0.733165],[1. , 0.848142],[1. , 0.771184],[1. , 0.429492],[1. , 0.162176],[1. , 0.917064],[1. , 0.315044],[1. , 0.201473],[1. , 0.297038],[1. , 0.336647],[1. , 0.666109],[1. , 0.583888],[1. , 0.085031],[1. , 0.687006],[1. , 0.949655],[1. , 0.189912],[1. , 0.844027],[1. , 0.333288],[1. , 0.427035],[1. , 0.466369],[1. , 0.550659],[1. , 0.278213],[1. , 0.918769],[1. , 0.886555],[1. , 0.569488],[1. , 0.066379],[1. , 0.335751],[1. , 0.426863],[1. , 0.395746],[1. , 0.694221],[1. , 0.27276 ],[1. , 0.503495],[1. , 0.067119],[1. , 0.038326],[1. , 0.599122],[1. , 0.947054],[1. , 0.671279],[1. , 0.434811],[1. , 0.509381],[1. , 0.749442],[1. , 0.058014],[1. , 0.482978],[1. , 0.466776],[1. , 0.357767],[1. , 0.949123],[1. , 0.41732 ],[1. , 0.920461],[1. , 0.156433],[1. , 0.656662],[1. , 0.616418],[1. , 0.853428],[1. , 0.133295],[1. , 0.693007],[1. , 0.178449],[1. , 0.199526],[1. , 0.073224],[1. , 0.286515],[1. , 0.182026],[1. , 0.621523],[1. , 0.344584],[1. , 0.398556],[1. , 0.480369],[1. , 0.15335 ],[1. , 0.171846],[1. , 0.867082],[1. , 0.223855],[1. , 0.528301],[1. , 0.890192],[1. , 0.106352],[1. , 0.917886],[1. , 0.014855],[1. , 0.567682],[1. , 0.068854],[1. , 0.603535],[1. , 0.53205 ],[1. , 0.651362],[1. , 0.901225],[1. , 0.204337],[1. , 0.696081],[1. , 0.963924],[1. , 0.98139 ],[1. , 0.987911],[1. , 0.990947],[1. , 0.736021],[1. , 0.253574],[1. , 0.674722],[1. , 0.939368],[1. , 0.235419],[1. , 0.110521],[1. , 0.218023],[1. , 0.869778],[1. , 0.19683 ],[1. , 0.958178],[1. , 0.972673],[1. , 0.745797],[1. , 0.445674],[1. , 0.470557],[1. , 0.549236],[1. , 0.335691],[1. , 0.884739],[1. , 0.918916],[1. , 0.441815],[1. , 0.116598],[1. , 0.359274],[1. , 0.814811],[1. , 0.387125],[1. , 0.982243],[1. , 0.78088 ],[1. , 0.652565],[1. , 0.87003 ],[1. , 0.604755],[1. , 0.255212],[1. , 0.730546],[1. , 0.493829],[1. , 0.257017],[1. , 0.833735],[1. , 0.070095],[1. , 0.52707 ],[1. , 0.116163]]) xMat[0] matrix([[1. , 0.067732]]) xMat[0] - xMat[1] matrix([[ 0. , -0.360078]])

2. 不同k值的結(jié)果圖

我們調(diào)整k值，然后查看不同k值對(duì)模型的影響

xMat,yMat = get_Mat(ex0) #將數(shù)據(jù)點(diǎn)排列（argsort()默認(rèn)升序排列，返回索引） srtInd = xMat[:,1].argsort(0) srtInd matrix([[151],[ 24],[ 7],[114],[ 26],[ 30],[ 45],[121],[106],[113],[ 21],[ 0],[153],[197],[ 20],[136],[ 93],[149],[169],[ 70],[ 69],[199],[183],[ 64],[ 31],[ 8],[132],[ 9],[ 17],[ 60],[143],[ 80],[128],[ 85],[ 40],[144],[ 79],[134],[ 52],[138],[ 96],[172],[135],[ 88],[158],[ 27],[170],[146],[ 19],[ 43],[168],[ 13],[ 10],[ 51],[165],[ 72],[192],[195],[ 42],[111],[ 35],[ 77],[102],[ 78],[137],[ 74],[ 89],[ 76],[ 59],[ 87],[ 98],[ 54],[ 25],[179],[107],[ 90],[ 55],[140],[124],[ 16],[184],[ 46],[ 6],[ 66],[ 67],[186],[ 36],[109],[141],[ 38],[126],[108],[ 99],[ 1],[ 84],[118],[182],[176],[ 56],[100],[123],[177],[142],[122],[ 48],[194],[ 47],[112],[119],[ 5],[198],[147],[155],[ 29],[178],[101],[ 44],[ 32],[152],[105],[ 92],[ 58],[115],[154],[191],[ 15],[130],[ 28],[ 65],[139],[ 18],[ 11],[156],[189],[129],[ 71],[ 41],[ 91],[ 63],[117],[166],[ 94],[133],[110],[ 57],[159],[ 61],[ 23],[ 62],[193],[ 12],[ 81],[164],[ 3],[175],[120],[ 50],[ 83],[ 68],[188],[ 49],[ 53],[185],[196],[ 97],[ 82],[131],[145],[171],[190],[ 37],[180],[104],[148],[ 33],[157],[ 39],[ 86],[150],[103],[181],[127],[ 22],[167],[116],[125],[ 95],[ 75],[173],[ 34],[160],[ 14],[174],[ 4],[161],[187],[162],[163],[ 73],[ 2]], dtype=int64) xMat[srtInd] matrix([[[1. , 0.014855]],[[1. , 0.015371]],[[1. , 0.033859]],[[1. , 0.038326]],[[1. , 0.040486]],[[1. , 0.045353]],[[1. , 0.047744]],[[1. , 0.058014]],[[1. , 0.066379]],[[1. , 0.067119]],[[1. , 0.067154]],[[1. , 0.067732]],[[1. , 0.068854]],[[1. , 0.070095]],[[1. , 0.070237]],[[1. , 0.073224]],[[1. , 0.085031]],[[1. , 0.106352]],[[1. , 0.110521]],[[1. , 0.115487]],[[1. , 0.116089]],[[1. , 0.116163]],[[1. , 0.116598]],[[1. , 0.124931]],[[1. , 0.126762]],[[1. , 0.132791]],[[1. , 0.133295]],[[1. , 0.138306]],[[1. , 0.147846]],[[1. , 0.148364]],[[1. , 0.15335 ]],[[1. , 0.156383]],[[1. , 0.156433]],[[1. , 0.162176]],[[1. , 0.166819]],[[1. , 0.171846]],[[1. , 0.175724]],[[1. , 0.178449]],[[1. , 0.181951]],[[1. , 0.182026]],[[1. , 0.189912]],[[1. , 0.19683 ]],[[1. , 0.199526]],[[1. , 0.201473]],[[1. , 0.204337]],[[1. , 0.212575]],[[1. , 0.218023]],[[1. , 0.223855]],[[1. , 0.230372]],[[1. , 0.231214]],[[1. , 0.235419]],[[1. , 0.236833]],[[1. , 0.247809]],[[1. , 0.251821]],[[1. , 0.253574]],[[1. , 0.254884]],[[1. , 0.255212]],[[1. , 0.257017]],[[1. , 0.263727]],[[1. , 0.27276 ]],[[1. , 0.274561]],[[1. , 0.277261]],[[1. , 0.278213]],[[1. , 0.279101]],[[1. , 0.286515]],[[1. , 0.295434]],[[1. , 0.297038]],[[1. , 0.307047]],[[1. , 0.307928]],[[1. , 0.315044]],[[1. , 0.333288]],[[1. , 0.334116]],[[1. , 0.33507 ]],[[1. , 0.335691]],[[1. , 0.335751]],[[1. , 0.336647]],[[1. , 0.33863 ]],[[1. , 0.344584]],[[1. , 0.357767]],[[1. , 0.358622]],[[1. , 0.359274]],[[1. , 0.365746]],[[1. , 0.378887]],[[1. , 0.381086]],[[1. , 0.385643]],[[1. , 0.387125]],[[1. , 0.394396]],[[1. , 0.395746]],[[1. , 0.398556]],[[1. , 0.409932]],[[1. , 0.41732 ]],[[1. , 0.426863]],[[1. , 0.427035]],[[1. , 0.42781 ]],[[1. , 0.429492]],[[1. , 0.434811]],[[1. , 0.441815]],[[1. , 0.445674]],[[1. , 0.452584]],[[1. , 0.466369]],[[1. , 0.466776]],[[1. , 0.470557]],[[1. , 0.480369]],[[1. , 0.482978]],[[1. , 0.493466]],[[1. , 0.493829]],[[1. , 0.495002]],[[1. , 0.503495]],[[1. , 0.509381]],[[1. , 0.526171]],[[1. , 0.52707 ]],[[1. , 0.528301]],[[1. , 0.53205 ]],[[1. , 0.541196]],[[1. , 0.549236]],[[1. , 0.550659]],[[1. , 0.552928]],[[1. , 0.556486]],[[1. , 0.567682]],[[1. , 0.569488]],[[1. , 0.583888]],[[1. , 0.590902]],[[1. , 0.599122]],[[1. , 0.603535]],[[1. , 0.604755]],[[1. , 0.607492]],[[1. , 0.616418]],[[1. , 0.617218]],[[1. , 0.618286]],[[1. , 0.621523]],[[1. , 0.63782 ]],[[1. , 0.64827 ]],[[1. , 0.651362]],[[1. , 0.652565]],[[1. , 0.656662]],[[1. , 0.66351 ]],[[1. , 0.665016]],[[1. , 0.666109]],[[1. , 0.666886]],[[1. , 0.671279]],[[1. , 0.674722]],[[1. , 0.687006]],[[1. , 0.693007]],[[1. , 0.694221]],[[1. , 0.69477 ]],[[1. , 0.696081]],[[1. , 0.70218 ]],[[1. , 0.717733]],[[1. , 0.721544]],[[1. , 0.730546]],[[1. , 0.731209]],[[1. , 0.733165]],[[1. , 0.736021]],[[1. , 0.738336]],[[1. , 0.745797]],[[1. , 0.749442]],[[1. , 0.76966 ]],[[1. , 0.771184]],[[1. , 0.777175]],[[1. , 0.78088 ]],[[1. , 0.792101]],[[1. , 0.808177]],[[1. , 0.814811]],[[1. , 0.833735]],[[1. , 0.844027]],[[1. , 0.848142]],[[1. , 0.853428]],[[1. , 0.867082]],[[1. , 0.869778]],[[1. , 0.87003 ]],[[1. , 0.87248 ]],[[1. , 0.884739]],[[1. , 0.886555]],[[1. , 0.890192]],[[1. , 0.901144]],[[1. , 0.901225]],[[1. , 0.908969]],[[1. , 0.917064]],[[1. , 0.917886]],[[1. , 0.918769]],[[1. , 0.918916]],[[1. , 0.920461]],[[1. , 0.925577]],[[1. , 0.939368]],[[1. , 0.947054]],[[1. , 0.949123]],[[1. , 0.949655]],[[1. , 0.952523]],[[1. , 0.958178]],[[1. , 0.958476]],[[1. , 0.963924]],[[1. , 0.969788]],[[1. , 0.972673]],[[1. , 0.981083]],[[1. , 0.98139 ]],[[1. , 0.982243]],[[1. , 0.987911]],[[1. , 0.990947]],[[1. , 0.993888]],[[1. , 0.995731]]]) xSort=xMat[srtInd][:,0] xSort matrix([[1. , 0.014855],[1. , 0.015371],[1. , 0.033859],[1. , 0.038326],[1. , 0.040486],[1. , 0.045353],[1. , 0.047744],[1. , 0.058014],[1. , 0.066379],[1. , 0.067119],[1. , 0.067154],[1. , 0.067732],[1. , 0.068854],[1. , 0.070095],[1. , 0.070237],[1. , 0.073224],[1. , 0.085031],[1. , 0.106352],[1. , 0.110521],[1. , 0.115487],[1. , 0.116089],[1. , 0.116163],[1. , 0.116598],[1. , 0.124931],[1. , 0.126762],[1. , 0.132791],[1. , 0.133295],[1. , 0.138306],[1. , 0.147846],[1. , 0.148364],[1. , 0.15335 ],[1. , 0.156383],[1. , 0.156433],[1. , 0.162176],[1. , 0.166819],[1. , 0.171846],[1. , 0.175724],[1. , 0.178449],[1. , 0.181951],[1. , 0.182026],[1. , 0.189912],[1. , 0.19683 ],[1. , 0.199526],[1. , 0.201473],[1. , 0.204337],[1. , 0.212575],[1. , 0.218023],[1. , 0.223855],[1. , 0.230372],[1. , 0.231214],[1. , 0.235419],[1. , 0.236833],[1. , 0.247809],[1. , 0.251821],[1. , 0.253574],[1. , 0.254884],[1. , 0.255212],[1. , 0.257017],[1. , 0.263727],[1. , 0.27276 ],[1. , 0.274561],[1. , 0.277261],[1. , 0.278213],[1. , 0.279101],[1. , 0.286515],[1. , 0.295434],[1. , 0.297038],[1. , 0.307047],[1. , 0.307928],[1. , 0.315044],[1. , 0.333288],[1. , 0.334116],[1. , 0.33507 ],[1. , 0.335691],[1. , 0.335751],[1. , 0.336647],[1. , 0.33863 ],[1. , 0.344584],[1. , 0.357767],[1. , 0.358622],[1. , 0.359274],[1. , 0.365746],[1. , 0.378887],[1. , 0.381086],[1. , 0.385643],[1. , 0.387125],[1. , 0.394396],[1. , 0.395746],[1. , 0.398556],[1. , 0.409932],[1. , 0.41732 ],[1. , 0.426863],[1. , 0.427035],[1. , 0.42781 ],[1. , 0.429492],[1. , 0.434811],[1. , 0.441815],[1. , 0.445674],[1. , 0.452584],[1. , 0.466369],[1. , 0.466776],[1. , 0.470557],[1. , 0.480369],[1. , 0.482978],[1. , 0.493466],[1. , 0.493829],[1. , 0.495002],[1. , 0.503495],[1. , 0.509381],[1. , 0.526171],[1. , 0.52707 ],[1. , 0.528301],[1. , 0.53205 ],[1. , 0.541196],[1. , 0.549236],[1. , 0.550659],[1. , 0.552928],[1. , 0.556486],[1. , 0.567682],[1. , 0.569488],[1. , 0.583888],[1. , 0.590902],[1. , 0.599122],[1. , 0.603535],[1. , 0.604755],[1. , 0.607492],[1. , 0.616418],[1. , 0.617218],[1. , 0.618286],[1. , 0.621523],[1. , 0.63782 ],[1. , 0.64827 ],[1. , 0.651362],[1. , 0.652565],[1. , 0.656662],[1. , 0.66351 ],[1. , 0.665016],[1. , 0.666109],[1. , 0.666886],[1. , 0.671279],[1. , 0.674722],[1. , 0.687006],[1. , 0.693007],[1. , 0.694221],[1. , 0.69477 ],[1. , 0.696081],[1. , 0.70218 ],[1. , 0.717733],[1. , 0.721544],[1. , 0.730546],[1. , 0.731209],[1. , 0.733165],[1. , 0.736021],[1. , 0.738336],[1. , 0.745797],[1. , 0.749442],[1. , 0.76966 ],[1. , 0.771184],[1. , 0.777175],[1. , 0.78088 ],[1. , 0.792101],[1. , 0.808177],[1. , 0.814811],[1. , 0.833735],[1. , 0.844027],[1. , 0.848142],[1. , 0.853428],[1. , 0.867082],[1. , 0.869778],[1. , 0.87003 ],[1. , 0.87248 ],[1. , 0.884739],[1. , 0.886555],[1. , 0.890192],[1. , 0.901144],[1. , 0.901225],[1. , 0.908969],[1. , 0.917064],[1. , 0.917886],[1. , 0.918769],[1. , 0.918916],[1. , 0.920461],[1. , 0.925577],[1. , 0.939368],[1. , 0.947054],[1. , 0.949123],[1. , 0.949655],[1. , 0.952523],[1. , 0.958178],[1. , 0.958476],[1. , 0.963924],[1. , 0.969788],[1. , 0.972673],[1. , 0.981083],[1. , 0.98139 ],[1. , 0.982243],[1. , 0.987911],[1. , 0.990947],[1. , 0.993888],[1. , 0.995731]]) #計(jì)算不同k取值下的y估計(jì)值yHat ws1,yHat1 = LWLR(xMat,xMat,yMat,k=1.0) ws2,yHat2 = LWLR(xMat,xMat,yMat,k=0.01) ws3,yHat3 = LWLR(xMat,xMat,yMat,k=0.003) #創(chuàng)建畫布 fig = plt.figure(figsize=(6,8),dpi=100)#子圖1繪制k=1.0的曲線 fig1=fig.add_subplot(311) plt.scatter(xMat[:,1].A,yMat.A,c='b',s=2) plt.plot(xSort[:,1],yHat1[srtInd],linewidth=1,color='r') plt.title('局部加權(quán)回歸曲線，k=1.0',size=10,color='r')#子圖2繪制k=0.01的曲線 fig2=fig.add_subplot(312) plt.scatter(xMat[:,1].A,yMat.A,c='b',s=2) plt.plot(xSort[:,1],yHat2[srtInd],linewidth=1,color='r') plt.title('局部加權(quán)回歸曲線，k=0.01',size=10,color='r')#子圖3繪制k=0.003的曲線 fig3=fig.add_subplot(313) plt.scatter(xMat[:,1].A,yMat.A,c='b',s=2) plt.plot(xSort[:,1],yHat3[srtInd],linewidth=1,color='r') plt.title('局部加權(quán)回歸曲線，k=0.003',size=10,color='r')#調(diào)整子圖的間距 plt.tight_layout(pad=1.2) plt.show()

這三個(gè)圖是不同平滑值繪出的局部加權(quán)線性回歸結(jié)果。當(dāng)k=1.0時(shí)，模型的效果與最小二乘法差不多；k=0.01時(shí)，該模型基本上已經(jīng)挖出了數(shù)據(jù)的潛在規(guī)律，當(dāng)繼續(xù)減小到k=0.003時(shí)，會(huì)發(fā)現(xiàn)模型考慮了太多的噪音，進(jìn)而導(dǎo)致了過(guò)擬合現(xiàn)象。

#四種模型相關(guān)系數(shù)比較 np.corrcoef(yHat.T,yMat.T) # 最小二乘法 array([[1. , 0.98647356],[0.98647356, 1. ]]) np.corrcoef(yHat1,yMat.T) # k=1.0模型 array([[1. , 0.98647703],[0.98647703, 1. ]]) np.corrcoef(yHat2,yMat.T) # k=0.01模型 array([[1. , 0.9985249],[0.9985249, 1. ]]) np.corrcoef(yHat3,yMat.T) # k=0.003模型 array([[1. , 0.99931945],[0.99931945, 1. ]])

局部加權(quán)線性回歸也存在一個(gè)問(wèn)題——增加了計(jì)算量，因?yàn)樗鼘?duì)每個(gè)點(diǎn)預(yù)測(cè)都要使用整個(gè)數(shù)據(jù)集。從不同k值的結(jié)果圖中可以看出，當(dāng)k=0.01時(shí)模型可以很好地?cái)M合數(shù)據(jù)潛在規(guī)律，但是同時(shí)看一下，k值與權(quán)重關(guān)系圖，可以發(fā)現(xiàn)，當(dāng)k=0.01時(shí)，大部分?jǐn)?shù)據(jù)點(diǎn)的權(quán)重都接近0，也就是說(shuō)他們基本上可以不用帶入計(jì)算。所以如果一開始就能去掉這些數(shù)據(jù)點(diǎn)的計(jì)算，那么就可以大大減少程序的運(yùn)行時(shí)間了，從而緩解計(jì)算量增加帶來(lái)的問(wèn)題。后面我們會(huì)講解這個(gè)操作。

總結(jié)

以上是生活随笔為你收集整理的十、简单线性回归的python实现（详解）的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。