纯干货:Linux抓包命令集锦(tcpdump)
/******************************************************************************************
* 版權聲明
* 本文為本人原創,本人擁有此文的版權。鑒于本人持續受益于開源軟件社區,
* 本人聲明:任何個人及團體均可不受限制的轉載和復制本文,無論是否用于盈利
* 之目的,但不得修改文章內容,并必須在轉載及復制時同時保留本版權聲明,否
*??則為侵權行為,本人保留追究相應主體法律責任之權利。
* ?speng2005@gmail.com
* ? ?2016-1
******************************************************************************************/
????????大道至簡!
????????相信抓包是程序員,運維工程師,架構師,都必不可少的一項技能。但是能夠深入掌握好這門技藝的人,確實需要有開發,網絡,運維,架構等"跨界”背景才能比較好的發揮抓包神技的威力。本文是純干貨,重點不在于理論,更注重實戰技能,尤其注重對抓包數據的分析。本文中的命令追求的是使用最簡單,最普及的Linux系統自帶工具包實現各種抓包分析,具有盡可能廣泛的移植性和可用性。文中給出的命令均在Centos 6.3,tcpdump 4.1版本下測試可用;其他平臺及環境,可能需要你自己微調部分命令及腳本才可以運行。文中多數命令及腳本都嚴重依賴于tcpdump命令輸出文本數據格式,微調代碼時應格外注意這一點。注意,本文中的命令適用于一般的基于tcp連接的,請求響應模型的網絡服務,但不適用于使用pipeline模式的網絡服務。如果想理解本文命令思路的話,需要你熟悉tcp/ip協議,網絡osi模型,常見網絡通訊協議,socket編程,linux腳本編程,awk腳本編程,數據挖掘思維方式等知識,不足者請自行腦補。理解本文的思路后,還可以在這些命令基礎上有許多種靈活搭配和變種,請自行研究。不想費腦細胞的,運氣好的話,很多命令都可以直接使用。盡管這里講的是Linux抓包,但如果使用流行的wireshark在Windows上抓包后保存成tcpdump格式文件,然后上傳到Linux系統上照樣可以使用本文中的命令進行分析。
1.盲抓
????????盲抓就是瞎抓,尤其在你接觸到一臺陌生機器,感覺有點抓瞎的時候,應當使用本節中的方法,其目的是找出這臺機器上的重點網絡服務是哪些ip和端口。如果你明確的知道要抓的包是哪個ip,哪個端口,可以直接跳至下一小節。
在目標機器上抓取任意tcp數據(需要root權限):
tcpdump -i any -nn tcp > 123.pkg.head.txt
分析目標機器上熱點ip:port(按數據包數量統計):
分析目標機器上熱點ip:port(按字節數,即帶寬統計):
cat 123.pkg.head.txt | awk 'BEGIN{header_len=0x36;inp=0;inb=0;inbb=0;outp=0;outb=0;outbb=0;start="";end=""} function getBodyLen(){if("length"==$(NF-1)) return $NF;t_c=split($7,a,/[()]/);if(3==t_c)return a[2];else return 0;} NF>5 && $2=="IP" {len=header_len + getBodyLen();print $3,"[out]", len; print substr($5,1,length($5)-1), "[in]", len; }' | sort | awk 'BEGIN{ipport="";dir="";bc=0} {if($1!=ipport || $2!=dir){if(bc>0)print ipport,dir,bc;ipport=$1;dir=$2;bc=$3;}else{bc+=$3;}} END{if(bc>0)print ipport,dir,bc;}' | awk '{buf[NR]=$0;sum+=$3} END{for(i=1;i<=NR;i++) print buf[i], sum}' | awk '{printf("%s %s %s %.2f %%\n",$1, $2, $3, 100*$3/$4)}' | sort -nr -k 4 | less 示例: 10.71.28.21.80 [out] 2559403 42.46 % 218.92.220.60.39147 [in] 417277 6.92 % 10.71.28.21.80 [in] 306507 5.08 % 101.26.37.38.39906 [in] 271432 4.50 % 122.225.28.115.48239 [in] 170901 2.84 % 218.92.220.56.60503 [in] 151630 2.52 % 218.92.220.62.59711 [in] 151322 2.51 % 218.92.220.56.25777 [in] 138038 2.29 % 60.210.23.239.7490 [in] 131700 2.18 % 10.71.28.20.8080 [out] 121366 2.01 %分析數據流量概要信息(調整ipportRegex的值來控制統計口徑):
cat 123.pkg.head.txt | awk 'BEGIN{ipportRegex="10.71.28.21.80";header_len=0x36;inp=0;inb=0;inbb=0;outp=0;outb=0;outbb=0;start="";end=""} function getBodyLen(){if("length"==$(NF-1)) return $NF;t_c=split($7,a,/[()]/);if(3==t_c)return a[2];else return 0;} $3~ipportRegex{outp++;bl=getBodyLen();outb+=header_len+bl;outbb+=bl;if(""==start)start=$1;end=$1;} $5~ipportRegex{inp++;bl=getBodyLen();inb+=header_len+bl;inbb+=bl;if(""==start)start=$1;end=$1;} function timeSub(s,e){split(substr(s,1,8),sA1,":");s1=sA1[1]*3600+sA1[2]*60+sA1[3];us1=substr(s,10);split(substr(e,1,8),sA2,":");s2=sA2[1]*3600+sA2[2]*60+sA2[3];us2=substr(e,10);return s2*1000000+us2-s1*1000000-us1;} END{second=timeSub(start,end)/1000000;if(second<1) second=1; printf("total time: %.1f seconds\n", second);printf("[In] pkg count: %s, pkg rate: %.2f pkg/s, bytes count: %s, bytes rate: %.2f KB/s, body bytes: %s, body bytes rate: %.2f KB/s, payload percent: %.1f%%\n", inp,inp/second,inb,inb/second/1024,inbb,inbb/second/1024,100*inbb/inb);printf("[Out] pkg count: %s, pkg rate: %.2f pkg/s, bytes count: %s, bytes rate: %.2f KB/s, body bytes: %s, body bytes rate: %.2f KB/s, payload percent: %.1f%%\n", outp,outp/second,outb,outb/second/1024,outbb,outbb/second/1024,100*outbb/outb);}' cat 123.pkg.head.txt | awk 'BEGIN{ipportRegex="10.71.28.21.";header_len=0x36;inp=0;inb=0;inbb=0;outp=0;outb=0;outbb=0;start="";end=""} function getBodyLen(){if("length"==$(NF-1)) return $NF;t_c=split($7,a,/[()]/);if(3==t_c)return a[2];else return 0;} $3~ipportRegex{outp++;bl=getBodyLen();outb+=header_len+bl;outbb+=bl;if(""==start)start=$1;end=$1;} $5~ipportRegex{inp++;bl=getBodyLen();inb+=header_len+bl;inbb+=bl;if(""==start)start=$1;end=$1;} function timeSub(s,e){split(substr(s,1,8),sA1,":");s1=sA1[1]*3600+sA1[2]*60+sA1[3];us1=substr(s,10);split(substr(e,1,8),sA2,":");s2=sA2[1]*3600+sA2[2]*60+sA2[3];us2=substr(e,10);return s2*1000000+us2-s1*1000000-us1;} END{second=timeSub(start,end)/1000000;if(second<1) second=1; printf("total time: %.1f seconds\n", second);printf("[In] pkg count: %s, pkg rate: %.2f pkg/s, bytes count: %s, bytes rate: %.2f KB/s, body bytes: %s, body bytes rate: %.2f KB/s, payload percent: %.1f%%\n", inp,inp/second,inb,inb/second/1024,inbb,inbb/second/1024,100*inbb/inb);printf("[Out] pkg count: %s, pkg rate: %.2f pkg/s, bytes count: %s, bytes rate: %.2f KB/s, body bytes: %s, body bytes rate: %.2f KB/s, payload percent: %.1f%%\n", outp,outp/second,outb,outb/second/1024,outbb,outbb/second/1024,100*outbb/outb);}' cat 123.pkg.head.txt | awk 'BEGIN{ipportRegex="10.";header_len=0x36;inp=0;inb=0;inbb=0;outp=0;outb=0;outbb=0;start="";end=""} function getBodyLen(){if("length"==$(NF-1)) return $NF;t_c=split($7,a,/[()]/);if(3==t_c)return a[2];else return 0;} $3~ipportRegex{outp++;bl=getBodyLen();outb+=header_len+bl;outbb+=bl;if(""==start)start=$1;end=$1;} $5~ipportRegex{inp++;bl=getBodyLen();inb+=header_len+bl;inbb+=bl;if(""==start)start=$1;end=$1;} function timeSub(s,e){split(substr(s,1,8),sA1,":");s1=sA1[1]*3600+sA1[2]*60+sA1[3];us1=substr(s,10);split(substr(e,1,8),sA2,":");s2=sA2[1]*3600+sA2[2]*60+sA2[3];us2=substr(e,10);return s2*1000000+us2-s1*1000000-us1;} END{second=timeSub(start,end)/1000000;if(second<1) second=1; printf("total time: %.1f seconds\n", second);printf("[In] pkg count: %s, pkg rate: %.2f pkg/s, bytes count: %s, bytes rate: %.2f KB/s, body bytes: %s, body bytes rate: %.2f KB/s, payload percent: %.1f%%\n", inp,inp/second,inb,inb/second/1024,inbb,inbb/second/1024,100*inbb/inb);printf("[Out] pkg count: %s, pkg rate: %.2f pkg/s, bytes count: %s, bytes rate: %.2f KB/s, body bytes: %s, body bytes rate: %.2f KB/s, payload percent: %.1f%%\n", outp,outp/second,outb,outb/second/1024,outbb,outbb/second/1024,100*outbb/outb);}' 示例: total time: 45.7 seconds [In] pkg count: 2359, pkg rate: 51.59 pkg/s, bytes count: 306507, bytes rate: 6.55 KB/s, body bytes: 179121, body bytes rate: 3.83 KB/s, payload percent: 58.4% [Out] pkg count: 2057, pkg rate: 44.99 pkg/s, bytes count: 2559403, bytes rate: 54.66 KB/s, body bytes: 2448325, body bytes rate: 52.29 KB/s, payload percent: 95.7%2.抓取源數據
??????? 明確了要抓取的目標服務的ip和端口后,就要開始正式抓包了,這些抓包數據是后續分析工作的基礎。當然,你可能在抓包之前還要做一些其他工作,以保證抓包時的工作場景就是你想要分析的目標場景。抓包時你需要root權限。
抓取不含包體的包:
tcpdump -nn port 80 > 123.pkg.txt抓取含有包體的包并進行初步解包:
tcpdump port 80 -w 123.pkg -s 120 tcpdump port 80 -w 123.pkg -s 0 tcpdump -r 123.pkg -XXnn > 123.pkg.txt 示例: 11:12:01.894222 IP 10.64.12.14.59493 > 10.70.60.56.1521: Flags [P.], seq 3862499245:3862499262, ack 627056474, win 501, options [nop,nop,TS val 2916000715 ecr 736856730], length 170x0000: 0000 0c07 ace9 0022 195d 2445 0800 4500 .......".]$E..E.0x0010: 0045 cb3b 4000 4006 12ac 0a40 0c0e 0a46 .E.;@.@....@...F0x0020: 3c38 e865 05f1 e639 0fad 2560 1f5a 8018 <8.e...9..%`.Z..0x0030: 01f5 5d03 0000 0101 080a adce a3cb 2beb ..]...........+.0x0040: 8a9a 0011 0000 0600 0000 0000 0305 0001 ................0x0050: 0301 0a ... 11:12:01.894226 IP 10.64.12.14.59514 > 10.70.60.56.1521: Flags [P.], seq 3903163064:3903163580, ack 4171729186, win 501, options [nop,nop,TS val 2916000715 ecr 736856729], length 5160x0000: 0000 0c07 ace9 0022 195d 2445 0800 4500 .......".]$E..E.0x0010: 0238 f2e9 4000 4006 e90a 0a40 0c0e 0a46 .8..@.@....@...F0x0020: 3c38 e87a 05f1 e8a5 8ab8 f8a7 8922 8018 <8.z........."..0x0030: 01f5 5ef6 0000 0101 080a adce a3cb 2beb ..^...........+.0x0040: 8a99 0204 0000 0600 0000 0000 1169 0001 .............i..3.過濾
?????? 有時候,需要分析的問題是包含在抓取目標服務的全部數據包中的一小部分,這個時候就要對包數據進行過濾。當然也可以在上一節進行抓取時直接進行過濾,但這里主要滿足事后的過濾需求。
過濾掉含有包體的數據文件,只保留數據包頭:
cat 123.pkg.txt | awk 'substr($1,1,3)!="0x0"' > 123.pkg.head.txt 示例: 11:12:01.894222 IP 10.64.12.14.59493 > 10.70.60.56.1521: Flags [P.], seq 3862499245:3862499262, ack 627056474, win 501, options [nop,nop,TS val 2916000715 ecr 736856730], length 17 11:12:01.894226 IP 10.64.12.14.59514 > 10.70.60.56.1521: Flags [P.], seq 3903163064:3903163580, ack 4171729186, win 501, options [nop,nop,TS val 2916000715 ecr 736856729], length 516 11:12:01.894228 IP 10.64.12.14.59518 > 10.70.60.56.1521: Flags [P.], seq 3897659932:3897660175, ack 3171500152, win 501, options [nop,nop,TS val 2916000715 ecr 736856729], length 243 11:12:01.894231 IP 10.64.12.14.59512 > 10.70.60.56.1521: Flags [P.], seq 3898476274:3898476788, ack 1748466768, win 501, options [nop,nop,TS val 2916000715 ecr 736856730], length 514按時間段過濾含有包體的數據文件(調整start和end的值以改變時間范圍):
cat 123.pkg.txt | awk 'BEGIN {start="11:13:49.0";end="11:13:49.9";flag=0} {if(substr($1,1,3)=="0x0"){if(1==flag){print $0}}else{cur=substr($1,1,length(start));if(cur>=start && cur<=end){print $0;flag=1}else{flag=0}}}' > new123.pkg.txt按ip端口過濾含有包體的數據文件(調整ip和port的值):
cat new123.pkg.txt | awk 'BEGIN {ip="192.168.122.180";port="54365";flag=0} {if(substr($1,1,3)=="0x0"){if(1==flag){print $0}}else if($0~ip"."port){print $0;flag=1;}else{flag=0;}}' > newnew123.pkg.txt按文本關鍵字過濾含有包體的數據文件(調整keyword的值):
cat newnew123.pkg.txt | awk 'BEGIN{keyword="id_10bc3c9";buf="";found=0} {if(substr($1,1,2)=="0x"){if($0~keyword) found=1;buf=buf"\n"$0}else{if(1==found){print buf;found=0;}if($0~keyword) found=1;buf=$0}} END{if(1==found) print buf}' > tmp.pkg.txt按二進制串關鍵字從二進制數據包中過濾含有包體的數據文件(性能稍低,對于大文件耐心等待;例子中是過濾包含"test_dava_11968@163.com"字符串的數據包):
cat newnew123.pkg.txt | awk 'BEGIN{keyword="746573745f646176615f3131393638403136332e636f6d";bin="";buf=""} {if(substr($1,1,2)=="0x"){for(i=2;i<NF;i++) bin=bin""$i;buf=buf"\n"$0}else{if(length(bin)>0&&bin~keyword) print buf;buf=$0;bin=""}} END{if(length(bin)>0&&bin~keyword) print buf}' > tmp2.pkg.txt4.分類
?????? 通過分類,將抓包文件中混雜在一起的多個客戶端的數據包拆分成各自單獨的文件,以利于人工查看以及后續分析。新生成的文件放在當前目錄下的子目錄中,例如10.64.12.14.443_detail。
按client端ip端口將混雜在一起的含有包體的包拆分的單獨文件:
cat 123.pkg.txt | awk 'BEGIN{server="10.70.60.56.1521";buf="";client="";dir=server"_detail";system("mkdir "dir" 2>/dev/null")} {if(substr($1,1,2)=="0x"){buf=buf"\n"$0}else{if(length(client)>0) print buf>>dir"/"client".txt";if($3==server){client=substr($5,1,length($5)-1);}else if($5~server){client=$3}else{client=""} buf=$0}}'按client端ip端口將混雜在一起的含有包體的包拆分的單獨文件(超過1000個client后因受同時打開文件數限制awk性能非常低,可增加dd命令的bs緩沖區大小,并使用limit參數進行限制):
dd if=123.pkg.txt bs=100M 2>/dev/null | awk 'BEGIN{server="10.70.60.56.1521";limit=1000;buf="";client="";count=0;dir=server"_detail";system("mkdir "dir" 2>/dev/null")} {if(substr($1,1,2)=="0x"){if(length(client)>0) buf=buf"\n"$0}else{if(length(client)>0) print buf>>dir"/"client".txt";if($3==server){client=substr($5,1,length($5)-1)}else if($5~server){client=$3}else{client=""}if(length(client)>0){if("ok"==all_clients[client]){buf=$0}else if(count<limit){count++;all_clients[client]="ok";buf=$0}else{client=""}}}}'5.交互分析
???????前面的分析處理僅限于數據包本身的ip和port,而本節的交互分析則將數據包歸類到一個個具體的tcp連接之中,提取有關的交互“成本”數據,并嘗試分析數據包在請求響應模型中的交互語義。本節中的命令需要系列awk腳本支持,為不影響理解本節主要內容,相關腳本源碼附錄在文章末尾供參考,其中包含主腳本parseTcp.awk,以及若干插件腳本:fixLenParser.awk,lineParser.awk,oracleSqlParser.awk,httpParser.awk。
分析單個不含包體的數據文件的交互過程:
cat 123.pkg.txt | /data/speng/tcpdump/parseTcp.awk 61.135.169.125.80 > interaction.txt
分析單個不含包體的數據文件中只與某特定客戶端有關的交互過程:
分析多個分類后含有包體的數據文件的交互過程(cd進入上一節分類后所生成的目錄下執行命令):
ls | xargs -n 10 awk 'BEGIN{f=""} {if(FILENAME!=f){print "newfile",FILENAME;f=FILENAME}if(substr($1,1,2)!="0x") print $0}' | ../parseTcp.awk 10.70.60.56.1521 > ../interaction.txt分析多個分類后含有包體的數據文件的交互過程(cd進入分類后文件所在目錄下執行命令。支持識別包體中的特征數據,并打印出來便于后續業務分析。可編寫類似oracleSqlParser.awk或httpParser.awk插件腳本進行新協議或業務擴展。源碼附錄在文章末尾):
ls | xargs -n 10 awk -f ../oracleSqlParser.awk --source 'function getLen(){if("length"==$(NF-1)) return $NF;t_c=split($7,a,/[()]/);if(3==t_c) return a[2];else return 0;} BEGIN{f="";h="";b=0;} {if(FILENAME!=f){if(1==b)parserEnd();if(""!=h)print h;print "newfile",FILENAME;f=FILENAME;h="";b=0;}if(substr($1,1,2)=="0x"){if(1==b)parserWork();}else{if(1==b)parserEnd();b=0;if(""!=h)print h;h=$0;bl=getLen();if(bl>0){parserInit(bl);b=1;}}} END{if(1==b)parserEnd();if(""!=h)print h;}' | ../parseTcp.awk 10.70.60.56.1521 > ../interaction.txt ls | xargs -n 10 awk -f ../httpParser.awk --source 'function getLen(){if("length"==$(NF-1)) return $NF;t_c=split($7,a,/[()]/);if(3==t_c) return a[2];else return 0;} BEGIN{f="";h="";b=0;} {if(FILENAME!=f){if(1==b)parserEnd();if(""!=h)print h;print "newfile",FILENAME;f=FILENAME;h="";b=0;}if(substr($1,1,2)=="0x"){if(1==b)parserWork();}else{if(1==b)parserEnd();b=0;if(""!=h)print h;h=$0;bl=getLen();if(bl>0){parserInit(bl);b=1;}}} END{if(1==b)parserEnd();if(""!=h)print h;}' | ../parseTcp.awk 10.70.66.120.80 > ../interaction.txt 示例: sql: 11:12:29.422380 client 10.64.12.14.59573_3961647293 connect take(us): 567 slient_before(us): 0 result: ok 11:12:29.423124 client 10.64.12.14.59573_3961647293 round-trip 1 slient_before(us): 177 total take(us): 54042 req_time(us): 0 req_bytes: 211 server_process(us): 54042 response_time(us): 0 response_bytes: 8 11:12:29.478165 client 10.64.12.14.59573_3961647293 round-trip 2 slient_before(us): 999 total take(us): 2021 req_time(us): 1409 req_bytes: 363 server_process(us): 612 response_time(us): 0 response_bytes: 127 11:12:29.480257 client 10.64.12.14.59573_3961647293 round-trip 3 slient_before(us): 71 total take(us): 1420 req_time(us): 0 req_bytes: 33 server_process(us): 1420 response_time(us): 0 response_bytes: 188 11:12:29.481825 client 10.64.12.14.59573_3961647293 round-trip 4 slient_before(us): 148 total take(us): 1036 req_time(us): 0 req_bytes: 779 server_process(us): 1036 response_time(us): 0 response_bytes: 834 11:12:29.483414 client 10.64.12.14.59573_3961647293 round-trip 5 slient_before(us): 553 total take(us): 3983 req_time(us): 810 req_bytes: 184 server_process(us): 3173 response_time(us): 0 response_bytes: 73 11:12:29.487836 client 10.64.12.14.59573_3961647293 round-trip 6 slient_before(us): 439 total take(us): 4891 req_time(us): 0 req_bytes: 574 server_process(us): 4891 response_time(us): 0 response_bytes: 1180 11:12:29.492988 client 10.64.12.14.59573_3961647293 round-trip 7 slient_before(us): 261 total take(us): 3220 req_time(us): 0 req_bytes: 505 server_process(us): 3220 response_time(us): 0 response_bytes: 358 11:12:29.496568 client 10.64.12.14.59573_3961647293 round-trip 8 slient_before(us): 360 total take(us): 2276 req_time(us): 0 req_bytes: 245 server_process(us): 2276 response_time(us): 0 response_bytes: 393 req_appflag: select.usrid,mailadd,mailflg,mobile,verifyflg,user_seq_id,is_oauth_type,lastlogin,pwd_strict.from.tab_users.where.mailadd.=:1--con_id_52d2c4 11:12:29.499283 client 10.64.12.14.59573_3961647293 round-trip 9 slient_before(us): 439 total take(us): 704 req_time(us): 0 req_bytes: 17 server_process(us): 704 response_time(us): 0 response_bytes: 208 11:12:29.500267 client 10.64.12.14.59573_3961647293 round-trip 10 slient_before(us): 280 total take(us): 766 req_time(us): 0 req_bytes: 245 server_process(us): 766 response_time(us): 0 response_bytes: 393 req_appflag: select.usrid,mailadd,mailflg,mobile,verifyflg,user_seq_id,is_oauth_type,lastlogin,pwd_strict.from.tab_users.where.mailadd.=:1--con_id_52d2c4 11:12:29.501945 client 10.64.12.14.59573_3961647293 round-trip 11 slient_before(us): 912 total take(us): 681 req_time(us): 0 req_bytes: 17 server_process(us): 681 response_time(us): 0 response_bytes: 208 11:12:29.503286 client 10.64.12.14.59573_3961647293 round-trip 12 slient_before(us): 660 total take(us): 4956 req_time(us): 0 req_bytes: 513 server_process(us): 4956 response_time(us): 0 response_bytes: 64 req_appflag: update.tab_users.tu.set.tu.lastlogin_third=tu.lastlogin,tu.lastlogin=to_date(:1,'yyyy-mm-dd.hh24:mi:ss'),tu.uct_time=to_date(:2,'yyyy-mm-dd.hh24:mi:ss'),tu.uct=:3,tu.auto_login_flag=:4,login_ip=:5.where.mailadd.=:6--con_id_52d2c4 11:12:29.510362 client 10.64.12.14.59573_3961647293 round-trip 13 slient_before(us): 2120 total take(us): 3591 req_time(us): 0 req_bytes: 515 server_process(us): 3591 response_time(us): 0 response_bytes: 63 req_appflag: update.tab_users.tu.set.tu.lastlogin_third=tu.lastlogin,tu.lastlogin=to_date(:1,'yyyy-mm-dd.hh24:mi:ss'),tu.uct_time=to_date(:2,'yyyy-mm-dd.hh24:mi:ss'),tu.uct=:3,tu.auto_login_flag=:4,login_ip=:5.where.mailadd.=:6--con_id_52d2c4 11:12:29.515085 client 10.64.12.14.59573_3961647293 round-trip 14 slient_before(us): 1132 total take(us): 3660 req_time(us): 0 req_bytes: 513 server_process(us): 3660 response_time(us): 0 response_bytes: 63 req_appflag: update.tab_users.tu.set.tu.lastlogin_third=tu.lastlogin,tu.lastlogin=to_date(:1,'yyyy-mm-dd.hh24:mi:ss'),tu.uct_time=to_date(:2,'yyyy-mm-dd.hh24:mi:ss'),tu.uct=:3,tu.auto_login_flag=:4,login_ip=:5.where.mailadd.=:6--con_id_52d2c4 11:12:29.518969 client 10.64.12.14.59573_3961647293 round-trip 15 slient_before(us): 224 total take(us): 724 req_time(us): 0 req_bytes: 245 server_process(us): 724 response_time(us): 0 response_bytes: 392 req_appflag: select.usrid,mailadd,mailflg,mobile,verifyflg,user_seq_id,is_oauth_type,lastlogin,pwd_strict.from.tab_users.where.mailadd.=:1--con_id_52d2c4 11:12:29.519737 client 10.64.12.14.59573_3961647293 round-trip 16 slient_before(us): 44 total take(us): 664 req_time(us): 0 req_bytes: 17 server_process(us): 664 response_time(us): 0 response_bytes: 208 11:12:29.522030 client 10.64.12.14.59573_3961647293 round-trip 17 slient_before(us): 1629 total take(us): 4037 req_time(us): 0 req_bytes: 513 server_process(us): 4037 response_time(us): 0 response_bytes: 63 req_appflag: update.tab_users.tu.set.tu.lastlogin_third=tu.lastlogin,tu.lastlogin=to_date(:1,'yyyy-mm-dd.hh24:mi:ss'),tu.uct_time=to_date(:2,'yyyy-mm-dd.hh24:mi:ss'),tu.uct=:3,tu.auto_login_flag=:4,login_ip=:5.where.mailadd.=:6--con_id_52d2c4 11:12:29.526610 client 10.64.12.14.59573_3961647293 round-trip 18 slient_before(us): 543 total take(us): 709 req_time(us): 0 req_bytes: 245 server_process(us): 709 response_time(us): 0 response_bytes: 392 req_appflag: select.usrid,mailadd,mailflg,mobile,verifyflg,user_seq_id,is_oauth_type,lastlogin,pwd_strict.from.tab_users.where.mailadd.=:1--con_id_52d2c4 11:12:29.527617 client 10.64.12.14.59573_3961647293 round-trip 19 slient_before(us): 298 total take(us): 1490 req_time(us): 0 req_bytes: 17 server_process(us): 1490 response_time(us): 0 response_bytes: 208 http: 14:00:53.123517 client 172.20.154.45.3752_2455096098 connect take(us): 3291 slient_before(us): 72266270 result: ok 14:00:53.132710 client 172.20.154.45.3752_2455096098 round-trip 1 slient_before(us): 5902 total take(us): 5927 req_time(us): 76 req_bytes: 1635 server_process(us): 5402 response_time(us): 449 response_bytes: 6587 req_appflag: GET./static/api/js/share.js?v=89860593.js?cdnversion=403517 response_appflag: 200.OK 14:00:53.263446 client 172.20.154.45.3752_2455096098 round-trip 2 slient_before(us): 124809 total take(us): 17767 req_time(us): 88 req_bytes: 1620 server_process(us): 17679 response_time(us): 0 response_bytes: 946 req_appflag: GET./static/js/shell_v2.js?cdnversion=403519 response_appflag: 200.OK 14:00:53.472286 client 172.20.154.45.3752_2455096098 round-trip 3 slient_before(us): 191073 total take(us): 9319 req_time(us): 53 req_bytes: 1620 server_process(us): 8517 response_time(us): 749 response_bytes: 10361 req_appflag: GET./static/js/bds_s_v2.js?cdnversion=403519 response_appflag: 200.OK 14:00:54.587117 client 172.20.154.45.3752_2455096098 round-trip 4 slient_before(us): 1105512 total take(us): 13233 req_time(us): 92 req_bytes: 1618 server_process(us): 13113 response_time(us): 28 response_bytes: 2777 req_appflag: GET./static/js/logger.js?cdnversion=403519 response_appflag: 200.OK 14:01:14.598513 client 172.20.154.45.3752_2455096098 close take(us): 589 slient_before(us): 19998163 close_bytes: 0 direction: server->client result: full_closehttps: 16:53:29.658744 client 10.64.12.16.32769_169947140 connect take(us): 89 slient_before(us): 0 result: ok 16:53:29.658857 client 10.64.12.16.32769_169947140 round-trip 1 slient_before(us): 24 total take(us): 6865 req_time(us): 0 req_bytes: 117 server_process(us): 1535 response_time(us): 5330 response_bytes: 5140 16:53:29.670379 client 10.64.12.16.32769_169947140 round-trip 2 slient_before(us): 4657 total take(us): 6071 req_time(us): 0 req_bytes: 198 server_process(us): 6071 response_time(us): 0 response_bytes: 59 16:53:29.676530 client 10.64.12.16.32769_169947140 round-trip 3 slient_before(us): 80 total take(us): 983303 req_time(us): 0 req_bytes: 133 server_process(us): 983303 response_time(us): 0 response_bytes: 842 16:53:30.659883 client 10.64.12.16.32769_169947140 close take(us): 4412 slient_before(us): 50 close_bytes: 37 direction: server->client result: full_close 16:54:39.540752 client 10.64.12.16.32769_239829149 connect take(us): 91 slient_before(us): 68876457 result: ok 16:54:39.540866 client 10.64.12.16.32769_239829149 round-trip 1 slient_before(us): 23 total take(us): 7459 req_time(us): 0 req_bytes: 117 server_process(us): 2127 response_time(us): 5332 response_bytes: 5140 16:54:39.552971 client 10.64.12.16.32769_239829149 round-trip 2 slient_before(us): 4646 total take(us): 5452 req_time(us): 0 req_bytes: 198 server_process(us): 5452 response_time(us): 0 response_bytes: 59 16:54:39.558487 client 10.64.12.16.32769_239829149 round-trip 3 slient_before(us): 64 total take(us): 965470 req_time(us): 0 req_bytes: 133 server_process(us): 965470 response_time(us): 0 response_bytes: 842 16:54:40.524004 client 10.64.12.16.32769_239829149 close take(us): 115 slient_before(us): 47 close_bytes: 37 direction: server->client result: full_close6.統計分析
????????統計分析用于掌握一批數據包反映的整體交互規律和指標數據。配合使用上一節交互分析中獲得的業務關鍵字過濾,還可以分析具有某些特定特征的交互指標數據。
分析多行模式輸出的交互過程的請求響應對的平均響應時間(含請求響應的網絡傳輸時間,但不含socket建立階段):
?
分析多行模式輸出的交互過程的請求響應對的服務器平均處理時間:
cat interaction.txt | grep round-trip | awk '{sum+=($16/1000000)} END {print "Lines = "NR", Sum = "sum"(seconds), Average = "sum/NR"(seconds)"}' 示例: Lines = 678796, Sum = 1612.32(seconds), Average = 0.00237526(seconds)分析多行模式輸出的交互過程的請求響應對的平均請求大小:
cat interaction.txt | grep round-trip | awk '{sum+=$14} END {print "Lines = "NR", Sum = "sum"(bytes), Average = "sum/NR"(bytes)"}'分析多行模式輸出的交互過程的請求響應對的平均響應大小:
cat interaction.txt | grep round-trip | awk '{sum+=$20} END {print "Lines = "NR", Sum = "sum"(bytes), Average = "sum/NR"(bytes)"}'分析某多行模式輸出的https短連接交互過程的ssl握手階段的平均處理時間:
cat interaction.txt | grep round-trip | awk '$5==1 {start=$1;r1_time=$10} $5==2 {slient=$7;r2_time=$10;print start, r1_time+slient+r2_time}' | awk '{sum+=($2/1000000)} END {print "Lines = "NR", Sum = "sum"(seconds), Average = "sum/NR"(seconds)"}' 示例: Lines = 284861, Sum = 12216.7(seconds), Average = 0.0428866(seconds)分析某多行模式輸出的https短連接交互過程的ssl握手階段的tps及每秒平均處理時間(可導入excel作圖):
cat interaction.txt | grep round-trip | awk '$5==1 {start=$1;r1_time=$10} $5==2 {slient=$7;r2_time=$10;print start, r1_time+slient+r2_time}' | sort | awk '{print $2,$1}' | awk -F '.' '{print $1}' | awk 'BEGIN{sum=0;count=0;time=""} {if($2!=time){if(count>0) print time,count,sum/count/1000000;sum=$1;count=1;time=$2}else{sum+=$1;count++}} END{if(count>0) print time,count,sum/count/1000000}'分析某多行模式輸出的https短連接交互過程的socket建立完成至服務器accept此連接經歷的平均等待時間(可導入excel作圖):
cat interaction.txt | grep "round-trip 1 " | awk '{print $1,$7}' | sort | awk '{print $2,$1}' | awk -F '.' '{print $1}' | awk 'BEGIN{sum=0;count=0;time=""} {if($2!=time){if(count>0) printf("%s %s %.6f\n", time,count,sum/count/1000000);sum=$1;count=1;time=$2}else{sum+=$1;count++}} END{if(count>0) printf("%s %s %.6f\n", time,count,sum/count/1000000)}'分析某多行模式輸出含有包體特征數據的oracle長連接交互過程中select語句及update語句的平均響應時間:
cat interaction.txt | grep select | awk '{sum+=($10/1000000)} END {print "Lines = "NR", Sum = "sum"(seconds), Average = "sum/NR"(seconds)"}' cat interaction.txt | grep update | awk '{sum+=($10/1000000)} END {print "Lines = "NR", Sum = "sum"(seconds), Average = "sum/NR"(seconds)"}'?
?
?
?
?
?
總結
以上是生活随笔為你收集整理的纯干货:Linux抓包命令集锦(tcpdump)的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: (六)深度Q网络
- 下一篇: 分布式网络游戏百万人同时在线服务器架构实