本文中介紹前幾個序列函數,NTILE,ROW_NUMBER,RANK,DENSE_RANK,下面會一一解釋各自的用途。 Hive版本為 apache-hive-0.13.1 數據準備:
cookie1,2015-04-10,1cookie1,2015-04-11,5cookie1,2015-04-12,7cookie1,2015-04-13,3cookie1,2015-04-14,2cookie1,2015-04-15,4cookie1,2015-04-16,4cookie2,2015-04-10,2cookie2,2015-04-11,3cookie2,2015-04-12,5cookie2,2015-04-13,6cookie2,2015-04-14,3cookie2,2015-04-15,9cookie2,2015-04-16,7CREATE EXTERNAL TABLE lxw1234 (cookieid string,createtime string, --daypv INT) ROW FORMAT DELIMITEDFIELDS TERMINATED BY ','stored as textfile location '/tmp/lxw11/';DESC lxw1234;cookieid STRINGcreatetime STRINGpv INThive> select * from lxw1234;OKcookie1 2015-04-10 1cookie1 2015-04-11 5cookie1 2015-04-12 7cookie1 2015-04-13 3cookie1 2015-04-14 2cookie1 2015-04-15 4cookie1 2015-04-16 4cookie2 2015-04-10 2cookie2 2015-04-11 3cookie2 2015-04-12 5cookie2 2015-04-13 6cookie2 2015-04-14 3cookie2 2015-04-15 9cookie2 2015-04-16 7
NTILE NTILE(n),用于將分組數據按照順序切分成n片,返回當前切片值 NTILE不支持ROWS BETWEEN,比如 NTILE(2) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3 PRECEDING AND CURRENT ROW) 如果切片不均勻,默認增加第一個切片的分布
SELECTcookieid,createtime,pv,NTILE(2) OVER(PARTITION BY cookieid ORDER BY createtime) AS rn1, --分組內將數據分成2片NTILE(3) OVER(PARTITION BY cookieid ORDER BY createtime) AS rn2, --分組內將數據分成3片NTILE(4) OVER(ORDER BY createtime) AS rn3 --將所有數據分成4片FROM lxw1234ORDER BY cookieid,createtime;cookieid day pv rn1 rn2 rn3-------------------------------------------------cookie1 2015-04-10 1 1 1 1cookie1 2015-04-11 5 1 1 1cookie1 2015-04-12 7 1 1 2cookie1 2015-04-13 3 1 2 2cookie1 2015-04-14 2 2 2 3cookie1 2015-04-15 4 2 3 3cookie1 2015-04-16 4 2 3 4cookie2 2015-04-10 2 1 1 1cookie2 2015-04-11 3 1 1 1cookie2 2015-04-12 5 1 1 2cookie2 2015-04-13 6 1 2 2cookie2 2015-04-14 3 2 2 3cookie2 2015-04-15 9 2 3 4cookie2 2015-04-16 7 2 3 4
?比如,統計一個cookie,pv數最多的前1/3的天
SELECTcookieid,createtime,pv,NTILE(3) OVER(PARTITION BY cookieid ORDER BY pv DESC) AS rnFROM lxw1234;--rn = 1 的記錄,就是我們想要的結果cookieid day pv rn----------------------------------cookie1 2015-04-12 7 1cookie1 2015-04-11 5 1cookie1 2015-04-15 4 1cookie1 2015-04-16 4 2cookie1 2015-04-13 3 2cookie1 2015-04-14 2 3cookie1 2015-04-10 1 3cookie2 2015-04-15 9 1cookie2 2015-04-16 7 1cookie2 2015-04-13 6 1cookie2 2015-04-12 5 2cookie2 2015-04-14 3 2cookie2 2015-04-11 3 3cookie2 2015-04-10 2 3
ROW_NUMBER ROW_NUMBER() –從1開始,按照順序,生成分組內記錄的序列 –比如,按照pv降序排列,生成分組內每天的pv名次 ROW_NUMBER() 的應用場景非常多,再比如,獲取分組內排序第一的記錄;獲取一個session中的第一條refer等。
SELECTcookieid,createtime,pv,ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rnFROM lxw1234;cookieid day pv rn-------------------------------------------cookie1 2015-04-12 7 1cookie1 2015-04-11 5 2cookie1 2015-04-15 4 3cookie1 2015-04-16 4 4cookie1 2015-04-13 3 5cookie1 2015-04-14 2 6cookie1 2015-04-10 1 7cookie2 2015-04-15 9 1cookie2 2015-04-16 7 2cookie2 2015-04-13 6 3cookie2 2015-04-12 5 4cookie2 2015-04-14 3 5cookie2 2015-04-11 3 6cookie2 2015-04-10 2 7
RANK 和 DENSE_RANK —RANK() 生成數據項在分組中的排名,排名相等會在名次中留下空位 —DENSE_RANK() 生成數據項在分組中的排名,排名相等會在名次中不會留下空位
SELECTcookieid,createtime,pv,RANK() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn1,DENSE_RANK() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn2,ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY pv DESC) AS rn3FROM lxw1234WHERE cookieid = 'cookie1';cookieid day pv rn1 rn2 rn3--------------------------------------------------cookie1 2015-04-12 7 1 1 1cookie1 2015-04-11 5 2 2 2cookie1 2015-04-15 4 3 3 3cookie1 2015-04-16 4 3 3 4cookie1 2015-04-13 3 5 4 5cookie1 2015-04-14 2 6 5 6cookie1 2015-04-10 1 7 6 7rn1: 15號和16號并列第3, 13號排第5rn2: 15號和16號并列第3, 13號排第4rn3: 如果相等,則按記錄值排序,生成唯一的次序,如果所有記錄值都相等,或許會隨機排吧。
轉發:http://www.aboutyun.com/thread-12834-1-1.html
總結
以上是生活随笔 為你收集整理的Hive分析窗口函数 NTILE,ROW_NUMBER,RANK,DENSE_RANK 的全部內容,希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔 網站內容還不錯,歡迎將生活随笔 推薦給好友。