本章以京東商品詳情頁為例,京東商品詳情頁雖然僅是單個頁面,但是其數據聚合源是非常多的,除了一些實時性要求比較高的如價格、庫存、服務支持等通過AJAX異步加載加載之外,其他的數據都是在后端做數據聚合然后拼裝網頁模板的。
http://item.jd.com/1217499.html
?
如圖所示,商品頁主要包括商品基本信息(基本信息、圖片列表、顏色/尺碼關系、擴展屬性、規格參數、包裝清單、售后保障等)、商品介紹、其他信息(分類、品牌、店鋪【第三方賣家】、店內分類【第三方賣家】、同類相關品牌)。更多細節此處就不闡述了。
?
整個京東有數億商品,如果每次動態獲取如上內容進行模板拼裝,數據來源之多足以造成性能無法滿足要求;最初的解決方案是生成靜態頁,但是靜態頁的最大的問題:1、無法迅速響應頁面需求變更;2、很難做多版本線上對比測試。如上兩個因素足以制約商品頁的多樣化發展,因此靜態化技術不是很好的方案。
?
通過分析,數據主要分為四種:商品頁基本信息、商品介紹(異步加載)、其他信息(分類、品牌、店鋪等)、其他需要實時展示的數據(價格、庫存等)。而其他信息如分類、品牌、店鋪是非常少的,完全可以放到一個占用內存很小的Redis中存儲;而商品基本信息我們可以借鑒靜態化技術將數據做聚合存儲,這樣的好處是數據是原子的,而模板是隨時可變的,吸收了靜態頁聚合的優點,彌補了靜態頁的多版本缺點;另外一個非常嚴重的問題就是嚴重依賴這些相關系統,如果它們掛了或響應慢則商品頁就掛了或響應慢;商品介紹我們也通過AJAX技術惰性加載(因為是第二屏,只有當用戶滾動鼠標到該屏時才顯示);而實時展示數據通過AJAX技術做異步加載;因此我們可以做如下設計:
1、接收商品變更消息,做商品基本信息的聚合,即從多個數據源獲取商品相關信息如圖片列表、顏色尺碼、規格參數、擴展屬性等等,聚合為一個大的JSON數據做成數據閉環,以key-value存儲;因為是閉環,即使依賴的系統掛了我們商品頁還是能繼續服務的,對商品頁不會造成任何影響;
2、接收商品介紹變更消息,存儲商品介紹信息;
3、介紹其他信息變更消息,存儲其他信息。
?
整個架構如下圖所示:?
技術選型
MQ可以使用如Apache ActiveMQ;
Worker/動態服務可以通過如Java技術實現;
RPC可以選擇如alibaba Dubbo;
KV持久化存儲可以選擇SSDB(如果使用SSD盤則可以選擇SSDB+RocksDB引擎)或者ARDB(LMDB引擎版);
緩存使用Redis;
SSDB/Redis分片使用如Twemproxy,這樣不管使用Java還是Nginx+Lua,它們都不關心分片邏輯;
前端模板拼裝使用Nginx+Lua;
數據集群數據存儲的機器可以采用RAID技術或者主從模式防止單點故障;
因為數據變更不頻繁,可以考慮SSD替代機械硬盤。
?
核心流程
1、首先我們監聽商品數據變更消息;
2、接收到消息后,數據聚合Worker通過RPC調用相關系統獲取所有要展示的數據,此處獲取數據的來源可能非常多而且響應速度完全受制于這些系統,可能耗時幾百毫秒甚至上秒的時間;
3、將數據聚合為JSON串存儲到相關數據集群;
4、前端Nginx通過Lua獲取相關集群的數據進行展示;商品頁需要獲取基本信息+其他信息進行模板拼裝,即拼裝模板僅需要兩次調用(另外因為其他信息數據量少且對一致性要求不高,因此我們完全可以緩存到Nginx本地全局內存,這樣可以減少遠程調用提高性能);當頁面滾動到商品介紹頁面時異步調用商品介紹服務獲取數據;
5、如果從聚合的SSDB集群/Redis中獲取不到相關數據;則回源到動態服務通過RPC調用相關系統獲取所有要展示的數據返回(此處可以做限流處理,因為如果大量請求過來的話可能導致服務雪崩,需要采取保護措施),此處的邏輯和數據聚合Worker完全一樣;然后發送MQ通知數據變更,這樣下次訪問時就可以從聚合的SSDB集群/Redis中獲取數據了。
?
基本流程如上所述,主要分為Worker、動態服務、數據存儲和前端展示;因為系統非常復雜,只介紹動態服務和前端展示、數據存儲架構;Worker部分不做實現。
??
項目搭建
項目部署目錄結構。
/usr/chapter7
? ssdb_basic_7770.conf
? ssdb_basic_7771.conf
? ssdb_basic_7772.conf
? ssdb_basic_7773.conf
? ssdb_desc_8880.conf
? ssdb_desc_8881.conf
? ssdb_desc_8882.conf
? ssdb_desc_8883.conf
? redis_other_6660.conf
? redis_other_6661.conf
? nginx_chapter7.conf
? nutcracker.yml
? nutcracker.init
? item.html
??header.html
? footer.html
??item.lua
? desc.lua
? lualib
? ? item.lua
? ? item
? ? ? common.lua
? webapp
WEB-INF
?? lib
?? classes
?? web.xml
?
數據存儲實現
??
整體架構為主從模式,寫數據到主集群,讀數據從從集群讀取數據,這樣當一個集群不足以支撐流量時可以使用更多的集群來支撐更多的訪問量;集群分片使用Twemproxy實現。
?
商品基本信息SSDB集群配置
vim /usr/chapter7/ssdb_basic_7770.conf
Java代碼??
work_dir?=?/usr/data/ssdb_7770??pidfile?=?/usr/data/ssdb_7770.pid????server:??????????ip:?0.0.0.0??????????port:?7770??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??logger:??????????level:?error??????????output:?/usr/data/ssdb_7770.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ?
vim /usr/chapter7/ssdb_basic_7771.conf ?
Java代碼??
work_dir?=?/usr/data/ssdb_7771??pidfile?=?/usr/data/ssdb_7771.pid????server:??????????ip:?0.0.0.0??????????port:?7771??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??logger:??????????level:?error??????????output:?/usr/data/ssdb_7771.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ?
vim /usr/chapter7/ssdb_basic_7772.conf?
Java代碼??
work_dir?=?/usr/data/ssdb_7772??pidfile?=?/usr/data/ssdb_7772.pid????server:??????????ip:?0.0.0.0??????????port:?7772??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??????????????????type:?sync??????????????????ip:?127.0.0.1??????????????????port:?7770????logger:??????????level:?error??????????output:?/usr/data/ssdb_7772.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ?
vim /usr/chapter7/ssdb_basic_7773.conf?
Java代碼??
work_dir?=?/usr/data/ssdb_7773??pidfile?=?/usr/data/ssdb_7773.pid????server:??????????ip:?0.0.0.0??????????port:?7773??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??????????????????type:?sync??????????????????ip:?127.0.0.1??????????????????port:?7771????logger:??????????level:?error??????????output:?/usr/data/ssdb_7773.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? 配置文件使用Tab而不是空格做縮排,(復制到配置文件后請把空格替換為Tab)。主從關系:7770(主)-->7772(從),7771(主)--->7773(從);配置文件如何配置請參考https://github.com/ideawu/ssdb-docs/blob/master/src/zh_cn/config.md。 ??
?
創建工作目錄
Java代碼??
mkdir?-p?/usr/data/ssdb_7770??mkdir?-p?/usr/data/ssdb_7771??mkdir?-p?/usr/data/ssdb_7772??mkdir?-p?/usr/data/ssdb_7773?? ?
啟動
Java代碼??
nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_basic_7770.conf?&??nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_basic_7771.conf?&??nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_basic_7772.conf?&??nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_basic_7773.conf?&?? 通過ps -aux | grep ssdb命令看是否啟動了,tail -f nohup.out查看錯誤信息。
?
商品介紹SSDB集群配置
vim /usr/chapter7/ssdb_desc_8880.conf
Java代碼??
work_dir?=?/usr/data/ssdb_8880??pidfile?=?/usr/data/ssdb8880.pid????server:??????????ip:?0.0.0.0??????????port:?8880??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??logger:??????????level:?error??????????output:?/usr/data/ssdb_8880.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ??
vim /usr/chapter7/ssdb_desc_8881.conf ?
Java代碼??
work_dir?=?/usr/data/ssdb_8881??pidfile?=?/usr/data/ssdb8881.pid????server:??????????ip:?0.0.0.0??????????port:?8881??????????allow:?127.0.0.1??????????allow:?192.168????logger:??????????level:?error??????????output:?/usr/data/ssdb_8881.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ??
vim /usr/chapter7/ssdb_desc_8882.conf?
Java代碼??
work_dir?=?/usr/data/ssdb_8882??pidfile?=?/usr/data/ssdb_8882.pid????server:??????????ip:?0.0.0.0??????????port:?8882??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??????????????????type:?sync??????????????????ip:?127.0.0.1??????????????????port:?8880????logger:??????????level:?error??????????output:?/usr/data/ssdb_8882.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ??
vim /usr/chapter7/ssdb_desc_8883.conf?
Java代碼??
work_dir?=?/usr/data/ssdb_8883??pidfile?=?/usr/data/ssdb_8883.pid????server:??????????ip:?0.0.0.0??????????port:?8883??????????allow:?127.0.0.1??????????allow:?192.168????replication:??????????binlog:?yes??????????sync_speed:?-1??????????slaveof:??????????????????type:?sync??????????????????ip:?127.0.0.1??????????????????port:?8881????logger:??????????level:?error??????????output:?/usr/data/ssdb_8883.log??????????rotate:??????????????????size:?1000000000????leveldb:??????????cache_size:?500??????????block_size:?32??????????write_buffer_size:?64??????????compaction_speed:?1000??????????compression:?yes?? ??
配置文件使用Tab而不是空格做縮排(復制到配置文件后請把空格替換為Tab)。主從關系:7770(主)-->7772(從),7771(主)--->7773(從);配置文件如何配置請參考https://github.com/ideawu/ssdb-docs/blob/master/src/zh_cn/config.md。???
?
創建工作目錄
Java代碼??
mkdir?-p?/usr/data/ssdb_888{0,1,2,3}?? ?
啟動
Java代碼??
nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_desc_8880.conf?&??nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_desc_8881.conf?&??nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_desc_8882.conf?&??nohup?/usr/servers/ssdb-1.8.0/ssdb-server??/usr/chapter7/ssdb_desc_8883.conf?&?? 通過ps -aux | grep ssdb命令看是否啟動了,tail -f nohup.out查看錯誤信息。
?
其他信息Redis配置
vim /usr/chapter7/redis_6660.conf ?
Java代碼??
port?6660??pidfile?"/var/run/redis_6660.pid"??#設置內存大小,根據實際情況設置,此處測試僅設置20mb??maxmemory?20mb??#內存不足時,所有KEY按照LRU算法刪除??maxmemory-policy?allkeys-lru??#Redis的過期算法不是精確的而是通過采樣來算的,默認采樣為3個,此處我們改成10??maxmemory-samples?10??#不進行RDB持久化??save?“”??#不進行AOF持久化??appendonly?no?? ? ?
vim /usr/chapter7/redis_6661.conf?
Java代碼??
port?6661??pidfile?"/var/run/redis_6661.pid"??#設置內存大小,根據實際情況設置,此處測試僅設置20mb??maxmemory?20mb??#內存不足時,所有KEY按照LRU算法進行刪除??maxmemory-policy?allkeys-lru??#Redis的過期算法不是精確的而是通過采樣來算的,默認采樣為3個,此處我們改成10??maxmemory-samples?10??#不進行RDB持久化??save?“”??#不進行AOF持久化??appendonly?no??#主從??slaveof?127.0.0.1?6660??
vim /usr/chapter7/redis_6662.conf?
Java代碼??
port?6662??pidfile?"/var/run/redis_6662.pid"??#設置內存大小,根據實際情況設置,此處測試僅設置20mb??maxmemory?20mb??#內存不足時,所有KEY按照LRU算法進行刪除??maxmemory-policy?allkeys-lru??#Redis的過期算法不是精確的而是通過采樣來算的,默認采樣為3個,此處我們改成10??maxmemory-samples?10??#不進行RDB持久化??save?“”??#不進行AOF持久化??appendonly?no??#主從??slaveof?127.0.0.1?6660??
如上配置放到配置文件最末尾即可;此處內存不足時的驅逐算法為所有KEY按照LRU進行刪除(實際是內存基本上不會遇到滿的情況);主從關系:6660(主)-->6661(從)和6660(主)-->6662(從)。
啟動
Java代碼??
nohup?/usr/servers/redis-2.8.19/src/redis-server?/usr/chapter7/redis_6660.conf?&??nohup?/usr/servers/redis-2.8.19/src/redis-server?/usr/chapter7/redis_6661.conf?&??nohup?/usr/servers/redis-2.8.19/src/redis-server?/usr/chapter7/redis_6662.conf?&?? 通過ps -aux | grep redis命令看是否啟動了,tail -f nohup.out查看錯誤信息。
?
測試?
測試時在主SSDB/Redis中寫入數據,然后從從SSDB/Redis能讀取到數據即表示配置主從成功。
測試商品基本信息SSDB集群
Java代碼??
root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli??-p?7770??127.0.0.1:7770>?set?i?1??OK??127.0.0.1:7770>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli??-p?7772??127.0.0.1:7772>?get?i??"1"?? 測試商品介紹SSDB集群 Java代碼??
root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli??-p?8880??127.0.0.1:8880>?set?i?1??OK??127.0.0.1:8880>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli??-p?8882??127.0.0.1:8882>?get?i??"1"?? ??
測試其他信息集群 Java代碼??
root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli??-p?6660??127.0.0.1:6660>?set?i?1??OK??127.0.0.1:6660>?get?i??"1"??127.0.0.1:6660>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli??-p?6661??127.0.0.1:6661>?get?i??"1"?? ?
Twemproxy配置
vim /usr/chapter7/nutcracker.yml??
Java代碼??
basic_master:????listen:?127.0.0.1:1111????hash:?fnv1a_64????distribution:?ketama????redis:?true????timeout:?1000????hash_tag:?"::"????servers:?????-?127.0.0.1:7770:1?server1?????-?127.0.0.1:7771:1?server2????basic_slave:????listen:?127.0.0.1:1112????hash:?fnv1a_64????distribution:?ketama????redis:?true????timeout:?1000????hash_tag:?"::"????servers:?????-?127.0.0.1:7772:1?server1?????-?127.0.0.1:7773:1?server2??????desc_master:????listen:?127.0.0.1:1113????hash:?fnv1a_64????distribution:?ketama????redis:?true????timeout:?1000????hash_tag:?"::"????servers:?????-?127.0.0.1:8880:1?server1?????-?127.0.0.1:8881:1?server2????desc_slave:????listen:?127.0.0.1:1114????hash:?fnv1a_64????distribution:?ketama????redis:?true????timeout:?1000????servers:?????-?127.0.0.1:8882:1?server1?????-?127.0.0.1:8883:1?server2????other_master:????listen:?127.0.0.1:1115????hash:?fnv1a_64????distribution:?random????redis:?true????timeout:?1000????hash_tag:?"::"????servers:?????-?127.0.0.1:6660:1?server1??????other_slave:????listen:?127.0.0.1:1116????hash:?fnv1a_64????distribution:?random????redis:?true????timeout:?1000????hash_tag:?"::"????servers:?????-?127.0.0.1:6661:1?server1?????-?127.0.0.1:6662:1?server2?? 1、因為我們使用了主從,所以需要給server起一個名字如server1、server2;否則分片算法默認根據ip:port:weight,這樣就會主從數據的分片算法不一致;
2、其他信息Redis因為每個Redis是對等的,因此分片算法可以使用random;
3、我們使用了hash_tag,可以保證相同的tag在一個分片上(本例配置了但沒有用到該特性)。
?
?
復制第六章的nutcracker.init,幫把配置文件改為usr/chapter7/nutcracker.yml。然后通過/usr/chapter7/nutcracker.init start啟動Twemproxy。
?
測試主從集群是否工作正常:
Java代碼??
root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli?-p?1111??127.0.0.1:1111>?set?i?1??OK??127.0.0.1:1111>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli?-p?1112??127.0.0.1:1112>?get?i??"1"??127.0.0.1:1112>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli?-p?1113??127.0.0.1:1113>?set?i?1??OK??127.0.0.1:1113>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli?-p?1114??127.0.0.1:1114>?get?i??"1"??127.0.0.1:1114>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli?-p?1115??127.0.0.1:1115>?set?i?1??OK??127.0.0.1:1115>???root@kaitao:/usr/chapter7#?/usr/servers/redis-2.8.19/src/redis-cli?-p?1116??127.0.0.1:1116>?get?i??"1"?? ?
到此數據集群配置成功。
?
?
動態服務實現
因為真實數據是從多個子系統獲取,很難模擬這么多子系統交互,所以此處我們使用假數據來進行實現。
?
項目搭建?
我們使用Maven搭建Web項目,Maven知識請自行學習。
?
項目依賴
本文將最小化依賴,即僅依賴我們需要的servlet、jackson、guava、jedis。?
Java代碼??
<dependencies>????<dependency>??????<groupId>javax.servlet</groupId>??????<artifactId>javax.servlet-api</artifactId>??????<version>3.0.1</version>??????<scope>provided</scope>????</dependency>????<dependency>??????<groupId>com.google.guava</groupId>??????<artifactId>guava</artifactId>??????<version>17.0</version>????</dependency>????<dependency>??????<groupId>redis.clients</groupId>??????<artifactId>jedis</artifactId>??????<version>2.5.2</version>????</dependency>????<dependency>??????<groupId>com.fasterxml.jackson.core</groupId>??????<artifactId>jackson-core</artifactId>??????<version>2.3.3</version>????</dependency>????<dependency>??????<groupId>com.fasterxml.jackson.core</groupId>??????<artifactId>jackson-databind</artifactId>??????<version>2.3.3</version>????</dependency>??</dependencies>?? guava是類似于apache commons的一個基礎類庫,用于簡化一些重復操作,可以參考http://ifeve.com/google-guava/。?
?
核心代碼
com.github.zhangkaitao.chapter7.servlet.ProductServiceServlet
Java代碼??
@Override??protected?void?doGet(HttpServletRequest?req,?HttpServletResponse?resp)?throws?ServletException,?IOException?{??????String?type?=?req.getParameter("type");??????String?content?=?null;??????try?{??????????if("basic".equals(type))?{??????????????content?=?getBasicInfo(req.getParameter("skuId"));??????????}?else?if("desc".equals(type))?{??????????????content?=?getDescInfo(req.getParameter("skuId"));??????????}?else?if("other".equals(type))?{??????????????content?=?getOtherInfo(req.getParameter("ps3Id"),?req.getParameter("brandId"));??????????}??????}?catch?(Exception?e)?{??????????e.printStackTrace();??????????resp.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);??????????return;??????}??????if(content?!=?null)?{??????????resp.setCharacterEncoding("UTF-8");??????????resp.getWriter().write(content);??????}?else?{??????????resp.setStatus(HttpServletResponse.SC_NOT_FOUND);??????}??}?? 根據請求參數type來決定調用哪個服務獲取數據。
?
基本信息服務?
Java代碼??
private?String?getBasicInfo(String?skuId)?throws?Exception?{??????Map<String,?Object>?map?=?new?HashMap<String,?Object>();??????//商品編號??????map.put("skuId",?skuId);??????//名稱??????map.put("name",?"蘋果(Apple)iPhone?6?(A1586)?16GB?金色?移動聯通電信4G手機");??????//一級二級三級分類??????map.put("ps1Id",?9987);??????map.put("ps2Id",?653);??????map.put("ps3Id",?655);??????//品牌ID??????map.put("brandId",?14026);??????//圖片列表??????map.put("imgs",?getImgs(skuId));??????//上架時間??????map.put("date",?"2014-10-09?22:29:09");??????//商品毛重??????map.put("weight",?"400");??????//顏色尺碼??????map.put("colorSize",?getColorSize(skuId));??????//擴展屬性??????map.put("expands",?getExpands(skuId));??????//規格參數??????map.put("propCodes",?getPropCodes(skuId));??????map.put("date",?System.currentTimeMillis());??????String?content?=?objectMapper.writeValueAsString(map);??????//實際應用應該是發送MQ??????asyncSetToRedis(basicInfoJedisPool,?"p:"?+?skuId?+?":",?content);??????return?objectMapper.writeValueAsString(map);??}????private?List<String>?getImgs(String?skuId)?{??????return?Lists.newArrayList(??????????????"jfs/t277/193/1005339798/768456/29136988/542d0798N19d42ce3.jpg",??????????????"jfs/t352/148/1022071312/209475/53b8cd7f/542d079bN3ea45c98.jpg",??????????????"jfs/t274/315/1008507116/108039/f70cb380/542d0799Na03319e6.jpg",??????????????"jfs/t337/181/1064215916/27801/b5026705/542d079aNf184ce18.jpg"??????);??}????private?List<Map<String,?Object>>?getColorSize(String?skuId)?{??????return?Lists.newArrayList(??????????makeColorSize(1217499,?"金色",?"公開版(16GB?ROM)"),??????????makeColorSize(1217500,?"深空灰",?"公開版(16GB?ROM)"),??????????makeColorSize(1217501,?"銀色",?"公開版(16GB?ROM)"),??????????makeColorSize(1217508,?"金色",?"公開版(64GB?ROM)"),??????????makeColorSize(1217509,?"深空灰",?"公開版(64GB?ROM)"),??????????makeColorSize(1217509,?"銀色",?"公開版(64GB?ROM)"),??????????makeColorSize(1217493,?"金色",?"移動4G版?(16GB)"),??????????makeColorSize(1217494,?"深空灰",?"移動4G版?(16GB)"),??????????makeColorSize(1217495,?"銀色",?"移動4G版?(16GB)"),??????????makeColorSize(1217503,?"金色",?"移動4G版?(64GB)"),??????????makeColorSize(1217503,?"金色",?"移動4G版?(64GB)"),??????????makeColorSize(1217504,?"深空灰",?"移動4G版?(64GB)"),??????????makeColorSize(1217505,?"銀色",?"移動4G版?(64GB)")??????);??}??private?Map<String,?Object>?makeColorSize(long?skuId,?String?color,?String?size)?{??????Map<String,?Object>?cs1?=?Maps.newHashMap();??????cs1.put("SkuId",?skuId);??????cs1.put("Color",?color);??????cs1.put("Size",?size);??????return?cs1;??}????private?List<List<?>>?getExpands(String?skuId)?{??????return?Lists.newArrayList(??????????????(List<?>)Lists.newArrayList("熱點",?Lists.newArrayList("超薄7mm以下",?"支持NFC")),??????????????(List<?>)Lists.newArrayList("系統",?"蘋果(IOS)"),??????????????(List<?>)Lists.newArrayList("系統",?"蘋果(IOS)"),??????????????(List<?>)Lists.newArrayList("購買方式",?"非合約機")??????);??}????private?Map<String,?List<List<String>>>?getPropCodes(String?skuId)?{??????Map<String,?List<List<String>>>?map?=?Maps.newHashMap();??????map.put("主體",?Lists.<List<String>>newArrayList(??????????????Lists.<String>newArrayList("品牌",?"蘋果(Apple)"),??????????????Lists.<String>newArrayList("型號",?"iPhone?6?A1586"),??????????????Lists.<String>newArrayList("顏色",?"金色"),??????????????Lists.<String>newArrayList("上市年份",?"2014年")??????));??????map.put("存儲",?Lists.<List<String>>newArrayList(??????????????Lists.<String>newArrayList("機身內存",?"16GB?ROM"),??????????????Lists.<String>newArrayList("儲存卡類型",?"不支持")??????));??????map.put("顯示",?Lists.<List<String>>newArrayList(??????????????Lists.<String>newArrayList("屏幕尺寸",?"4.7英寸"),??????????????Lists.<String>newArrayList("觸摸屏",?"Retina?HD"),??????????????Lists.<String>newArrayList("分辨率",?"1334?x?750")??????));??????return?map;??}?? 本例基本信息提供了如商品名稱、圖片列表、顏色尺碼、擴展屬性、規格參數等等數據;而為了簡化邏輯大多數數據都是List/Map數據結構。?
?
商品介紹服務?
Java代碼??
private?String?getDescInfo(String?skuId)?throws?Exception?{??????Map<String,?Object>?map?=?new?HashMap<String,?Object>();??????map.put("content",?"<div><img?data-lazyload='http://img30.360buyimg.com/jgsq-productsoa/jfs/t448/127/574781110/103911/b3c80634/5472ba22N45400f4e.jpg'?alt=''?/><img?data-lazyload='http://img30.360buyimg.com/jgsq-productsoa/jfs/t802/133/19465528/162152/e463e43/54e2b34aN11bceb70.jpg'?alt=''?height='386'?width='750'?/></div>");??????map.put("date",?System.currentTimeMillis());??????String?content?=?objectMapper.writeValueAsString(map);??????//實際應用應該是發送MQ??????asyncSetToRedis(descInfoJedisPool,?"d:"?+?skuId?+?":",?content);??????return?objectMapper.writeValueAsString(map);??}?? ??
其他信息服務
Java代碼??
private?String?getOtherInfo(String?ps3Id,?String?brandId)?throws?Exception?{??????Map<String,?Object>?map?=?new?HashMap<String,?Object>();??????//面包屑??????List<List<?>>?breadcrumb?=?Lists.newArrayList();??????breadcrumb.add(Lists.newArrayList(9987,?"手機"));??????breadcrumb.add(Lists.newArrayList(653,?"手機通訊"));??????breadcrumb.add(Lists.newArrayList(655,?"手機"));??????//品牌??????Map<String,?Object>?brand?=?Maps.newHashMap();??????brand.put("name",?"蘋果(Apple)");??????brand.put("logo",?"BrandLogo/g14/M09/09/10/rBEhVlK6vdkIAAAAAAAFLXzp-lIAAHWawP_QjwAAAVF472.png");??????map.put("breadcrumb",?breadcrumb);??????map.put("brand",?brand);??????//實際應用應該是發送MQ??????asyncSetToRedis(otherInfoJedisPool,?"s:"?+?ps3Id?+?":",?objectMapper.writeValueAsString(breadcrumb));??????asyncSetToRedis(otherInfoJedisPool,?"b:"?+?brandId?+?":",?objectMapper.writeValueAsString(brand));??????return?objectMapper.writeValueAsString(map);??}?? 本例中其他信息只使用了面包屑和品牌數據。
?
輔助工具
Java代碼??
private?ObjectMapper?objectMapper?=?new?ObjectMapper();??private?JedisPool?basicInfoJedisPool?=?createJedisPool("127.0.0.1",?1111);??private?JedisPool?descInfoJedisPool?=?createJedisPool("127.0.0.1",?1113);??private?JedisPool?otherInfoJedisPool?=?createJedisPool("127.0.0.1",?1115);????private?JedisPool?createJedisPool(String?host,?int?port)?{??????GenericObjectPoolConfig?poolConfig?=?new?GenericObjectPoolConfig();??????poolConfig.setMaxTotal(100);??????return?new?JedisPool(poolConfig,?host,?port);??}????private?ExecutorService?executorService?=?Executors.newFixedThreadPool(10);??private?void?asyncSetToRedis(final?JedisPool?jedisPool,?final?String?key,?final?String?content)?{??????executorService.submit(new?Runnable()?{??????????@Override??????????public?void?run()?{??????????????Jedis?jedis?=?null;??????????????try?{??????????????????jedis?=?jedisPool.getResource();??????????????????jedis.set(key,?content);??????????????}?catch?(Exception?e)?{??????????????????e.printStackTrace();??????????????????jedisPool.returnBrokenResource(jedis);??????????????}?finally?{??????????????????jedisPool.returnResource(jedis);??????????????}????????????}??????});??}?? 本例使用Jackson進行JSON的序列化;Jedis進行Redis的操作;使用線程池做異步更新(實際應用中可以使用MQ做實現)。?
?
web.xml配置 Java代碼??
<servlet>??????<servlet-name>productServiceServlet</servlet-name>??????<servlet-class>com.github.zhangkaitao.chapter7.servlet.ProductServiceServlet</servlet-class>??</servlet>??<servlet-mapping>??????<servlet-name>productServiceServlet</servlet-name>??????<url-pattern>/info</url-pattern>??</servlet-mapping>?? ??
打WAR包?
Java代碼??
cd?D:\workspace\chapter7??mvn?clean?package?? 此處使用maven命令打包,比如本例將得到chapter7.war,然后將其上傳到服務器的/usr/chapter7/webapp,然后通過unzip chapter6.war解壓。
?
配置Tomcat
復制第六章使用的tomcat實例:
Java代碼??
cd?/usr/servers/??cp?-r?tomcat-server1?tomcat-chapter7/??vim?/usr/servers/tomcat-chapter7/conf/Catalina/localhost/ROOT.xml??? ?
Java代碼??
<!--?訪問路徑是根,web應用所屬目錄為/usr/chapter7/webapp?-->??<Context?path=""?docBase="/usr/chapter7/webapp"></Context>?? 指向第七章的web應用路徑。
?
測試?
啟動tomcat實例。
Java代碼??
/usr/servers/tomcat-chapter7/bin/startup.sh??? 訪問如下URL進行測試。?
Java代碼??
http://192.168.1.2:8080/info?type=basic&skuId=1??http://192.168.1.2:8080/info?type=desc&skuId=1??http://192.168.1.2:8080/info?type=other&ps3Id=1&brandId=1?? ?
nginx配置
vim /usr/chapter7/nginx_chapter7.conf?
Java代碼??
upstream?backend?{??????server?127.0.0.1:8080?max_fails=5?fail_timeout=10s?weight=1;??????check?interval=3000?rise=1?fall=2?timeout=5000?type=tcp?default_down=false;??????keepalive?100;??}????server?{??????listen???????80;??????server_name??item2015.jd.com?item.jd.com?d.3.cn;????????location?~?/backend/(.*)?{??????????#internal;??????????keepalive_timeout???30s;??????????keepalive_requests??1000;??????????#支持keep-alive??????????proxy_http_version?1.1;??????????proxy_set_header?Connection?"";????????????rewrite?/backend(/.*)?$1?break;??????????proxy_pass_request_headers?off;??????????#more_clear_input_headers?Accept-Encoding;??????????proxy_next_upstream?error?timeout;??????????proxy_pass?http://backend;??????}??}?? 此處server_name 我們指定了item.jd.com(商品詳情頁)和d.3.cn(商品介紹)。其他配置可以參考第六章內容。另外實際生產環境要把#internal打開,表示只有本nginx能訪問。
?
vim /usr/servers/nginx/conf/nginx.conf
Java代碼??
include?/usr/chapter7/nginx_chapter7.conf;??#為了方便測試,注釋掉example.conf??include?/usr/chapter6/nginx_chapter6.conf;?? Java代碼??
#lua模塊路徑,其中”;;”表示默認搜索路徑,默認到/usr/servers/nginx下找??lua_package_path?"/usr/chapter7/lualib/?.lua;;";??#lua?模塊??lua_package_cpath?"/usr/chapter7/lualib/?.so;;";??#c模塊?? lua模塊從/usr/chapter7目錄加載,因為我們要寫自己的模塊使用。
?
重啟nginx?
/usr/servers/nginx/sbin/nginx -s reload ? ? ?
?
綁定hosts
192.168.1.2 item.jd.com
192.168.1.2 item2015.jd.com?
192.168.1.2 d.3.cn
?
訪問如http://item.jd.com/backend/info?type=basic&skuId=1即看到結果。
?
前端展示實現?
我們分為三部分實現:基礎組件、商品介紹、前端展示部分。
?
基礎組件
首先我們進行基礎組件的實現,商品介紹和前端展示部分都需要讀取Redis和Http服務,因此我們可以抽取公共部分出來復用。
vim /usr/chapter7/lualib/item/common.lua
Java代碼??
local?redis?=?require("resty.redis")??local?ngx_log?=?ngx.log??local?ngx_ERR?=?ngx.ERR??local?function?close_redis(red)??????if?not?red?then??????????return??????end??????--釋放連接(連接池實現)??????local?pool_max_idle_time?=?10000?--毫秒??????local?pool_size?=?100?--連接池大小??????local?ok,?err?=?red:set_keepalive(pool_max_idle_time,?pool_size)????????if?not?ok?then??????????ngx_log(ngx_ERR,?"set?redis?keepalive?error?:?",?err)??????end??end????local?function?read_redis(ip,?port,?keys)??????local?red?=?redis:new()??????red:set_timeout(1000)??????local?ok,?err?=?red:connect(ip,?port)??????if?not?ok?then??????????ngx_log(ngx_ERR,?"connect?to?redis?error?:?",?err)??????????return?close_redis(red)??????end??????local?resp?=?nil??????if?#keys?==?1?then??????????resp,?err?=?red:get(keys[1])??????else??????????resp,?err?=?red:mget(keys)??????end??????if?not?resp?then??????????ngx_log(ngx_ERR,?"get?redis?content?error?:?",?err)??????????return?close_redis(red)??????end????????--得到的數據為空處理??????if?resp?==?ngx.null?then??????????resp?=?nil??????end??????close_redis(red)????????return?resp??end????local?function?read_http(args)??????local?resp?=?ngx.location.capture("/backend/info",?{??????????method?=?ngx.HTTP_GET,??????????args?=?args??????})????????if?not?resp?then??????????ngx_log(ngx_ERR,?"request?error")??????????return??????end??????if?resp.status?~=?200?then??????????ngx_log(ngx_ERR,?"request?error,?status?:",?resp.status)??????????return??????end??????return?resp.body??end????local?_M?=?{??????read_redis?=?read_redis,??????read_http?=?read_http??}??return?_M?? 整個邏輯和第六章類似;只是read_redis根據參數keys個數支持get和mget。 比如read_redis(ip, port, {"key1"})則調用get而read_redis(ip, port, {"key1", "key2"})則調用mget。
?
商品介紹
核心代碼
vim /usr/chapter7/desc.lua
Java代碼??
local?common?=?require("item.common")??local?read_redis?=?common.read_redis??local?read_http?=?common.read_http??local?ngx_log?=?ngx.log??local?ngx_ERR?=?ngx.ERR??local?ngx_exit?=?ngx.exit??local?ngx_print?=?ngx.print??local?ngx_re_match?=?ngx.re.match??local?ngx_var?=?ngx.var????local?descKey?=?"d:"?..?skuId?..?":"??local?descInfoStr?=?read_redis("127.0.0.1",?1114,?{descKey})??if?not?descInfoStr?then?????ngx_log(ngx_ERR,?"redis?not?found?desc?info,?back?to?http,?skuId?:?",?skuId)?????descInfoStr?=?read_http({type="desc",?skuId?=?skuId})??end??if?not?descInfoStr?then?????ngx_log(ngx_ERR,?"http?not?found?basic?info,?skuId?:?",?skuId)?????return?ngx_exit(404)??end??ngx_print("showdesc(")??ngx_print(descInfoStr)??ngx_print(")")?? 通過復用邏輯后整體代碼簡化了許多;此處讀取商品介紹從集群;另外前端展示使用JSONP技術展示商品介紹。?
?
nginx配置?
vim /usr/chapter7/nginx_chapter7.conf?
Java代碼??
location?~^/desc/(\d+)$?{??????if?($host?!=?"d.3.cn")?{?????????return?403;??????}??????default_type?application/x-javascript;??????charset?utf-8;??????lua_code_cache?on;??????set?$skuId?$1;??????content_by_lua_file?/usr/chapter7/desc.lua;??}?? 因為item.jd.com和d.3.cn復用了同一個配置文件,此處需要限定只有d.3.cn域名能訪問,防止惡意訪問。?
?
重啟nginx后,訪問如http://d.3.cn/desc/1即可得到JSONP結果。
?
前端展示
核心代碼
vim /usr/chapter7/item.lua?
Java代碼??
local?common?=?require("item.common")??local?item?=?require("item")??local?read_redis?=?common.read_redis??local?read_http?=?common.read_http??local?cjson?=?require("cjson")??local?cjson_decode?=?cjson.decode??local?ngx_log?=?ngx.log??local?ngx_ERR?=?ngx.ERR??local?ngx_exit?=?ngx.exit??local?ngx_print?=?ngx.print??local?ngx_var?=?ngx.var??????local?skuId?=?ngx_var.skuId????--獲取基本信息??local?basicInfoKey?=?"p:"?..?skuId?..?":"??local?basicInfoStr?=?read_redis("127.0.0.1",?1112,?{basicInfoKey})??if?not?basicInfoStr?then?????ngx_log(ngx_ERR,?"redis?not?found?basic?info,?back?to?http,?skuId?:?",?skuId)?????basicInfoStr?=?read_http({type="basic",?skuId?=?skuId})??end??if?not?basicInfoStr?then?????ngx_log(ngx_ERR,?"http?not?found?basic?info,?skuId?:?",?skuId)?????return?ngx_exit(404)??end????local?basicInfo?=?cjson_decode(basicInfoStr)??local?ps3Id?=?basicInfo["ps3Id"]??local?brandId?=?basicInfo["brandId"]??--獲取其他信息??local?breadcrumbKey?=?"s:"?..?ps3Id?..?":"??local?brandKey?=?"b:"?..?brandId?..":"??local?otherInfo?=?read_redis("127.0.0.1",?1116,?{breadcrumbKey,?brandKey})?or?{}??local?breadcrumbStr?=?otherInfo[1]??local?brandStr?=?otherInfo[2]??if?breadcrumbStr?then?????basicInfo["breadcrumb"]?=?cjson_decode(breadcrumbStr)??end??if?brandStr?then?????basicInfo["brand"]?=?cjson_decode(brandStr)??end??if?not?breadcrumbStr?and?not?brandStr?then?????ngx_log(ngx_ERR,?"redis?not?found?other?info,?back?to?http,?skuId?:?",?brandId)?????local?otherInfoStr?=?read_http({type="other",?ps3Id?=?ps3Id,?brandId?=?brandId})?????if?not?otherInfoStr?then?????????ngx_log(ngx_ERR,?"http?not?found?other?info,?skuId?:?",?skuId)?????else???????local?otherInfo?=?cjson_decode(otherInfoStr)???????basicInfo["breadcrumb"]?=?otherInfo["breadcrumb"]???????basicInfo["brand"]?=?otherInfo["brand"]?????end??end????local?name?=?basicInfo["name"]??--name?to?unicode??basicInfo["unicodeName"]?=?item.utf8_to_unicode(name)??--字符串截取,超長顯示...??basicInfo["moreName"]?=?item.trunc(name,?10)??--初始化各分類的url??item.init_breadcrumb(basicInfo)??--初始化擴展屬性??item.init_expand(basicInfo)??--初始化顏色尺碼??item.init_color_size(basicInfo)??local?template?=?require?"resty.template"??template.caching(true)??template.render("item.html",?basicInfo)?? 整個邏輯分為四部分:1、獲取基本信息;2、根據基本信息中的關聯關系獲取其他信息;3、初始化/格式化數據;4、渲染模板。??
?
初始化模塊?
vim /usr/chapter7/lualib/item.lua?
Java代碼??
local?bit?=?require("bit")??local?utf8?=?require("utf8")??local?cjson?=?require("cjson")??local?cjson_encode?=?cjson.encode??local?bit_band?=?bit.band??local?bit_bor?=?bit.bor??local?bit_lshift?=?bit.lshift??local?string_format?=?string.format??local?string_byte?=?string.byte??local?table_concat?=?table.concat????--utf8轉為unicode??local?function?utf8_to_unicode(str)??????if?not?str?or?str?==?""?or?str?==?ngx.null?then??????????return?nil??????end??????local?res,?seq,?val?=?{},?0,?nil??????for?i?=?1,?#str?do??????????local?c?=?string_byte(str,?i)??????????if?seq?==?0?then??????????????if?val?then??????????????????res[#res?+?1]?=?string_format("%04x",?val)??????????????end???????????????seq?=?c?<?0x80?and?1?or?c?<?0xE0?and?2?or?c?<?0xF0?and?3?or????????????????????????????????c?<?0xF8?and?4?or?--c?<?0xFC?and?5?or?c?<?0xFE?and?6?or????????????????????????????????0??????????????if?seq?==?0?then??????????????????ngx.log(ngx.ERR,?'invalid?UTF-8?character?sequence'?..?",,,"?..?tostring(str))??????????????????return?str??????????????end????????????????val?=?bit_band(c,?2?^?(8?-?seq)?-?1)??????????else??????????????val?=?bit_bor(bit_lshift(val,?6),?bit_band(c,?0x3F))??????????end??????????seq?=?seq?-?1??????end??????if?val?then??????????res[#res?+?1]?=?string_format("%04x",?val)??????end??????if?#res?==?0?then??????????return?str??????end??????return?"\\u"?..?table_concat(res,?"\\u")??end????--utf8字符串截取??local?function?trunc(str,?len)?????if?not?str?then???????return?nil?????end???????if?utf8.len(str)?>?len?then????????return?utf8.sub(str,?1,?len)?..?"..."?????end?????return?str??end????--初始化面包屑??local?function?init_breadcrumb(info)??????local?breadcrumb?=?info["breadcrumb"]??????if?not?breadcrumb?then?????????return??????end????????local?ps1Id?=?breadcrumb[1][1]??????local?ps2Id?=?breadcrumb[2][1]??????local?ps3Id?=?breadcrumb[3][1]????????--此處應該根據一級分類查找url??????local?ps1Url?=?"http://shouji.jd.com/"??????local?ps2Url?=?"http://channel.jd.com/shouji.html"??????local?ps3Url?=?"http://list.jd.com/list.html?cat="?..?ps1Id?..?","?..?ps2Id?..?","?..?ps3Id????????breadcrumb[1][3]?=?ps1Url??????breadcrumb[2][3]?=?ps2Url??????breadcrumb[3][3]?=?ps3Url??end????--初始化擴展屬性??local?function?init_expand(info)?????local?expands?=?info["expands"]?????if?not?expands?then???????return?????end?????for?_,?e?in?ipairs(expands)?do????????if?type(e[2])?==?"table"?then???????????e[2]?=?table_concat(e[2],?",")????????end?????end??end????--初始化顏色尺碼??local?function?init_color_size(info)?????local?colorSize?=?info["colorSize"]???????--顏色尺碼JSON串?????local?colorSizeJson?=?cjson_encode(colorSize)?????--顏色列表(不重復)?????local?colorList?=?{}?????--尺碼列表(不重復)?????local?sizeList?=?{}?????info["colorSizeJson"]?=?colorSizeJson?????info["colorList"]?=?colorList?????info["sizeList"]?=?sizeList???????local?colorSet?=?{}?????local?sizeSet?=?{}?????for?_,?cz?in?ipairs(colorSize)?do????????local?color?=?cz["Color"]????????local?size?=?cz["Size"]????????if?color?and?color?~=?""?and?not?colorSet[color]?then???????????colorList[#colorList?+?1]?=?{color?=?color,?url?=?"http://item.jd.com/"?..cz["SkuId"]?..?".html"}???????????colorSet[color]?=?true????????end????????if?size?and?size?~=?""?and?not?sizeSet[size]?then???????????sizeList[#sizeList?+?1]?=?{size?=?size,?url?=?"http://item.jd.com/"?..cz["SkuId"]?..?".html"}???????????sizeSet[size]?=?""????????end?????end??end????local?_M?=?{?????utf8_to_unicode?=?utf8_to_unicode,?????trunc?=?trunc,?????init_breadcrumb?=?init_breadcrumb,?????init_expand?=?init_expand,?????init_color_size?=?init_color_size??}????return?_M?? 比如utf8_to_unicode代碼之前已經見過了,其他的都是一些邏輯代碼。
?
模板html片段??
Java代碼??
var?pageConfig?=?{???????compatible:?true,???????product:?{???????????skuid:?{*?skuId?*},???????????name:?'{*?unicodeName?*}',???????????skuidkey:'AFC266E971535B664FC926D34E91C879',???????????href:?'http://item.jd.com/{*?skuId?*}.html',???????????src:?'{*?imgs[1]?*}',???????????cat:?[{*?ps1Id?*},{*?ps2Id?*},{*?ps3Id?*}],???????????brand:?{*?brandId?*},???????????tips:?false,???????????pType:?1,???????????venderId:0,???????????shopId:'0',???????????specialAttrs:["HYKHSP-0","isDistribution","isHaveYB","isSelfService-0","isWeChatStock-0","packType","IsNewGoods","isCanUseDQ","isSupportCard","isCanUseJQ","isOverseaPurchase-0","is7ToReturn-1","isCanVAT"],???????????videoPath:'',???????????desc:?'http://d.3.cn/desc/{*?skuId?*}'???????}???};???var?warestatus?=?1;???????????????????{%?if?colorSizeJson?then?%}?var?ColorSize?=?{*?colorSizeJson?*};{%?end?%}???????????{-raw-}???????????try{(function(flag){?if(!flag){return;}?if(window.location.hash?==?'#m'){var?exp?=?new?Date();exp.setTime(exp.getTime()?+?30?*?24?*?60?*?60?*?1000);document.cookie?=?"pcm=1;expires="?+?exp.toGMTString()?+?";path=/;domain=jd.com";return;}else{var?cook=document.cookie.match(new?RegExp("(^|?)pcm=([^;]*)(;|$)"));if(cook&&cook.length>2&&unescape(cook[2])=="2"){flag=false;}}?var?userAgent?=?navigator.userAgent;?if(userAgent){?userAgent?=?userAgent.toUpperCase();if(userAgent.indexOf("PAD")>-1){return;}?var?mobilePhoneList?=?["IOS","IPHONE","ANDROID","WINDOWS?PHONE"];for(var?i=0,len=mobilePhoneList.length;i<len;i++){?if(userAgent.indexOf(mobilePhoneList[i])>-1){var?url="http://m.jd.com/product/"+pageConfig.product.skuid+".html";if(flag){window.showtouchurl=true;}else{window.location.href?=?url;}break;}}}})((function(){var?json={"6881":3,"1195":3,"10011":3,"6980":3,"12360":3};if(json[pageConfig.product.cat[0]+""]==1||json[pageConfig.product.cat[1]+""]==2||json[pageConfig.product.cat[2]+""]==3){return?false;}else{return?true;}})());}catch(e){}???????????{-raw-}?? {* var *}輸出變量,{% code %} 寫代碼片段,{-raw-} 不進行任何處理直接輸出。
?
面包屑
Java代碼??
<div?class="breadcrumb">??????<strong><a?href='{*?breadcrumb[1][3]?*}'>{*?breadcrumb[1][2]?*}</a></strong>??????<span>?????????? > ??????????<a?href='{*?breadcrumb[2][3]?*}'>{*?breadcrumb[2][2]?*}</a>?????????? > ??????????<a?href='{*?breadcrumb[3][3]?*}'>{*?breadcrumb[3][2]?*}</a>?????????? > ??????</span>??????<span>??????????{%?if?brand?then?%}??????????<a?href='http://www.jd.com/pinpai/{*?ps3Id?*}-{*?brandId?*}.html'>{*?brand['name']?*}</a>?????????? > ?????????{%?end?%}?????????<a?href='http://item.jd.com/{*?skuId?*}.html'>{*?moreName?*}</a>??????</span>??</div>?? ?
圖片列表
Java代碼??
<div?id="spec-n1"?class="jqzoom"?οnclick="window.open('http://www.jd.com/bigimage.aspx?id={*?skuId?*}')"?clstag="shangpin|keycount|product|spec-n1">??????<img?data-img="1"?width="350"?height="350"?src="http://img14.360buyimg.com/n1/{*?imgs[1]?*}"?alt="{*?name?*}"/>??</div>??<div?id="spec-list"?clstag="shangpin|keycount|product|spec-n5">??????<a?href="javascript:;"?class="spec-control"?id="spec-forward"></a>??????<a?href="javascript:;"?class="spec-control"?id="spec-backward"></a>??????<div?class="spec-items">??????????<ul?class="lh">??????????????{%?for?_,?img?in?ipairs(imgs)?do?%}??????????????<li><img?class='img-hover'?alt='{*?name?*}'?src='http://img14.360buyimg.com/n5/{*?img?*}'?data-url='{*?img?*}'?data-img='1'?width='50'?height='50'></li>??????????????{%?end?%}??????????</ul>??????</div>??</div>?? ??
顏色尺碼選擇
Java代碼??
<div?class="dt">選擇顏色:</div>??????<div?class="dd">??????????{%?for?_,?color?in?ipairs(colorList)?do?%}??????????????<div?class="item"><b></b><a?href="{*?color['url']?*}"?title="{*?color['color']?*}"><i>{*?color['color']?*}</i></a></div>??????????{%?end?%}??????</div>??</div>??<div?id="choose-version"?class="li">??????<div?class="dt">選擇版本:</div>??????<div?class="dd">??????????{%?for?_,?size?in?ipairs(sizeList)?do?%}??????????????<div?class="item"><b></b><a?href="{*?size['url']?*}"?title="{*?size['size']?*}">{*?size['size']?*}</a></div>??????????{%?end?%}??????</div>??</div>?? ?
擴展屬性
Java代碼??
<ul?id="parameter2"?class="p-parameter-list">??????<li?title='{*?name?*}'>商品名稱:{*?name?*}</li>??????<li?title='{*?skuId?*}'>商品編號:{*?skuId?*}</li>??????{%?if?brand?then?%}??????<li?title='{*?brand["name"]?*}'>品牌:?<a?href='http://www.jd.com/pinpai/{*?ps3Id?*}-{*?brandId?*}.html'?target='_blank'>{*?brand["name"]?*}</a></li>??????{%?end?%}??????{%?if?date?then?%}??????<li?title='{*?date?*}'>上架時間:{*?date?*}</li>??????{%?end?%}??????{%?if?weight?then?%}??????<li?title='{*?weight?*}'>商品毛重:{*?weight?*}</li>??????{%?end?%}??????{%?for?_,?e?in?pairs(expands)?do?%}??????<li?title='{*?e[2]?*}'>{*?e[1]?*}:{*?e[2]?*}</li>??????{%?end?%}??</ul>?? ?
規格參數
Java代碼??
<table?cellpadding="0"?cellspacing="1"?width="100%"?border="0"?class="Ptable">??????{%?for?group,?pc?in?pairs(propCodes)?do??%}??????<tr><th?class="tdTitle"?colspan="2">{*?group?*}</th><tr>??????{%?for?_,?v?in?pairs(pc)?do?%}??????<tr><td?class="tdTitle">{*?v[1]?*}</td><td>{*?v[2]?*}</td></tr>??????{%?end?%}??????{%?end?%}??</table>?? ??
nginx配置?
vim /usr/chapter7/nginx_chapter7.conf??
Java代碼??
#模板加載位置??set?$template_root?"/usr/chapter7";??????location?~?^/(\d+).html$?{??????if?($host?!~?"^(item|item2015)\.jd\.com$")?{?????????return?403;??????}??????default_type?'text/html';??????charset?utf-8;??????lua_code_cache?on;??????set?$skuId?$1;??????content_by_lua_file?/usr/chapter7/item.lua;??}?? ??
測試
重啟nginx,訪問http://item.jd.com/1217499.html可得到響應內容,本例和京東的商品詳情頁的數據是有些出入的,輸出的頁面可能是缺少一些數據的。
?
優化
local cache
對于其他信息,對數據一致性要求不敏感,而且數據量很少,完全可以在本地緩存全量;而且可以設置如5-10分鐘的過期時間是完全可以接受的;因此可以lua_shared_dict全局內存進行緩存。具體邏輯可以參考
Java代碼??
local?nginx_shared?=?ngx.shared??--item.jd.com配置的緩存??local?local_cache?=?nginx_shared.item_local_cache??local?function?cache_get(key)??????if?not?local_cache?then??????????return?nil??????end??????return?local_cache:get(key)??end????local?function?cache_set(key,?value)??????if?not?local_cache?then??????????return?nil??????end??????return?local_cache:set(key,?value,?10?*?60)?--10分鐘??end????local?function?get(ip,?port,?keys)??????local?tables?=?{}??????local?fetchKeys?=?{}??????local?resp?=?nil??????local?status?=?STATUS_OK??????--如果tables是個map?#tables拿不到長度??????local?has_value?=?false??????--先讀取本地緩存??????for?i,?key?in?ipairs(keys)?do??????????local?value?=?cache_get(key)??????????if?value?then??????????????if?value?==?""?then??????????????????value?=?nil??????????????end??????????????tables[key]?=?value??????????????has_value?=?true??????????else??????????????fetchKeys[#fetchKeys?+?1]?=?key??????????end??????end????????--如果還有數據沒獲取?從redis獲取??????if?#fetchKeys?>?0?then??????????if?#fetchKeys?==?1?then??????????????status,?resp?=?redis_get(ip,?port,?fetchKeys[1])??????????else??????????????status,?resp?=?redis_mget(ip,?port,?fetchKeys)??????????end??????????if?status?==?STATUS_OK?then??????????????for?i?=?1,?#fetchKeys?do???????????????????local?key?=?fetchKeys[i]???????????????????local?value?=?nil???????????????????if?#fetchKeys?==?1?then??????????????????????value?=?resp???????????????????else??????????????????????value?=?get_data(resp,?i)???????????????????end???????????????????tables[key]?=?value????????????????????has_value?=?true????????????????????cache_set(key,?value?or?"",?ttl)??????????????end??????????end??????end??????--如果從緩存查到?就認為ok??????if?has_value?and?status?==?STATUS_NOT_FOUND?then??????????status?=?STATUS_OK??????end??????return?status,?tables??end?? ?
nginx proxy cache
為了防止惡意刷頁面/熱點頁面訪問頻繁,我們可以使用nginx proxy_cache做頁面緩存,當然更好的選擇是使用CDN技術,如通過Apache Traffic Server、Squid、Varnish。
1、nginx.conf配置
Java代碼??
proxy_buffering?on;??proxy_buffer_size?8k;??proxy_buffers?256?8k;??proxy_busy_buffers_size?64k;??proxy_temp_file_write_size?64k;??proxy_temp_path?/usr/servers/nginx/proxy_temp;??#設置Web緩存區名稱為cache_one,內存緩存空間大小為200MB,1分鐘沒有被訪問的內容自動清除,硬盤緩存空間大小為30GB。??proxy_cache_path??/usr/servers/nginx/proxy_cache?levels=1:2?keys_zone=cache_item:200m?inactive=1m?max_size=30g;?? 增加proxy_cache的配置,可以通過掛載一塊內存作為緩存的存儲空間。更多配置規則請參考http://nginx.org/cn/docs/http/ngx_http_proxy_module.html。?
?
?
2、nginx_chapter7.conf配置
與server指令配置同級
Java代碼??
############?測試時使用的動態請求??map?$host?$item_dynamic?{??????default????????????????????"0";??????item2015.jd.com????????????"1";??}?? 即如果域名為item2015.jd.com則item_dynamic=1。 Java代碼??
location?~?^/(\d+).html$?{??????set?$skuId?$1;??????if?($host?!~?"^(item|item2015)\.jd\.com$")?{?????????return?403;??????}????????expires?3m;??????proxy_cache?cache_item;??????proxy_cache_key?$uri;??????proxy_cache_bypass?$item_dynamic;??????proxy_no_cache?$item_dynamic;??????proxy_cache_valid?200?301?3m;??????proxy_cache_use_stale?updating?error?timeout?invalid_header?http_500?http_502?http_503?http_504;??????proxy_pass_request_headers?off;??????proxy_set_header?Host?$host;??????#支持keep-alive??????proxy_http_version?1.1;??????proxy_set_header?Connection?"";??????proxy_pass?http://127.0.0.1/proxy/$skuId.html;??????add_header?X-Cache?'$upstream_cache_status';??}????location?~?^/proxy/(\d+).html$?{??????allow?127.0.0.1;??????deny?all;??????keepalive_timeout???30s;??????keepalive_requests??1000;??????default_type?'text/html';??????charset?utf-8;??????lua_code_cache?on;??????set?$skuId?$1;??????content_by_lua_file?/usr/chapter7/item.lua;??}?? expires:設置響應緩存頭信息,此處是3分鐘;將會得到Cache-Control:max-age=180和類似Expires:Sat, 28 Feb 2015 10:01:10 GMT的響應頭; proxy_cache:使用之前在nginx.conf中配置的cache_item緩存;
proxy_cache_key:緩存key為uri,不包括host和參數,這樣不管用戶怎么通過在url上加隨機數都是走緩存的;
proxy_cache_bypass:nginx不從緩存取響應的條件,可以寫多個;如果存在一個字符串條件且不是“0”,那么nginx就不會從緩存中取響應內容;此處如果我們使用的host為item2015.jd.com時就不會從緩存取響應內容;
proxy_no_cache:nginx不將響應內容寫入緩存的條件,可以寫多個;如果存在一個字符串條件且不是“0”,那么nginx就不會從將響應內容寫入緩存;此處如果我們使用的host為item2015.jd.com時就不會將響應內容寫入緩存;
proxy_cache_valid:為不同的響應狀態碼設置不同的緩存時間,此處我們對200、301緩存3分鐘;
proxy_cache_use_stale:什么情況下使用不新鮮(過期)的緩存內容;配置和proxy_next_upstream內容類似;此處配置了如果連接出錯、超時、404、500等都會使用不新鮮的緩存內容;此外我們配置了updating配置,通過配置它可以在nginx正在更新緩存(其中一個Worker進程)時(其他的Worker進程)使用不新鮮的緩存進行響應,這樣可以減少回源的數量;
proxy_pass_request_headers:我們不需要請求頭,所以不傳遞;
proxy_http_version 1.1和proxy_set_header Connection "":支持keepalive;
add_header X-Cache '$upstream_cache_status':添加是否緩存命中的響應頭;比如命中HIT、不命中MISS、不走緩存BYPASS;比如命中會看到X-Cache:HIT響應頭;
allow/deny:允許和拒絕訪問的ip列表,此處我們只允許本機訪問;
keepalive_timeout 30s和keepalive_requests 1000:支持keepalive;
?
nginx_chapter7.conf清理緩存配置
Java代碼??
location?/purge?{??????allow?????127.0.0.1;??????allow?????192.168.0.0/16;??????deny??????all;??????proxy_cache_purge??cache_item?$arg_url;??}?? 只允許內網訪問。訪問如http://item.jd.com/purge?url=/11.html;如果看到Successful purge說明緩存存在并清理了。
?
3、修改item.lua代碼?
Java代碼??
--添加Last-Modified,用于響應304緩存??ngx.header["Last-Modified"]?=?ngx.http_time(ngx.now())????local?template?=?require?"resty.template"??template.caching(true)??template.render("item.html",?basicInfo)??~??????????????????????????????????????????? 在渲染模板前設置Last-Modified,用于判斷內容是否變更的條件,默認Nginx通過等于去比較,也可以通過配置if_modified_since指令來支持小于等于比較;如果請求頭發送的If-Modified-Since和Last-Modified匹配則返回304響應,即內容沒有變更,使用本地緩存。此處可能看到了我們的Last-Modified是當前時間,不是商品信息變更的時間;商品信息變更時間由:商品信息變更時間、面包屑變更時間和品牌變更時間三者決定的,因此實際應用時應該取三者最大的;還一個問題就是模板內容可能變了,但是商品信息沒有變,此時使用Last-Modified得到的內容可能是錯誤的,所以可以通過使用ETag技術來解決這個問題,ETag可以認為是內容的一個摘要,內容變更后摘要就變了。
?
GZIP壓縮
修改nginx.conf配置文件
Java代碼??
gzip?on;??gzip_min_length??4k;??gzip_buffers?????4?16k;??gzip_http_version?1.0;??gzip_proxied????????any;??#前端是squid的情況下要加此參數,否則squid上不緩存gzip文件??gzip_comp_level?2;??gzip_types???????text/plain?application/x-javascript?text/css?application/xml;??gzip_vary?on;?? 此處我們指定至少4k時才壓縮,如果數據太小壓縮沒有意義。??
?
?
到此整個商品詳情頁邏輯就介紹完了,一些細節和運維內容需要在實際開發中實際處理,無法做到面面俱到。
來源:http://jinnianshilongnian.iteye.com/blog/2188538
總結
以上是生活随笔為你收集整理的第七章 Web开发实战2——商品详情页的全部內容,希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔網站內容還不錯,歡迎將生活随笔推薦給好友。