1.Scrapy安裝問題

一開始是按照官方文檔上直接用pip安裝的，創(chuàng)建項(xiàng)目的時候并沒有報錯，

然而在運(yùn)行?scrapy crawl dmoz?的時候錯誤百粗/(ㄒoㄒ)/~~比如：

ImportError: No module named _cffi_backend

Unhandled error in Deferred 等等,發(fā)現(xiàn)是依賴包好多沒有裝上，就去百度安裝各種包，
有好多大神把這些都總結(jié)好了：膜拜！^_^

?http://blog.csdn.net/niying/article/details/27103081

http://blog.csdn.net/pleasecallmewhy/article/details/19354723

2.沒有得到數(shù)據(jù)，發(fā)現(xiàn)是拼寫錯誤.

E:\tutorial>scrapy crawl dmoz 2015-10-30 13:44:02 [scrapy] INFO: Scrapy 1.0.3 started (bot: tutorial) 2015-10-30 13:44:02 [scrapy] INFO: Optional features available: ssl, http11 2015-10-30 13:44:02 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tu torial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'BOT_NAME': 'tutorial'}2015-10-30 13:44:02 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsol e, LogStats, CoreStats, SpiderState 2015-10-30 13:44:03 [scrapy] INFO: Enabled downloader middlewares: HttpAuthMiddl eware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultH eadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMidd leware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats 2015-10-30 13:44:03 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddlewa re, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware 2015-10-30 13:44:03 [scrapy] INFO: Enabled item pipelines: 2015-10-30 13:44:03 [scrapy] INFO: Spider opened 2015-10-30 13:44:03 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 i tems (at 0 items/min) 2015-10-30 13:44:03 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023 2015-10-30 13:44:03 [scrapy] INFO: Closing spider (finished) 2015-10-30 13:44:03 [scrapy] INFO: Dumping Scrapy stats: {'finish_reason': 'finished','finish_time': datetime.datetime(2015, 10, 30, 5, 44, 3, 292000),'log_count/DEBUG': 1,'log_count/INFO': 7,'start_time': datetime.datetime(2015, 10, 30, 5, 44, 3, 282000)} 2015-10-30 13:44:03 [scrapy] INFO: Spider closed (finished)

?在spiders目錄下的dmoz_spiders.py文件中將start_urls寫成了start_url ，哎，╮(╯▽╰)╭

1 start_urls = [ 2 "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/", 3 "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/" 4 ]

?

轉(zhuǎn)載于:https://www.cnblogs.com/RoundGirl/p/4920426.html

總結(jié)

以上是生活随笔為你收集整理的Scrapy--1安装和运行的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。