黃聰：Python+NLTK自然語(yǔ)言處理學(xué)習(xí)（一）：環(huán)境搭建

http://www.cnblogs.com/huangcong/archive/2011/08/29/2157437.html

安裝NLTK可能出現(xiàn)的問(wèn)題：

1. pip install ntlk

2. 如果遇到缺少stopwords報(bào)錯(cuò)如下：（http://johnlaudun.org/20130126-nltk-stopwords/）

LookupError:

**********************************************************************

Resource u'corpora/stopwords' not found. Please use the

NLTK Downloader to obtain the resource: >>> nltk.download()

Searched in:

- 'C:\\Users\\Tree/nltk_data'

- 'C:\\nltk_data'

- 'D:\\nltk_data'

- 'E:\\nltk_data'

- 'F:\\Program Files (x86)\\python\\nltk_data'

- 'F:\\Program Files (x86)\\python\\lib\\nltk_data'

- 'C:\\Users\\Tree\\AppData\\Roaming\\nltk_data'

**********************************************************************

則有一下輸入：

In[3]: import nltk

In[4]: nltk.download()

showing info http://www.nltk.org/nltk_data/

彈出窗口：

選擇Corpora 然后找到stopword list確認(rèn)，刷新

Out[4]: True

3.如果遇到缺少punkt報(bào)錯(cuò)如下：

LookupError:

**********************************************************************

Resource u'tokenizers/punkt/english.pickle' not found. Please

use the NLTK Downloader to obtain the resource:

>>>nltk.download()

Searched in:

- 'C:\\Users\\Tree/nltk_data'

- 'C:\\nltk_data'

- 'D:\\nltk_data'

- 'E:\\nltk_data'

- 'F:\\Program Files (x86)\\python\\nltk_data'

- 'F:\\Program Files (x86)\\python\\lib\\nltk_data'

- 'C:\\Users\\Tree\\AppData\\Roaming\\nltk_data'

**********************************************************************

解決方法

In[5]: nltk.download('punkt')

[nltk_data] Downloading package punkt to

[nltk_data] C:\Users\Tree\AppData\Roaming\nltk_data...

[nltk_data] Unzipping tokenizers\punkt.zip.

Out[5]: True

文章：http://www.52nlp.cn/%E5%A6%82%E4%BD%95%E8%AE%A1%E7%AE%97%E4%B8%A4%E4%B8%AA%E6%96%87%E6%A1%A3%E7%9A%84%E7%9B%B8%E4%BC%BC%E5%BA%A6%E4%B8%89 文章： http://www.52nlp.cn/%E5%A6%82%E4%BD%95%E8%AE%A1%E7%AE%97%E4%B8%A4%E4%B8%AA%E6%96%87%E6%A1%A3%E7%9A%84%E7%9B%B8%E4%BC%BC%E5%BA%A6%E4%B8%89

詳細(xì)講述了如何使用NLTK進(jìn)行英文分詞、去除停用詞、詞干化、訓(xùn)練LSI、等等文本預(yù)處理的步驟。

在使用sumy demo時(shí)候出錯(cuò)：

C:\Python27\python.exe D:/Python/jieba/demo/sklearn/sumy_demo1.py

Traceback (most recent call last):

File "D:/Python/jieba/demo/sklearn/sumy_demo1.py", line 20, in

parser = HtmlParser.from_url(url, Tokenizer(LANGUAGE))

File "C:\Python27\lib\site-packages\sumy\nlp\tokenizers.py", line 33, in __init__

self._sentence_tokenizer = self._sentence_tokenizer(tokenizer_language)

File "C:\Python27\lib\site-packages\sumy\nlp\tokenizers.py", line 45, in _sentence_tokenizer

"NLTK tokenizers are missing. Download them by following command: "

LookupError: NLTK tokenizers are missing. Download them by following command: python -c "import nltk; nltk.download('punkt')"

總結(jié)

以上是生活随笔為你收集整理的python去停用词用nltk_【NLTK】安装和使用NLTK分词和去停词的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。