因為魔方公寓的評論信息是在一個單獨的div中, 需要模擬鼠標(biāo)滾動才能拿到評論信息, 并且每次只能拿到在頁面顯示的內(nèi)容, 頁面沒有顯示的取出來的是空. 因此只能在每次滾動后取值保存.

代碼:

import time import win32api, win32con from win32api import GetSystemMetrics from selenium.webdriver.common.action_chains import ActionChains from selenium import webdriver from selenium.webdriver.chrome.options import Options from win32con import SM_CXSCREEN, SM_CYSCREENoptions = Options() # options.add_argument("--headless") driver = webdriver.Chrome(executable_path=r"D:\tools\chromedriver.exe", options=options) url = "https://www.52mf.com.cn/shopDetails/8069" headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36","Referer":"https://www.52mf.com.cn/" } driver.get(url=url) for i in range(3):driver.execute_script("window.scrollBy(0,200)")time.sleep(0.2) all_ping = driver.find_element_by_css_selector("#anchor6 > div.all-comment.cursor-pointer") all_ping.click() time.sleep(1) # 找到div的位置, 并將鼠標(biāo)移動到div上 x = GetSystemMetrics(SM_CXSCREEN) y = GetSystemMetrics(SM_CYSCREEN) driver.maximize_window() ActionChains(driver).move_by_offset(x/2,y/2) # 設(shè)置鼠標(biāo)滾動次數(shù) for i in range(260):# 模擬鼠標(biāo)滾動win32api.mouse_event(win32con.MOUSEEVENTF_WHEEL, 0, 0, -448)time.sleep(0.2)ping_list = driver.find_elements_by_xpath('//div[@id="modal-root"]//ul/li/p')time.sleep(0.2)for ping in ping_list:ping_str = ping.text.strip()# 頁面沒顯示的取出的為空, 判斷取出內(nèi)容是否為空, 為空跳過 不為空寫入文件if ping_str == "":passelse:with open("ping.txt","a",encoding="utf8") as fp:fp.write(ping_str + "\n")time.sleep(0.1) driver.close()

總結(jié)

以上是生活随笔為你收集整理的selenium模拟 + 鼠标滚动爬取魔方公寓租房评论信息的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。