Comment high-frequency word directional crawler method

• Main Authors: CHEN WEI, ZHOU ZIXUAN, GE YUXUAN, DING MEI, ZHANG TIANHANG, BAO JUNZE
• Format: Patent
• Language: Chinese; English
• Published: 09.01.2024
• Subjects: CALCULATING; COMPUTING; COUNTING; DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FORADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORYOR FORECASTING PURPOSES; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE,COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTINGPURPOSES, NOT OTHERWISE PROVIDED FOR

The invention discloses a comment high-frequency word directional crawler method, which aims at a complex json encryption mode at a webpage end, adopts de-automation mark setting, avoids monitoring of an automatic tool, uses a Selenium automatic tool to carry out login waiting, creatively writes a dejection function for slide block detection, and improves the efficiency and the reliability of the comment high-frequency word directional crawler method. The method comprises the steps that firstly, webpage elements are analyzed through the Xpath technology, user behavior simulation is conducted by changing a page rolling mode, comments are successfully crawled, finally, high-frequency word statistics and extraction can be conducted on crawled comment information in combination with the jieba word segmentation technology, and public opinion tendencies of dispute hot events can be deeply mined. The comment high-frequency word directional crawler method has high practicability and innovativeness, comment informatio