Following the previous article , which shared how to understand users from comments from the perspective of theory and tool recommendation, in this article, I will present the acquisition of user comments to the content analysis from the beginning to the end, in order to provide insights into the analysis of comments. Students who need more detailed help. If you haven't read the previous article, you can click on my personal homepage to view it. In order to make the writing more convenient, in this article, I will use the tourism industry with rich comments and loose anti-climbing measures as an example. If I were the manager of Feng Xiaogang Film Commune, I started my own tourist research journey in the face of massive tourist reviews
from various OTA platforms. First, the acquisition of comment content According to browsing, online reviews of scenic spots are concentrated on several top OTA websites, namely Ctrip, Qunar, Tuniu, and Meituan. The first step is to collect the website of the scenic spot on this b2b data platform, and collect the comment content separately. Take Ctrip as an example. The second step is to open Python, write code (there are also many codes that can be called directly on the Internet), configure the range of pages you need to crawl and the fields you expect to crawl, and start collecting. If you need to comment on the source code crawled by Ctrip, you can leave a message in the comment area. If it is a non-programming method, here is an example of Houyi Collector. After installation, enter the URL,
click Smart Collection, wait for the page to load, and edit at the bottom of the page to remove unnecessary fields. Only the user id, comment content and comments are retained in the text. time. After the crawling is completed, an excel table is generated, and when you open the table, you can see that the collection has been completed, and a total of 3900 pieces of data have been obtained. So far, the data collection stage is over. Second, the second step, preprocessing comment data Word segmentation and stop word removal. Open the Rost cm6 software and find that it only supports content in text format, so export the excel sheet to text format. Then open the word segmentation window in the interface