Tutorial · Python · Twitter API · CSV
How to obtain and store tweets on a topic in a CSV document
This tutorial explains how to collect tweets on a specific topic and store them automatically in a file .csv for further analysis.
Explanatory video
If you need help, you can follow the complete process in the following explanatory video.
1. Objective
In this project we are going to generate a small investigation from the point of view of Big Data and social networks. Exactly, we have used Twitter as a textual source, a social network in which data can be collected more easily and, above all, immediately.
This allows establishing temporal correlations between the evolution of the virus and opinion of society on it.
This platform allows the sending of short plain text messages, with a maximum 280 characters. These messages, called tweets, can be captured using the API provided by the social network itself.
In this example a Python script will be implemented that continuously stores all tweets published in Spanish about the Coronavirus.
2. Context
The chosen theme of the project has been the disease Coronavirus or COVID-19 . This phenomenon has had an enormous social, health and economic impact, becoming an issue recurring and continuously present in media and social networks.
Following this premise, the objective is to develop a Python script that stores continuously tweets in Spanish about Coronavirus. With this dataset you can analyze the different perceptions of users taking into account the evolution of the virus and its consequences.
Furthermore, given that we live in a highly digitalized society, where social networks have become In a common space of public opinion, it is especially interesting to study this type of data.
Questions that can be investigated
- How has the virus evolved in Spain by autonomous communities and in total?
- How has the virus evolved worldwide?
- What impact have the positive or negative changes of the virus had on social opinion?
- How do people feel about the progress of the virus at the regional and global levels?
3. Implementation
The program used can be consulted and downloaded from the URL https://github.com/al118345/Tweepy_Example .
3.1 Get the API Keys and Tokens for your Twitter account
The keys necessary to use this program are the following:
- Consumer Key (API Key)
- Consumer Secret (API Secret)
- Access Token
- Access Token Secret
To do this, we will have to access the Twitter developer center at: https://developer.x.com/
We click on Create new App and we fill in the basic fields of the application:
- Name: name that we will give to the application or process.
- Description: brief description of the purpose.
- Website: website where the keys will be used.
The result will be shown as in the following images:
As can be seen, the data can be generic. The important thing is to complete the process to gain developer access and get the API Key.
Entering the tab Keys and Access Token we will be able to see both the API Key like the API Secret.
To get the token, you must scroll to the bottom of the page and press the corresponding button. Once generated, we will have all the necessary data to connect with Twitter.
Then you will have to modify the file authenticate_example.py:
Installation on Windows
Installation on Linux
3.2 Run the program
The next step is to run the main program located in the file Tweepy_example_2022.py.
This file contains the following query:
As can be seen, the program searches for all tweets published between start_date and end_date that contain the expression pfizer symptoms.
The complete program can be consulted and downloaded from: https://github.com/al118345/Tweepy_Example/blob/master/Tweepy_ejemplo_2022.py