If you are one of those who want to have all the data available to develop your SEO, SEM or content strategy, extracting trends from Google Trends through Pytrends will open up a range of possibilities for you.
Table of Contents
What is Google Trends?
Google Trends is a completely free tool where you can analyze the interest of a specific word or query over a specific period of time, either globally or geolocated.
This tool presents samples of the requested data from the year 2004 to the present day, although it began to work a few years later, in 2006. If you want more information about the origin and the data it shows, you will find it here.
What is Pytrends?
Pytrends is an unofficial Google Trends API for Python. It allows us to extract data of all kinds related to the interest of users in a specific topic or query.
As in Google’s own tool, we can extract information at a global or geolocated level, in specific time periods and even based on our own categorization of queries.
Installation
To start with, we need to install pytrends in our Google Colab notebook and import the libraries we need.
1. #Instalamos pytrends 2. !pip install pytrends 3. 4. #Importamos las librerías que vamos a utilizar 5. import pandas as pd 6. from pytrends.request import TrendReq 7. pytrend = TrendReq()
When executing the previous code we will see how different packages and libraries are installed.
Extracting data with Pytrends
In order to continue advancing and to be able to extract all the data we want from Google Trends, we must take into account what we can call API methods.
Basically, these are parameters that we use to define what we want to do with the request we make and to obtain a response.
These allow us to know almost completely what can be done with Pytrends and the data that we can obtain.
- Interest Over Time
- Historical Hourly Interest
- Interest By Region
- Related Topics
- Related Queries
- Trending Searches
- Top Charts
- Suggestions
We will analyze later what we can do with each one and we will see the result through our notebook in Google Colab.
On the other hand, all of the above use certain parameters that are practically common among all of them. These parameters help us to specify and filter our search much more.
Getting started with Pytrends
In order to extract data from Google Trends with Python, we need several lines of code that include the parameters that we mentioned before.
Much of this information is available in the Pytrends documentation, but I’ll try to explain it in a simple way:
1. #We connect with Google Trends and pass some base parameters 2. pytrends = TrendReq(hl="es") 3. #We establish a list variable that includes the word or words that we want to analyze 4. word_list = ["Query we are going to carry out"] 5. #We add all the parameters we want to the request (filtered) 6. pytrends.build_payload(lista_palabras, cat=0, timeframe="today 6-m", geo="ES")
As you can see, it consists of 3 lines of code that, even if you don’t know anything about Python, you can easily understand.
TrendReq function
The TrendReq() function allows us to do a first filtering of the data that we are going to request. In this case, we only indicate the language to access Google Trends, but there are many more parameters that can be used:
1. pytrends = TrendReq(hl='en-US', tz=360, timeout=(10,25), proxies=['https://34.203.233.13:80',], retries=2, backoff_factor=0.1, requests_args={'verify':False})
- hl: Language we will use in Google Trends. As you can see in the example above, different language combinations can also be used depending on the request you make..
- tz: The time zone. In this case it works through CST.
- Timeout: We set a timeout for the request in case there are problems with the server.
- Proxies: In case many requests are made, proxies can be used to avoid Google blocking us. It consists of proxy + port.
- Retries: Number of attempts to connect to the server.
- Backoff_factor: A delay is created between attempts.
- Request_args: It allows us to add other types of parameters. For example, in order to avoid SSL errors on the request.
In the second line we establish a variable, where we can put one or more words to analyze.
Building the request
The third line is made up of much more specific parameters to make the request. Basically, we add here what elements this tool must take into account when requesting information.
1. pytrends.build_payload(words_list, cat=0, timeframe="today 6-m", geo="ES")
- Words_list: It will take into account the list of words that we have associated with the previous variable. It is mandatory to pass this parameter.
- Cat: I think this is something really interesting, since Google Trends classifies your queries based on a series of specific or thematic categories. These categories are referenced with specific numbers. Later we will see how we can get the most out of this parameter.
- Timeframe: We indicate the date from which we want to extract the data. In the case of the example, we would extract the data of the last 12 months, but we can also use all for a complete history or specific dates.
- Geo: Here we geolocate the results. We can use abbreviations like ES or even more specific areas.
- Other less used parameters: Parameters such as grpop can also be used, which establishes a filter based on the search (news, images, etc. can be used) or tz, to indicate the time offset.
What can we do with Pytrends?
Now comes the best part, where we can see all the potential that Pytrends has and all the data it offers us.
Depending on each project, very different data may be of interest, so we will only see how to apply only the data that can offer the most relevant information.
InterestOverTime
With this method we check the history of interest over a given period on one or more queries.
In this case we will use the pfizer query.
1. #We connect with Google Trends and pass some base parameters 2. pytrends = TrendReq(hl="es") 3. 4. #We establish a list variable that includes the word or words that we want to analyze 5. words_list = ["pfizer"] 6. 7. #We pass all the parameters we want to the request (filtered) 8. pytrends.build_payload(lista_palabras, cat=0, timeframe="today 12-m", geo="ES") 9. 10. #We run pytrends.interest_over_time() pytrends.interest_over_time()
Result:
As you can see, the image shows us a table with a score established by Google Trends for each of those days.
We can replicate the same thing to several queries in order to compare this data.
For the example, we will use the terms pfizer and moderna.
1. #We set several queries inside the list variable 2. words_list = ["pfizer","moderna"] 3. 4. #We pass all the parameters we want to the request (filtered) 5. pytrends.build_payload(lista_palabras, cat=0, timeframe="today 12-m", geo="ES") 6. 7. #We run pytrends.interest_over_time() 8. pytrends.interest_over_time()
Result:
We can use this data to understand the trend of a keyword during a certain period of time or even to compare trends between several keywords. If you are working with seasonal products or want to compare brand interests between you and your competitors, you will have a lot of information.
We can use the matplotlib library to pass this data to a graph and see everything in a much more visual way.
1. pytrends.interest_over_time() 2. graphic = pytrends.interest_over_time() 3. graphic.plot(figsize=(12, 10), y=lista_palabras)
Result:
Interest By Region
Thanks to interest_by_region we can know the trend of one or several words depending on its geographical area.
1. pytrends.interest_by_region(resolution="COUNTRY")
Result:
If you work with local SEO projects, this option can really come in handy to find out search trends between communities or even to compare the trend of a product or brand based on the place.
On the other hand, if you work with a project that is very focused on one region and you want to expand it to the rest of Spain, with this data you can check the knowledge of your brand or product in the rest of the country.
Related Topics
Perhaps this is one of the options with the most potential from Pytrends. Thanks to it we can discover the topics most related to a specific word or entity.
I recommend that you only use one word in this request to obtain more precise and fully related data.
When using a single word we can assign a variable to our output and use the .get method to add the source word and specify the format of the data that is extracted.
Thanks to Pytrends we can obtain the entities with a growing volume in the specified timeframe up to the top of the most relevant entities during that period.
For the example, we will use the query cruises.
Topics with increasing volume during the established period:
1. #We indicate the query that we want to carry out
2. words_list = ["cruises"]
3.
4. #We pass all the parameters we want to the request (filtered)
5. pytrends.build_payload(words_list, cat=0, timeframe="today 12-m", geo="ES")
6.
7. #We run pytrends.related_topics()
8. pytrends.related_topics()
9.
10. #We indicate that we want the topics with an increasing volume
11. #during the established period of time
12. trend_analysis = pytrends.related_topics()
13. trend_analysis.get("cruises").get("rising")
Result:
Top topics during the established period:
1. #We run pytrends.related_topics()
2. pytrends.related_topics()
3.
4. #We indicate that we want the top of topics in that period
5. trend_analysis = pytrends.related_topics()
6. trend_analysis.get("cruceros").get("top")
Result:
Thanks to the related topics we can see which entities Google is associating with the word that we have indicated. This can help us to enrich our content and try to improve its context.
On the other hand, in the related queries, we find variations of the word that we have established as the origin.
Both this data and those that we will see in the next section are directly related to the blocks that are shown in Google Trends called “Related Topics” and “Related Queries”.
Apart from all the information that is extracted in the query that we made above, we can see the topic_mid, which would be the id associated with the entity displayed within Google’s Knowledge Graph.
Although it may not seem like a very relevant piece of information, it allows us to know what is included in the Knowledge Graph that Google has associated with this entity.
If we use the following url structure and add the entity id we have obtained, we can see the following:
https://www.google.com/search?q=love&kponly&kgmid=/g/11c7vyb8c5
If you want more information on this, I recommend that you take a look at this article by Koray Tuğberk on the Knowledge Graph.
Related Queries
As its name suggests, we can extract related queries from the original word.
Analyzing this data together with the data extracted from related_topics allows us to know how related queries totally change the entities with which they are associated.
Like the previous point, this method offers us a large amount of data that we can use in the SEO strategy of a project, either to enrich the content or assess the incorporation of new sections or pages on the web that talk about these topics.
Continuing with the previous example, we can also specify what type of data we want, from the queries that are growing the most during the specified period or the most relevant.
1. #We run pytrends.related_queries()
2. pytrends.related_queries()
3.
4. #We indicate we want queries to increase during the established period
6. trend_analysis = pytrends.related_queries()
7. trend_analysis.get("cruceros").get("rising")
Result:
Analyzing the result of this query, several conclusions can be drawn. In this case, we can interpret how the query “2022 all-inclusive cruises” is a trend, which can give us an idea of where to focus content creation or our SEO strategy.
Analysis of the top of related queries for the period of time that we have established:
1. #We run pytrends.related_queries()
2. pytrends.related_queries()
3.
4. #We indicate we want the top queries during the established period
6. analisis_tendencia = pytrends.related_queries()
7. analisis_tendencia.get("cruceros").get("top")
Result:
Trending Searches
This method allows to extract the queries in trend for a certain geographical area. Unfortunately, it only works for certain countries, among which Spain is not.
If you work with an international project or a news website in another language, this data can be really interesting to cover news or events before other media or websites.
It can be used in the following way:
1. pytrends.trending_searches(pn="country from which we want to extract data")
Suggestions
The suggestions method offers us the possibility of obtaining a series of suggested queries for the term we have chosen.
1. pytrends.suggestions("cruises")
2. data_frame = pytrends.suggestions("cruises")
3. data_frame = pd.DataFrame(data_frame)
data_frame
For this case we have converted the output into a data frame that allows us to visualize the extracted data much better, since the original output is difficult to interpret.
Result:
Although there are few suggestions, we can get an idea of what this data offers us. As you can see, we get a series of suggested queries, with the related entity and even the Google entity id.
If we analyze the data, we can see how Google takes into account the first brand listed, relating it directly to the most general word in that sector.
On the other hand, we can also see which is the entity or query that Google most relates to the source word or other queries that we can add to our content to cover all kinds of topics.
What possibilities do the categories established by Google Trends offer us?
As I mentioned before, Google Trends establishes a series of categories so that we can further refine the extracted data.
These categories are the same ones that we can see when searching for a term on your website, but in order to use them through Pytrends, we must know their reference number.
In this github you have a list of all the categories used by Google Trends, also including a link to their origin.
In order to understand the potential of specifying a category, I think it is best to see it with an example.
If, for example, we establish the term “panda” as the source word or query and refer to category number 66 (Animals), we obtain the following result:
1. #USE OF CATEGORIES 2. #Using category 66 associated with animals 3. pytrends = TrendReq(hl="es") 4. words_list = ["panda"] 5. pytrends.build_payload(words_list, cat=66, timeframe="today 12-m", geo="ES") 6. pytrends.related_queries()
Result:
The request refines the query and provides us with trend data on the source word within that category. This prevents the inclusion of other terms related, for example, to the mythical car “seat panda”.
On the other hand, if we use category 47, which is associated with vehicles, the data we obtain is the following, much more specific and related to the vehicle.
This allows us to know how Google takes into account a given query or word within a specific category and what importance it gives to it based on the search intention.
How can we use the information obtained in the data extraction with Pytrends?
Pytrends offers us a large number of possibilities when it comes to extracting data, but they are useless if there is no subsequent objective. I thought I’d comment on some things we can do with them.
- Analysis of user trends and interests that we can take advantage of: In any project we work on, we will be interested in knowing the interests of the users we are targeting and their trends over time. This will allow us to focus our strategy on new trends or reinforce the work already done.
- Compare brand or product interests: Although we have already discussed this, we can analyze the interest of users in our brand and compare it with that of our competitors. In the same way we can do it with our own products, etc.
- Analyze or compare the interest of a certain service or product in a specific place: By being able to geolocate the requests, we can check the knowledge or interest of a brand or product in a specific place. Very useful for local SEO.
- Check the most relevant entities for a query: Having this information will help us when it comes to improving the semantics of our content, but also in the focus of our SEO strategy.
- Have up-to-date and current data related to our sector or product: Thanks to this, we can cover current trends in our sector or create new URLs that attack new intentions and queries. This part seems very important to me for news websites.
- Understand how economic or health events affect the types of queries and user intentions: This is a reflection of all the changes that have been seen in intentions and searches during the pandemic.
- Create our own dashboard where we monitor all kinds of data: As you may have imagined, we can export all this data to external tools and generate a personalized dashboard based on the needs of each one.
There are a large number of options when applying this data. I am sure that at this moment you have already imagined all kinds of ways to combine these and obtain valuable and relevant information for your project.
Alternatives to Pytrends
For those who don’t want to see the code up close, I’ve thought of commenting on several alternatives that I’ve seen doing a little research.
On the one hand, the Keyword Tool has a section in its tool called “Google Trends”, which allows you to know some data about a given query.
On the other hand, these days I have also seen that Andrew Charlton, a well-known SEO in the United Kingdom, has shared a way to extract the daily trending topics without using any code, directly to a Google Sheets.
Conclusion
As you may have seen, extracting data with Pytrends in Python, directly from Google Trends, gives us infinite possibilities. Of course, we can always go to the tool itself and check the data, but if you have a very large website or you don’t want to waste that time, you can use Pytrends to get the information in the shortest time possible.
I hope you enjoyed the article and that you can use this Google Trends data in your projects.
See you in the serps!
Other links of interest:
