Google Suggestions API

One of the main tenets of SEO is understanding the search climate for the keywords you are targeting. Understanding what the trends are, what people are searching for, in what volume and generating ideas from those results. An often overlooked API is Google Suggestions. It’s essentially tapping into Google’s search autocomplete feature.

We can call that feature independent of Google’s search bar and have it return a collection of autocomplete suggestions based on what other people are searching. These are valuable clues on your audience and can give insight on other keyword options to target.

Amazingly this is one of the easiest APIs to use because there isn’t much to it. No keys, no OAuth, no multiple calls, and few parameters to configure. It almost feels like a side project Google forgot about but keeps the lights on. There isn’t even any official documentation. Let’s dig in using Python before they turn it off!

Requirements and Assumptions

Starting the Script

First we install the fake_useragent module to use in our call header. This can help ward off light abuse detection. The API doesn’t have documented usage limits, but Google defenses are always up and if you abuse the API, you may get temporarily or permanently IP blocked. Note if you are using Google Colab, put an exclamation mark at the beginning of pip3, so !pip3.

pip3 install fake_useragent

Next we import our requests module we will use to make the API call. Then import JSON module as the response will be in JSON (there is an XML option). Lastly we import the user_agent module we just installed.

import requests
import json
from fake_useragent import UserAgent

Assign the Keyword

It’s time we assign our keyword/keyphrase to a variable. This is what Google suggestion will use to collect the autocomplete data. Since this variable is going into a URL we need to replace spaces with a plus (+) character or you can use %20, doesn’t matter.

keyword = "dog collars"
keyword.replace(" ", "+")

Build the API Call

The base URL is http://suggestqueries.google.com/complete/search.

Some parameters are as follows:

  • client or output = What client to mask as (Firefox, Chrome, Toolbar). This determines the format of the output and if you get extra data. If set to toolbar you get XML. If set to Firefox or Chrome you get JSON. If you set to Chrome you’ll also get some extra data (I’ll explore that later on).
  • hl = Country to use (us, uk , fr…)
  • gl = Language to use (en, es, fr…)
  • q = Your query

We’re going to keep it super simple and not include HL or GL parameters and set output to Firefox to get JSON response, but feel free to experiment.

url = "http://suggestqueries.google.com/complete/search?output=firefox&q=" + keyword

Make the API Call

Next we build a user agent for the API call. Then finally use the url from above and user agent we just generated to make the call and assign to response to the variable response. verify=false is to disable TLS certificate verification. This is important for requesting untusted sites, but Google is good and it can cause issues, so we disable it here.

ua = UserAgent()
headers = {"user-agent": ua.chrome}
response = requests.get(url, headers=headers, verify=False)

Process the API Response

If you choose output of Firefox you just get this:

["dog collars",["dog collars","dog collars amazon","dog collars etsy","dog collars and leashes","dog collars with name","dog collars personalized","dog collars leather","dog collars near me","dog collars walmart","dog collars for large dogs"],[],{"google:suggestsubtypes":[[433],[],[],[],[],[],[],[],[],[]]}]

If you use output of Chrome you’ll see this JSON response below. It includes some extra data like google:clientdata, google:suggestrelevance, google:suggesttype and google:verbatimrelevance, but I am unsure how to interpret them, so I’ll choose Firefox output. If anyone has insight on those, let me know!

["dog collars",["dog collars amazon","dog collars with name","dog collars etsy","dog collars and leashes","dog collars personalized","dog collars leather","dog collars near me","dog collars walmart"],["","","","","","","",""],[],{"google:clientdata":{"bpc":false,"tlw":false},"google:suggestrelevance":[601,600,555,554,553,552,551,550],"google:suggesttype":["QUERY","QUERY","QUERY","QUERY","QUERY","QUERY","QUERY","QUERY"],"google:verbatimrelevance":1300}]

Output the Response

Since we have out JSON response it’s time to load into a variable as a list so we can output it cleanly.

suggestions = json.loads(response.text)
for word in suggestions[1]:
  print(word)

Example output

google suggestions

Extra

Colab Notebook
https://colab.research.google.com/drive/18Nta_IO3pi-iu5eYxgGIfjjZ63PIJuKV

Inspiration
https://github.com/Leszek-Sieminski
https://leszeksieminski.me/r/keyword-exploration-googlesuggestqueriesr/

Extend the script
Go 3 levels deep for each initial single result and then count the frequency. Awesome!
https://github.com/omkom-web/scripts-seo/blob/master/gsuggest-distrib.py

Greg Bernhardt
Follow me

Leave a Reply