Summary: SerpApi is a third-party service that captures and parses search engine results pages (SERPs) and returns structured data (typically JSON). It can support keyword research by revealing SERP intent, query refinements, “People also ask” questions, related searches, and competitor page sets. It does not provide keyword search volume on its own.
Table of contents
- What SerpApi is (and what it isn’t)
- When SerpApi is useful in keyword research
- Quick start (Python)
- Core parameters for keyword research (hl, gl, location, device)
- Extracting keyword signals from the SERP
- Pagination: expanding competitor and intent coverage
- Keyword clustering workflow (practical approach)
- Scaling responsibly: rate limits, caching, and storage
- Common pitfalls
- FAQs
What SerpApi is (and what it isn’t)
SerpApi provides programme access to SERP data by collecting results and returning a structured representation. In practice, that means an application can request a “SERP snapshot” for a query and receive parsed modules such as organic results, “People also ask”, related searches, and other features (where present).
Important clarification: SerpApi is not an official Google product or an official Google Search API. It is a third-party SERP extraction service.
Also important: SerpApi is not a keyword volume tool. It will not provide Google Ads keyword volumes unless another data source is used for that purpose.
Compliance note (non-legal guidance): Automated querying of search engines can raise terms-of-service, policy, and legal considerations. Implementations should be designed to operate responsibly (e.g., rate limiting, caching, minimal collection, and appropriate usage review by internal stakeholders).
When SerpApi is useful in keyword research
SerpApi is most valuable when the goal is to learn what the SERP reveals, rather than to estimate volume:
- Intent validation: identifying whether the SERP is informational, commercial, navigational, or local.
- Query expansion: collecting related searches and “People also ask” questions as modifiers and subtopics.
- Competitor discovery: building a list of ranking URLs and domains for a query, then analysing patterns.
- SERP feature mapping: spotting features that change content requirements (e.g., local packs, shopping, videos).
- Localisation research: comparing results across countries/languages/locations and devices.
Quick start (Python)
This example uses SerpApi’s Python client to retrieve a Google SERP response and print key modules.
# pip install google-search-results
from serpapi import GoogleSearch
params = {
"engine": "google",
"q": "ambient sound design tools",
"api_key": "YOUR_API_KEY",
"hl": "en", # interface language
"gl": "uk", # country code
# "location": "London, England, United Kingdom", # optional
# "device": "desktop", # optional (depends on engine support)
}
search = GoogleSearch(params)
results = search.get_dict()
organic = results.get("organic_results", [])
paa = results.get("related_questions", []) # often used for "People also ask"
related = results.get("related_searches", [])
print("Organic results:", len(organic))
print("PAA questions:", len(paa))
print("Related searches:", len(related))Tip: For keyword research, it is usually more helpful to persist the raw JSON response (for auditing and reproducibility) and extract a smaller “signals” dataset for analysis.
Core parameters for keyword research (hl, gl, location, device)
Keyword research frequently fails when SERPs are pulled without controlling for geography and language. At minimum, keep these parameters explicit:
- hl: interface language (e.g.,
en,pt,es). - gl: country code to influence results (e.g.,
uk,br,co). - location: city/region-level localisation (useful when local packs appear or for regional intent).
- device: device context (desktop vs mobile) can change SERP layouts and feature prevalence.
Practical workflow: Use a consistent default (e.g., hl=en, gl=uk) and only vary one dimension at a time during comparisons (location OR language OR device). This makes differences explainable.
Extracting keyword signals from the SERP
A SerpApi response can be mined for multiple research signals. The exact keys can vary by engine and by which SERP features appear, so extraction should be defensive (i.e., treat fields as optional).
1) Related searches (query expansion)
Related searches are a strong source of modifiers and adjacent topics. They can also be used to form cluster candidates.
def extract_related_searches(results: dict) -> list[str]:
related = results.get("related_searches", []) or []
out = []
for item in related:
q = item.get("query") or item.get("title")
if q:
out.append(q.strip())
return out2) “People also ask” (question-based intent)
Questions often map cleanly to FAQ sections, supporting information architecture and internal linking planning.
def extract_paa_questions(results: dict) -> list[str]:
paa = results.get("related_questions", []) or []
out = []
for item in paa:
q = item.get("question")
if q:
out.append(q.strip())
return out3) Organic results (competitor set + page patterns)
Collect URLs, titles, and snippets to identify page types, content formats, and repeated subtopics.
from urllib.parse import urlparse
def extract_organic(results: dict) -> list[dict]:
organic = results.get("organic_results", []) or []
out = []
for r in organic:
link = r.get("link")
title = r.get("title")
snippet = r.get("snippet")
if link:
domain = urlparse(link).netloc.replace("www.", "")
out.append({
"title": title,
"link": link,
"domain": domain,
"snippet": snippet,
"position": r.get("position"),
})
return outPagination: expanding competitor and intent coverage
Many keyword research tasks benefit from collecting more than the first page of results, especially when building competitor sets or when SERP intent is mixed. Pagination commonly uses a start parameter (e.g., 0, 10, 20…).
from serpapi import GoogleSearch
def fetch_pages(query: str, pages: int = 3, page_size: int = 10):
all_results = []
for i in range(pages):
params = {
"engine": "google",
"q": query,
"api_key": "YOUR_API_KEY",
"hl": "en",
"gl": "uk",
"start": i * page_size,
}
results = GoogleSearch(params).get_dict()
all_results.append(results)
return all_resultsPractical note: Pagination increases cost and request volume. It should be used selectively (e.g., for priority topics, not for every long-tail variation).
Keyword clustering workflow (practical approach)
SerpApi can support clustering by comparing SERP similarity across queries. A practical approach is to compare overlap of ranking domains or URLs and then group queries that share a high proportion of results.
Rule of thumb: A threshold such as “60%+ overlap” can work as a starting heuristic, but it is not universal. Some topics naturally produce more diverse SERPs, especially localised or news-driven queries.
Simple overlap example (domain overlap)
def domain_set(results: dict) -> set[str]:
items = results.get("organic_results", []) or []
out = set()
for r in items:
link = r.get("link")
if link:
out.add(urlparse(link).netloc.replace("www.", ""))
return out
def overlap(a: set[str], b: set[str]) -> float:
if not a or not b:
return 0.0
return len(a & b) / min(len(a), len(b))Suggested workflow:
- Choose a small set of seed queries per topic.
- Pull one SERP per query using consistent location/language parameters.
- Compare domain overlap (or URL overlap for stricter matching).
- Cluster queries above the chosen threshold.
- Assign a primary keyword per cluster based on relevance and SERP intent (not solely on wording similarity).
Scaling responsibly: rate limits, caching, and storage
At scale, keyword research runs can become expensive and fragile without guardrails.
- Rate limiting: implement throttling and retries with backoff. Avoid spikes in request volume.
- Caching: store responses keyed by query + parameters (q, hl, gl, location, device, start) and reuse within a reasonable TTL.
- Storage: persist raw JSON for auditability, then derive slim datasets (CSV/Parquet) for analysis.
- Change tracking: SERPs change; record timestamps and parameter sets to interpret differences correctly.
Minimal caching example:
import json
import hashlib
from pathlib import Path
from serpapi import GoogleSearch
CACHE_DIR = Path(".serp_cache")
CACHE_DIR.mkdir(exist_ok=True)
def cache_key(params: dict) -> str:
canonical = json.dumps(params, sort_keys=True, ensure_ascii=False)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def cached_search(params: dict) -> dict:
key = cache_key(params)
path = CACHE_DIR / f"{key}.json"
if path.exists():
return json.loads(path.read_text(encoding="utf-8"))
results = GoogleSearch(params).get_dict()
path.write_text(json.dumps(results, ensure_ascii=False), encoding="utf-8")
return resultsCommon pitfalls
- Uncontrolled localisation: mixing results from different regions/languages can invalidate comparisons.
- Assuming fields always exist: SERP features appear conditionally; extraction should treat fields as optional.
- Over-collecting: requesting multiple pages for every query inflates cost without necessarily improving decisions.
- Confusing SERP signals with volume: SERP patterns indicate intent and competition, not demand size.
- No audit trail: without timestamps and parameters, SERP differences become hard to explain later.
FAQs
Is SerpApi an official Google API?
No. SerpApi is a third-party service that collects and parses SERP data and returns a structured representation.
Does SerpApi provide keyword search volume?
No. SerpApi provides SERP data rather than keyword volume metrics. Search volumes require a separate data source.
Which SerpApi parameters matter most for keyword research?
Localisation parameters typically have the biggest impact: hl (language), gl (country), and location (regional targeting). Device context can also affect SERP composition.
What is the best way to use “People also ask” data?
Question extraction can support topic coverage, content outlines, and FAQ planning. Questions can also reveal sub-intents that deserve dedicated sections or separate cluster pages.
How can SERP overlap help with keyword clustering?
Queries that produce highly similar SERPs often share the same underlying intent. Comparing domain or URL overlap provides a pragmatic clustering signal, particularly when combined with manual intent checks.
What should be considered before scaling SerpApi usage?
Request volume, cost, stability, caching strategy, and compliance considerations should be addressed. Implementations typically benefit from rate limiting, retries with backoff, and response caching.
