Information sourcing - An underrated skill everyone should know

Ok, so what did I learn? With the ever-increasing pace at which content is created, SEO and search engines trying to make search more intuitive[4], it's becoming increasingly more and more difficult to separate high-quality information from the chaff.

🖖
Hi, Thanks for reading! I am on the Fediverse! If you like the way I explain stuff follow me if you want to. @nish@social.nishtahir.com

During a recent conversation, I was asked a random question that appeared in a poll on a friend's social media feed - "What is the most Googled question?". I found it to be a pretty thought-provoking question that sparked a fun discussion.

While we couldn't agree on an answer, the poll author did offer an answer: "What is my IP?".

Question Poll

I found this to be quite unintuitive. I would have guessed that non-technical internet users would exceed technical audiences. So I would expect that the most Googled question would be something way less technical. The poll did not provide any citations or attribution to any primary source nor did they make any claims of having access to Google's search data[1]. As a result, the data was presented as being a statement of fact. This started me down a rabbit hole of verifying the factuality of the claims.

Google Searching a source

I began my hunt on Google Search. The question is about how people use Google so it seemed like a good place to begin. After entering my search query and getting a list of search results, the top result corroborates the claim!

Google search page

I was presented with a nicely formatted table with "What is my IP" in first place with a search volume of 1.16 million monthly searches. As you probably have noticed, while this may appear to be content from Google, it is just rendered content sourced from the search result. So we must inspect the page to see how they arrived at their numbers.

Fortunately, towards the bottom of the page, the authors explain how they arrived at their numbers.

We analyzed over 4 billion search terms to identify the questions that people search for the most on Google.

...

Specifically, we searched for keywords containing a question, like "where" and "how"). We then manually curated the list to remove duplicate questions (ie. the same question phrased slightly differently).

Finally, we looked up the search volume for each term using third party SEO software. And ranked the questions based on which terms generate the most monthly searches in Google[2]

Hang on... They are the primary source!? They never attributed any information presented back to Google, so my interpretation is that this is an estimate that is, unfortunately, presented as authoritative.

Other search results followed a similar pattern. Interestingly, even when they agreed on the search terms, they often wildly disagreed on the search volume. This source claimed a 4.1 and 3.1 million monthly search volume for "what is my ip" and "what is my ip address" respectively.

This third page did attempt to provide some context on the data that they used, as well as the processing steps that they took before tabulating their results[3].

The values under the ‘Global Search Volume’ column are monthly averages calculated using the data collected over the last 6 months. The entire list of keywords has been generated using Mondovo’s Free Keyword Research Tool ...
...
To note, we did try to filter out some duplicates and really close variants. There were also some TV shows, movies and songs that we knew about and which we duly removed, like “how i met your mother”, “what do you mean”, and “how to get away with murder”.

Interestingly, their results have "what is my ip" and "what is my ip address" also appearing separately with a 3.3 and 0.45 million monthly search volume, which also suggests that this is based on their own data.

Conclusion

Ok, so what did I learn? With the ever-increasing pace at which content is created, SEO and search engines trying to make search more intuitive[4], it's becoming increasingly more and more difficult to separate high-quality information from the chaff. While going down this rabbit hole, I noticed quite a few articles, and blog posts repeating the original claims verbatim. Unfortunately, none of them cited any of their sources, so what I witnessed may have been the formation of, if not the propagation of an internet myth. Reflecting on the experience, I'm reminded of Sturgeon's law[5]

ninety percent of everything is crap

The ability to source information in this landscape is an underrated but extremely relevant skill that every engineer should invest some time into. If you do not feel comfortable with this or know where to start, I suggest starting with the basics

  1. What is a primary source?
  2. How did they arrive at their conclusion?

Then try digging into a topic that might seem benign and/or obscure. For example, does it really take 10000 hours to master a topic?

Back to the original question. Finding an authoritative primary source on this question has been tough to pin down. There are some datasets publicly available such as the Google Trends dataset[6], but it does not rank the data in a way that is easily accessible to the public. Querying it wonder if there are better sources or methods of answering this that I might not be aware of. So if anyone has an approach to getting better data I'd be interested in learning more.

So... What is the most Google'd question?

References


  1. I did not include a direct link to the content or the original post to maintain the authors' privacy. My goal is to focus on the subject and any learnings, not to point fingers or attack anyone. So if you choose to offer your own commentary, I only ask that you be kind and respectful. ↩︎

  2. Dean, B. (2023) “100 most asked questions on Google in 2023,” Exploding Topics, 22 September. Available at: https://explodingtopics.com/blog/top-google-questions (Accessed: January 13, 2024). ↩︎

  3. Kaushal, N. (2022) “100 most asked questions on Google in 2024 - PageTraffic,” PageTraffic Blog - All thing Search, Content & Social. PageTraffic, 13 June. Available at: https://www.pagetraffic.com/blog/most-asked-questions-on-google/ (Accessed: January 13, 2024). ↩︎

  4. Brereton, D. (2022) Google search is dying, DKB Blog. Available at: https://dkb.blog/p/google-search-is-dying (Accessed: January 13, 2024). ↩︎

  5. Wikipedia contributors (2023) Sturgeon’s law, Wikipedia, The Free Encyclopedia. Available at: https://en.wikipedia.org/w/index.php?title=Sturgeon's_law&oldid=1179711471. ↩︎

  6. Get started with the Google Trends dataset (no date) Google.com. Available at: https://support.google.com/trends/answer/12764470?hl=en (Accessed: January 13, 2024). ↩︎

Subscribe to Another Dev's Two Cents

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe