Out of all the things that happen during website optimization, traffic fluctuations stand out as the most intriguing phenomenon. The chance of sharp spikes and drops keeps us glued to our monitors all the time, creating a gambling-like addiction.
Sometimes, it’s hard to determine what causes these drastic changes. Even if you’ve been in the business for more than a decade, which is the case with MiroMind, some things still surprise you. In the end, Google is a continuously evolving beast, and you can never know if the search engine algorithms changed overnight.
One of the search engine’s newer changes comes in the form of Indexing API, a tool that website owners can use to speed up the indexing of their pages. Even the well-known WordPress plugin, Rank Math, has created an integration for the API allowing millions of website owners to use the two together.
In this short case study, we want to focus on Indexing API and related issues. We recently had a situation where Indexing API (probably) led to the complete demise of one of our pages. But before we get into the nitty-gritty, let’s first explain this concept and how it affects organic traffic.
What is a Google Indexing API?
Indexing API is a fantastic Google tool that helps solve several issues. The concept was initially developed as a way of regulating the freshness of websites’ pages. Its main purpose was to index and deindex pages with short lifecycles, such as job posts and streams. This is the official explanation from Google:
“Currently, the Indexing API can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject. For websites with many short-lived pages like job postings or livestream videos, the Indexing API keeps content fresh in search results because it allows updates to be pushed individually.”
Keep in mind that the Indexing API was introduced in 2018, so a lot might’ve changed since then. Among others, Google’s search engines might’ve started using the concept in a different way, expanding its purpose.
Here are a few things that Indexing API can help you with:
- With this tool, you can easily update ULRs on your site. Indexing API is fantastic for quick crawling, which makes it an excellent tool for sites that regularly update their content.
- Indexing API can also help webmasters remove pages from Google’s index.
- Among others, the tool helps you when trying to index a large number of pages. It’s one of the reasons that make it great for forums, job posting sites, streaming sites, and such.
- You can also use it to monitor how Google interacts with the page. By using Indexing API, you can track all the notifications that the search engine receives regarding a specific page on your site.
Indexing API works great for sites that regularly add new pages, short posts, images, and videos. By relying on this concept, you can index your content much faster and start gaining organic traffic almost instantly. As the post will reach a target audience much faster, it might help conversions, especially if your posts are time-sensitive.
Despite its benefits, Indexing API might also cause a few problems. In other words, it might cause issues for sites that focus on evergreen content. Here’s the situation we recently experienced.
Indexing API issue we encountered
In February this year, we noticed a weird situation with impressions/clicks on a website using Indexing API. Basically, one of its pages had extremely short lifecycles. After being quickly indexed, the page started gaining impressions and clicks. But after just a month, all that organic traffic vanished as if the page had never existed while the page remained indexed (site: operator).
While the small number of clicks and impressions wasn’t concerning (you win some, you lose some), the thing that stood out was the complete loss of traffic after a month. Given that the website focuses on long-form content, this was a concerning discovery.
Making presumptions
After seeing this happen, we made two valid presumptions:
- There is a good chance that Google uses different indices for different content. In this particular case, the search engine’s bot relied on Indexing API to categorize our posts.
- The second possible presumption is that Google initially indexes all posts with Indexing API. This would give the content a short-term boost allowing the search engine to gather additional data that would help properly rank it. Still, when testing different forms of indexing, we noticed that pages act differently when submitted via Indexing API compared to natural indexing.
Although the use of Indexing API sounds like a good idea, we were concerned about how it affected the website in the long run. The initial boost is nothing to scoff at, but given the price, the results weren’t suitable for our particular content. So, we decided to take action by performing A/B testing.
Making the move
The site owners were much more concerned about creating long-term value for their readers than sharing the latest news. Each post they made was there to stay, providing value long after it was written. To better assess the impact of Indexing API, we decided to turn it off.
Here’s what happened:
After checking the data in Google Search Console, we noticed a sharp spike in clicks and impressions. In just five or six weeks, we went from approximately 3,000 impressions to 6,000 impressions in Google. This is basically double! As you can presume, the number of clicks increased proportionately.
We double-checked the data with AhRefs, to make sure our findings were on the spot. According to the tool, we previously ranked for approximately 450 keywords, which rose to 1,100 ranking keywords in one month.
If you think that was a lot, you should see our traffic numbers. The number of clicks went from 45 to almost 200. This is a whopping 300% increase! Then again, higher click-through rates are to be expected once you start reaching the top spots for lucrative keywords.
While this doesn’t mean you should shut down Indexing API, it’s still something to consider. Again, the tool might be important for your brand, depending on the type of site you’re running.
If you want to read similar case studies or simply need SEO services, contact MiroMind today!