About Google’s "Supplemental Index" – and how to get pages (URLs) out of it.
If you don’t know what the "SI" is, crack open your copy of SEO Fast Start and begin reading at page 51.
If you have pages in the Supplemental Index, here’s how to remove them, step by step. These instructions assume you have fewer than 1000 pages on your site. If you have more than that, read them, follow them, and then watch this space for further instructions on how to troubleshoot a larger site.
1) Start by going to www.google.com, and clicking the "Advanced Search" link to the right of the search box. In the advanced search form, select "100 results" from the drop down menu, and type your domain name in where it says "Only return results from the site or domain." Then click the Google Search button.
2) The search result page (SERP) will probably say something like "1-100 of 682 results." If they’re showing you less than 100, skip to step 4.
3) Scroll to the bottom of the page, and look for the numbers (1 2 3 4 etc) under the Gooooooooogle graphic. Click on the largest number (for most sites, the number 10). This will take you to the last page of search results.
4) Scroll down to the bottom of the last page of search results, and look for a link that says "repeat the search with the omitted results included" – click that link.
5) This will take you back to page 1, showing 1-100 of however many results.
Now, as you move through these search results from page to page, you will notice that some of the pages are labeled as a "Supplemental Result." Bummer, eh?
Introducing The "Buddy System"
Just kidding… we learned all about the buddy system at camp, right? If you ever let your buddy get out of sight, one of you will be eaten by a bear. What we’re going to do next is similar to the buddy system anyway.
You have two kinds of pages (URLs) – either they’re in the main Google index, or they’re in the supplemental index. Actually you may also have some pages that aren’t in either index, but that’s a whole other problem that I am not going to solve today.
If you have the time, make a list of all the "supplemental" pages, and all the "main index" pages. I’m sure there’s some easier way to do this than doing it by hand… but I don’t know what it is. That’s why I say, if you have the time. Maybe you’re real busy. I don’t know. Maybe too busy to deal with this whole SI problem. If you are too busy, pick ten (10) of each, or some other number you can manage with.
For every page that’s in the SI, pick a page that’s in the main index to be it’s "linking buddy." So if Page A is in the main index and page B is supplemental, maybe Page A can be Page B’s linking buddy. Now, go put a link on Page A that points to Page B. It helps, of course, if these pages are logically related in some way. Do this for all of your linking buddies.
Wash, Rinse, Repeat…
Now go away for a couple of weeks. Take a trip to the islands or stay at home and get some work done – maybe do some work to promote your web site so that it has more PageRank to begin with (start reading at page 63). Then repeat the steps above, observing which pages have just gone from the supplemental index to the main index, and giving those pages a little time before you assign them to be another page’s linking buddy.
If you discover that the number of supplemental pages far outnumbers the pages that are actually in the main index, so that you can’t find a linking buddy for everyone, you may want to look at your site structure (Chapter 4, don’t make me look up the page #), and of course, you could always help yourself out by building more links.
This process can be time consuming, but as long as you’re doing a reasonable job of building links to your site, you shouldn’t have to keep doing this stuff forever. Moving pages into the main index means that those pages can now participate in building your site’s PageRank, and that they can pass link reputation to your other pages.
It’s a good thing. Trust me. Supplemental results aren’t the end of the world (thanks for letting us know, Matt Cutts) but they aren’t a delicious chocolate sundae either. I prefer sundaes. That’s why I need to go to the gym.
For large sites (over 1000 pages), see you next week…
If your site has more than 1000 pages, you should be able to find a few supplemental pages by following the steps above, but what you really want to do is use a tool like GSite Crawler to spider your site and map out all the URLs, then check Google to see which ones are supplemental.
I’ll be back next week with details on how to crawl your way out of the supplemental index with a large site.
For anyone who found that this moved a little too fast for you, please read SEO Fast Start (it’s free) and get up to speed. Thanks!