Google Sites vs Google Search

Intro

Once I had this website up and running, I figured some of the information might be useful, particularly for newcomers to the DIY audio hobby if they could find it. So I decided to add the website to the Google Search console. This is pretty easy to set up. After a few weeks, I started to see some clicks in the console reports (but not many). 

I started building the website with a very simple structure. That is, no structure at all. All the pages were at the same level under my domain name, jwtaudio.com. The main topic pages appeared in the nav bar and the detail pages within each category were set to be hidden from navigation. It worked OK, but as I added pages, organizing them was a bit of a hassle. Plus, I had not discovered a way to have drop-down menus in the navigation bar at the top of each page. 

I found out that it was easy to get the nav bar to have drop-downs. In the Pages menu in the Sites editor, you can drag pages under top-level pages and they become sub-pages. This was much nicer. I then set the pages to be visible to navigation and they show up when you hover on a nav menu item.

Oops

Moving the pages under top-level pages changed the "canonical" names of the pages that got moved. That is, for example, www.jwtaudio.com/if7-amplifier.html became www.jwtaudio.com/projects/if7-amplifier.html 

Not a problem, right? Wrong. This completely broke Google search. Google had already indexed my website and refused to recognize the new names of the web pages because they were considered "duplicates" of the old pages. But they were not duplicates, just renamed. 

This caused searches to return "404" when clicking on my old links in search results. What a mess. I then figured out you can request page removal from Google's index database, but you have to enter each page URL individually. This results in a "temporary" removal, which seems to have a timeline of 1-2 weeks. I moved all the old names to the removal queue. Presumably, when the temporary time limit expires and Google re-crawls the old page names, it will permanently remove them from the search database once it sees they are gone (404). 

But - here is another anomaly: even though a URL shows as "temporarily removed" in the removals page, it still can show up as indexed and on Google in the pages report!

Since the new page names were not indexed, I started requesting indexing for all of them, also one at a time. I ran across another issue: Google limits the number of manual requests you can make for indexing - it seemed to be around 10 per day. Since I had about 40 pages, this was cumbersome. 

Google likes to brag about how fast they return search results at the top of the page - e.g., 10,000 results in under 1 second. However, the crawling and indexing operations are very slow, taking days and even weeks. 

The performance reports, which measure clicks and impressions, lag by about 2 days (sometimes longer), so it is hard to see if changes are working. Gradually, over 2 weeks, I began to see most of my renamed pages getting indexed. I also saw some pages were "crawled but not indexed".

There is a URL inspection option in the tool that seems to be real-time. I saw that many pages had been accepted and indexed, even though the Indexing/Pages report said that they were not. The Indexing/Pages reports lag by several days (3-6) as well, even longer than the Performance reports. So these all seem to be disconnected background processes. 

The Mysterious Validation Button

So if Google thinks there is a problem with a page or set of pages, you can click the "Validation" button to "validate" your fixes. This is slightly ironic since they are not my fixes, but index errors. This also seems to be a long-running process and can take days to finish. And why would I want to validate 404 errors which I know to be correct since they no longer have reachable URLs? Puzzling.

Site Maps? 

Standard SEO (Search Engine Optimization) practices suggest including a sitemap XML file on your website. However, there is no way to do that with the current (2024) version of Google Sites. Many pundits say don't worry, Google's search engine will find and index all your pages (eventually) but many people complain that Google Sites do not get particularly good search rankings. 

There are third-party sitemap-generating services, like https://pro-sitemaps.com/ which I tried, but so far have not seen any uptick in clicks. 

Meta Tags? 

In addition, standard SEO practice is to have meta (not Facebook haha) tags in the <head> section of your web pages. In Sites, however, there is no way to do this. I tried to get Bing to index my site and it failed, declaring the lack of meta tags an error! It would be nice if Google Sites allowed this, similar to the way they let you embed HTML (just not in the <head> section). I did use the embed capability to add Disqus comment blocks to some pages.

Bottom Line

If you build a site with Google Sites and want it searchable, decide on the exact structure of the pages first. This will avoid the hassles I went through. 

Be Very Patient

Google search moves in mysterious ways - but eventually your web pages will probably show up in search results. If you are expecting results to be shown in the Search Console quickly, that is not going to happen. It may take days or weeks for updates to be visible.