Is Googlebot Getting Lazy
Posted on October 24th, 2006by Michael Gray in Google, SEO
If you're new here, you may want to subscribe to my RSS feed. Read my top posts or learn more about Michael Gray. Want more frequent updates follow me on Twitter. Thanks for visiting!
I’ve been watching some of my pages to see when they get into Google and am getting pretty frustrated by the lack of progress so I did a little checking, and think Googlebot is getting pretty lazy …
Looks like there’s all sorts of sites slow to index CNN.com October 20th 4 days old
Netscape.com Oct. 20th 4 days old
Digg.com Oct 19th 5 days old
I’m sure there are pages in the indexed that are more up to date, but if a big heavily trafficked site dripping with PageRank and oozing with trustworthiness and authority is 4 days old does seem like somethings not quite right doesn’t it?
Sphere: Related Content
October 24th, 2006 at 10:51 pm
I’m seeing “Oct 22, 2006 21:07:38 GMT” when I check CNN and similar times for Netscape. Sometimes it can vary when data is being pushed out and you might hit one data center vs. another.
October 24th, 2006 at 10:54 pm
P.S. You know, back in early 2003, the freshest search engines would be 30-45 days behind. I think SEOs these days are more caffeinated.
October 24th, 2006 at 11:02 pm
Now that you mention it, I’m noticing the same thing. From what I can see in my web analytics, Yahoo is a much hungrier spider…daily visits to my blog. But, the cache is the same for both Google & Yahoo.
October 24th, 2006 at 11:35 pm
LOL @ Matt!
I would hope they would give CNN the 4-5 day index and Fox News the 1-3 during this important politican season.
Fair and balanced indexing.
October 24th, 2006 at 11:55 pm
SEO’s aren’t the only one’s more caffeinated I seem to remember seeing you drinking a red bull during a morning session. I believe it was Boston Pubcon 2006 when you were on the panel with Robert Scoble and Jeremy Zawodny. I’m pretty sure it was a sugar free red bull lot # ….
October 25th, 2006 at 2:02 pm
LOL, Matt! Yer, I remember when the cycle used to be
1) create loads of content and upload it
2) Watch your logfile like a hawk for Gbot hitting the new pages (all kinds of PR pump tricks went on to entice the little fella down the “right” links)
3) Wait for the Google Dance (yes, the original meaning)
4) Watch your new pages grab top 5 spots overnight when a new index rolled out
5) Profit!
New content insertion was a major step forward, and was greeted with amazement at the time. I guess y’all are going to have to keep improving that cycle time - you’ve got the old skool “instant indexing” to replicate yet
October 25th, 2006 at 4:21 pm
Taking new (yes, BH) content fully indexed is hard this times. It takes me something around 30 days now.
October 26th, 2006 at 1:01 pm
Yes, it has kinda slowed down lately.
October 26th, 2006 at 1:14 pm
just maybe google is busy handling the copyright issue for those videos in youtube??! : P
October 26th, 2006 at 5:51 pm
I was kinda thinking that my sites were a little being left off by google having noticed these.. but seems googlebots trying to have a vacation or is now being picky hehe
October 28th, 2006 at 1:54 am
Hi Matt your awesome google is awesome you are all awesome !!! later