How Do You Archive Pages on a High Post Volume Website

Post image for How Do You Archive Pages on a High Post Volume Website

Today’s post is an answer to a question I took a few weeks ago:

How to organize archives/categories on WordPress for news site/blog that publishes a lot of articles, around 30-50 daily.

your site’s link equity will determine your crawling depth and crawling frequency
 The first thing I want to bring up is that generally, if you’re publishing that volume of posts per day, your posts probably are date sensitive, so you don’t mind Google attaching a date to your posts. You can include the date in your URL structure, but it’s not necessary. However, you do want to have date archives available and close to the top. As an example, in the top of my masthead, you’ll see a link to the archive page which has a link to every month I have published a post. So if you wanted to reach a post I wrote, the path is 4 levels deep

Home > Archives > October 2005 > Bacon Polenta

This is what people mean when they mention a flat site architecture or crawling path: you aren’t more than 4 clicks from any other post. You also want to make sure you have your robots.txt and robots meta tags configured properly to allow the spiders to crawl that path. If you are publishing a very high volume of posts/pages, you’re going to want to get as many links on the archive page as possible, without becoming excessive. Google recommends no more than 100 links per page. In reality that number is really affected by the trust and authority of your inbound links or your link equity.

If you aren’t familiar with the term link equity you should read this article by Eric Ward on link equity. Basically, your site’s link equity will determine your crawling depth and crawling frequency. The more links you have and the stronger those links are, the deeper the search engines will crawl and the more frequently they will re-crawl it. This is a difficult problem for new websites: they need to add content, but if they add too much too soon, it won’t get crawled. So new sites need to balance content creation with link building.

Some other tools you can use to help flatten out a website and increase crawling depth are breadcrumbs. Joost De Valk makes an excellent breadcrumb plugin for WordPress websites. A related posts plugin that changes the related posts at the end of each post will also help. I like yet another related posts plugin. Also make sure you are generating an HTML sitemap I like the dagon sitemap generator.

So, to wrap things up, here’s what you want to do:

  • Provide one or more short crawling paths for spiders. Try to keep the path as short as possible because anything beyond 5 levels is a problem.
  • Make sure your robots.txt and robots meta tags don’t accidentally block the crawling path.
  • Provide alternate crawling paths or access to the website content with breadcrumbs, related posts, and sitemaps.
  • Try to minimize maintenance and make sure all or as many of these solutions as possible update automatically.
tla starter kit

Related posts:

  1. Website Informational Pages Now that I’ve covered long term content and short term...
  2. Site Links and Search Volume One of the things that I admire about the Google...
  3. What is a Compilation Post I came across a post today that was actually pretty...
  4. How to Deal with Expired Product or Auction Pages Today’s post is question from Hicham Damahi of beezid.com/ who...
  5. Free High Paying Keywords from Threadwatch If you’re a player in the search engine field, and...

Advertisers:

  1. Text Link Ads - New customers can get $100 in free text links.
  2. BOTW.org - Get a premier listing in the internet's oldest directory.
  3. Ezilon.com Regional Directory - Check to see if your website is listed!
  4. Need an SEO Audit for your website, look at my SEO Consulting Services
  5. Directory Journal - Get permanent deep links in a search engine friendly directory
  6. LinkWheel SEO - Get Web 2.0 Backlinks
  7. TigerTech - Great Web Hosting service at a great price.
  8. Article-Writing-services.org - Article Writing Services creates quality content for websites and blogs at no cost to site owners.
  9. Link Building Services - Hire WeBuildLink.com for well-planned advanced link building campaigns. Very affordable. Contact us now for a FREE evaluation.
  10. Join 500+ top agencies & publishers who rely on The HOTH every day. US based, enterprise-grade support. High quality, contextually relevant links. 100% money back guarantee. Try it now.
  11. Professional website designs - Get a unique brand image with website designs that sets you apart and convert your visitors into customers. Make a brand, not just a website

See my disclaimer about advertising and affiliate links