How Can I Make Google Crawl My Site More Often?

Original post at: ezinearticles.com

1. Update content frequently

Now this is something you can’t do just for the sake of making Google visit you often, can you? But generally speaking, the sites or blogs that get updated more often would get visited by crawlers more often in the long run.

2. If you have a blog, ping Google automatically

There are several things you can do if you have a blog. Use feedburner, WordPress does the pinging work for you by default, so don’t need to worry a lot…If you have a site, you can always consider accompanying it with a blog, and create backlinks from the blog to new content.

3. Maintain the server in a highly available healthy state

Nothing surprising in this one of course…how can Google crawler index your pages if it finds a 404 error…the Google webmasters tool gives updates on errors encountered while crawling…so keep an eye on it. Googlebot is a busy man you know… 🙂 …so if he is giving you 1 hour to talk what you can, you should try to fit in a large amount of content in that one hour…that is to say, your site should have a decent download speed.

4. Put backlinks to updated content on forums (like Yahoo! answers)

Actually, I am myself not sure how practical this will be…you can of course link to some important pages like this manually…but surely this is not a scalable approach.

5. Create a dynamically generated XML sitemap

This one is pretty crucial I believe. There are some sitemap generators in the market…but I don’t think they will be good enough to replace a well thought out script written by you, which understands your DB schema and page structure very well. The Sitemap protocol is followed by all the crawlers, so no need to worry separately for Yahoo! and Live 🙂 . Having a decent HTML sitemap would also help, though I am not so sure if it should link to all of your pages…may be just the most important 100 pages of your site. You can also consider making them hierarchical.

6. Make site pages available as RSS feeds and distribute them

Making your content available as RSS has a lot of benefits. Several tools that help optimizing blogs, like feedburner can be used for your site as well, if the content is available as RSS.

7. Use social bookmarking sites like Digg

Again, nothing extraordinary about this. This is a good old tactic that has worked very well for people, but again it’s more applicable to blogs than sites.

8. Pagerank and authority of the site

Now this is something you need to improve over a longer period of time…can’t happen in a day or two!

9. Create a decent internal linking structure

Important point to note while creating the internal linking structure is to promote important pages by linking to them from several pages of the site. For example, you could link to “featured products” on the home page, and have these links dynamic so that they change every week.

The Google webmasters tool has a crawl rate option, but that will be useful only when the “Faster” option is available to you, which would not happen unless yours is a high authority site.