« Home | Guide to SEO for eCommerce Sites - Part 1, Product... » | Driving Traffic - Free Before Fee » | Five HTML Tags You May Not Know About » | Advanced Segments in Google Analytics » | AdWords:Seminars for Success in Columbus » | HTML email design shouldnt be difficult! » | Vector Magic: Convert Raster Images to Vector Imag... » | New Addition to the Family! » | Smarter Web Design with CSS Buttons » | How efficient are you? »

SlurpConfirm404

Mar
6
'09
Like many other site owners out there, I've got created a smart 404 error page that emails me when a user arrives on said page. This enables me to identify common problems that I can fix. (For example, I've moved my blog from cmoist.com/ to cmoist.com/blog/, so I still get a lot of visitors to these now defunct URLs) Anyhow, after sorting out some of the more common problems, I started seeing pages come through that looked nothing like any URL I've ever hosted. All of them looked something like "SlurpConfirm404/PhD7Cameron/melanson.htm".

After a bit of research, it appears that this is the Yahoo! slurp crawler. The purpose of this crawler is to ensure sites have a proper 404 error page in place. Yahoo creates a URL consisting of random words that it appends to the end of your domain, then crawls.

Read Yahoo!'s explanation for what exactly the purpose of this crawler is.

Labels: , ,

Posted by Christian on Mar 6, 2009 at 8:19 PM | Permalink 0 comments

Comments: Post a Comment