Click here - to use the wp menu builder

HomeTechnologyAnalysis of Search Bots

Analysis of Search Bots

By Joe

May 8, 2006

60

A large-scale experiment created over 2 billion web pages, linking to each other to analyse the patterns the various search engines crawled the site. Each page contained a link to two new pages, as well as a parent, and the three main search engines were watched as they scanned the site.

Very cool…

Previous article

“Fans who share music aren’t thieves”

Next article

RELATED ARTICLES

Most Popular

Recent Comments

Soledad Kerska on Google zooms in to “Street View”

Andrea_R on New Molson Canadian Ad

Joetek on When Will My Pebble Watch Ship?

Tom on When Will My Pebble Watch Ship?

Jocelyn Hurley on When Will My Pebble Watch Ship?

mohini on How the Zune will (eventually) kill the iPod

Austin Dwi on 5 free tools for collaboration with remote teams

Asia on 5 free tools for collaboration with remote teams

Jason Willis on 5 free tools for collaboration with remote teams

Joe on Toolbox – Twitter Apps

Matt Mauldin on Toolbox – Twitter Apps

Joe on Canadian Politicians on Twitter

Canadaka on Canadian Politicians on Twitter

Dennis on 5 free tools for collaboration with remote teams

Joe on Canadian Politicians on Twitter

Joseph Lavoie on Canadian Politicians on Twitter

The things that are better left unspoken : Blocking Internet Explorer 8 Automatic Delivery on 5 things every web developer should know about IE8

Lee Newton on Canada’s Do Not Call Registry

Joe on Canada’s Do Not Call Registry

Ace on Canada’s Do Not Call Registry

Lee Newton on Brand new server cluster at b5media

Aaron Brazell on Brand new server cluster at b5media

Sime on Brand new server cluster at b5media

Jan Karlsbjerg on Brand new server cluster at b5media

Sean on Brand new server cluster at b5media

Joe on Brand new server cluster at b5media

MikeD on Brand new server cluster at b5media

David on Brand new server cluster at b5media

Joe on Connected at the cottage

Goofingaround » Death of the newspaper on Connected at the cottage

face painting spplies on Shutting down for Earth Hour

Kim on Move along…

Darren Rowse on Move along…

Jeremy Wright on Move along…

Chad on Move along…

nicky on Free iTunes Low-Tech Hack

Joe on TSN.ca relaunches

Brian G on TSN.ca relaunches

Ornan on Joetek Server Move

Frank on Who wants a Joost invite?

Sunil on Joost update

joe on Who wants a Joost invite?

Nick Poulakis on Who wants a Joost invite?

joe on Joost update

alik on Joost update

Mason on Joost update

joe on Joost update

Jerry on Joost update

Mason on Joost update

Mason on Joost update

John Tobin on What is Web 2.0?

joe on Joost update

Drew on Joost update

John on Joost is television on the internet… plus more.

natestera on Optimus Mini Arrives!

John T on Q&A: Power cleaning

zettl on Q&A: Power cleaning

zettl on More Color Palette Generators

felix on Optimus Mini Arrives!

Nick on Joetek V3

umesh patil on Q&A: Power cleaning

Jon Norwood on How the Zune will (eventually) kill the iPod

un1xl0ser on How the Zune will (eventually) kill the iPod

Ingjerd on How the Zune will (eventually) kill the iPod

gian on How the Zune will (eventually) kill the iPod

thatedeguy on How the Zune will (eventually) kill the iPod

Amit on Google and YouTube

joe on Q&A: Power cleaning

John T on Q&A: Power cleaning

Brian G on ’10 and ’12

Joetek on ’10 and ’12

Brian G on ’10 and ’12

Brian G on Hybrid Pics

will aghajanian on Win a Spam Porsche

Brian G on Sounding like a broken record

Jeff Coleman on Sounding like a broken record

Brian G on RIAA’s Dilemna

Brian G on This is getting stupid

Joetek on This is getting stupid

Jeffrey Haas on This is getting stupid

Brian G on This is getting stupid

Jeffrey Haas on This is getting stupid

Brian G on An Internet Time Machine

Brian G on Universal Fights Back

Joetek on iTunes raises legal questions

George on iTunes raises legal questions

Michelle C on Surviving without power

BrianG on Surviving without power

Joetek on Blackout!

Brian G on Blackout!

Brian G on Bye bye Blogger