Bot Spot
|
"The Spot for All Bots on the Net".
|
The Web Robots Pages
|
Martijn Koster's pages on robots, specifically robot exclusion.
|
Japanese Search Engines
|
This is a comprehensive index for searching, submitting, and
navigating using Japanese search engines.
|
Search Engine Watch
|
A site with information about many search engines,
including comparisons.
Some information is available to subscribers only.
|
RoboGen
|
RoboGen is a visual editor for Robot Exclusion Files; it allows one to
create agent rules by logging onto your FTP server and selecting files
and directories.
|
