Robots -- Internet robots, spiders, web-walkers, etc.
About Robots	English (USA)
This list is for discussion of software for automated gathering of data from the Internet, especially Web robots and spiders, including the robots.txt robots exclusion standard. To see the collection of prior postings to the list, visit the Robots Archives.
Using Robots
To post a message to all the list members, send email to robots@mccmedia.com. You can subscribe to the list, or change your existing subscription, in the sections below.
Subscribing to Robots
Subscribe to Robots by filling out the following form. You will be sent email requesting confirmation, to prevent others from gratuitously subscribing you. This is a hidden list, which means that the list of members is available only to the list administrator. Your email address:   Your name (optional):   You may enter a privacy password below. This provides only mild security, but should prevent others from messing with your subscription. Do not use a valuable password as it will occasionally be emailed back to you in cleartext. If you choose not to enter a password, one will be automatically generated for you, and it will be sent to you once you've confirmed your subscription. You can always request a mail-back of your password when you edit your personal options. Once a month, your password will be emailed to you as a reminder. Pick a password:   Reenter password to confirm:   Which language do you prefer to display your messages? English (USA)   Would you like to receive list mail batched in a daily digest? No Yes
Robots Subscribers
(The subscribers list is only available to the list administrator.) Enter your admin address and password to visit the subscribers list: Admin address: Password:    To unsubscribe from Robots, get a password reminder, or change your subscription options enter your subscription email address: If you leave the field blank, you will be prompted for your email address