[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmca] Spam and xmca mailing list

Also, it's just an instruction that TELLS the spider bot to not index, it doesn't STOP the bot from indexing. Spider bots for legitimate search engines like google are programmed not to index pages with the "don't index" instruction, but a spider bot that's used nefariously to harvest email addresses could just ignore that instruction and index anyway.

I think it's not likely that xmca is significantly involved in the SPAM problem. Having said that, moving to something like a group blog that people could subscribe to would allow threaded posts to appear without email addresses, and would have other advantages as well.

On Fri, 28 Oct 2011, Bruce Jones wrote:

From huw.softdesigns@gmail.com  Fri Oct 28 09:58:40 2011

Website pages can be marked as _not_ for 'spider bots' that accumulate
pages for google and the like, I believe.  So checking the relevant
lchc pages would be a start.

This is the "robots.txt" file that tells the search engines which of
your pages to index.

If you don't allow the search engines to index your pages then their
contents won't show up the search listings.
xmca mailing list