More Spiders July 06, 2015, 06:03:58 am More SpidersNot too sure if the database entries for search engine spiders has been changed, but I'd love to be able to use this mod.I had it in the old forum, but I guess the converter did not do them.Any chance of getting this mod ported over?
Re: More Spiders Reply #1 – July 06, 2015, 10:17:50 am I wrote a quick scheduled task to fetch the list from http://user-agents.org/ but is slightly overwhelming... LOLThe final table is 1000+ entries and most are probably no more relevant.This one may be interesting as well:https://github.com/monperrus/crawler-user-agents/blob/master/crawler-user-agents.jsonthough it lacks some info that would be nice to show.TL;DR: it may be nice to have a repo to add to the list...
Re: More Spiders Reply #2 – July 06, 2015, 11:08:20 am I'm not certain what the license is on the SMF mod, nor if it's on GH.I can't PM Sleepy to ask, either. LOL
Re: More Spiders Reply #3 – July 07, 2015, 07:23:40 am I guess the more spiders list is a bit outdated, it was last updated in 08? Some are still right, some are gone, some have changed the crawler id. The list emanuele linked to looks like an interesting updated to the list.
Modern, Free, Powerful