Skip to main content
Topic: More Spiders (Read 2333 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: More Spiders

Reply #1

I think it should work with minor changes.

Re: More Spiders

Reply #2

Just minor changes, all you have to do is change the db calls which can be done with search and replace.  I'm not sure how relevant some of those spiders are anymore, or if they are even still around, but adding them will no harm anything either.


Re: More Spiders

Reply #3

Well, I'm not too sure on the license on it, so if it does get changed, not too sure it can be posted.

Re: More Spiders

Reply #4

You can always ping Sleepy if you are concerned, or update the list to use http://www.useragentstring.com/pages/Crawlerlist/ which is more update. 

Re: More Spiders

Reply #5

Discussions on the changes should be fine but sharing a repackaged code may need a consent.

Re: More Spiders

Reply #6

End of the day its an standard insert statement ... it simply adds data to an existing/standard table with a list of spider names ...  it falls under the "If code embodies the only way (or one of very few ways) to express its underlying functionality, that code will be considered unoriginal because the expression is inseparable from the functionality." 

Anyway like I said ping Sleepy (nice), update the list to the current set of spiders (nice), and give him back the updated code as well (nice)