Last post by badmonkey -
They did that to one of my sites recently. Hundreds of bots at a time. They scraped a couple of days, disappeared a few, then came back for a couple days. There is probably now a Chinese duplicate somewhere.
Last post by Spuds -
Glad you found the toggle, that is **alot** of spider traffic for sure. I had some issues with traffic a while back and ended up setting up a couple of Nginx connection and request limits which if you exceeded tossed that IP to fail2ban for a short while. Anyway something to consider looking into, I'd imagine there is something like that for Apache as well.
If you could summarize the main contributors to the log spam that would help, with that many entries there is probably just 1 or 2 things causing the log spam, usually from an addon, you know the drill.
I keep checking on this as I really think it has something to do with the attachment resize addon .... I just have not come across the right set of conditions ... but it really has to be something in that addon (and why I just want to make it part of the core in 2.0 since it has to kind of "hot wire" the attachment code to work right now.
Seems you're right! I finally caught this specimen in the error log: