Skip to main content
Topic: Bad Behavior Feature (Read 1180 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Bad Behavior Feature

Since 1.0 we have provided "Bad Behavior" anti spam functionality included.  I'm curious if this is still useful to anyone. 

The BB software has not been seriously updated in some time, and many of the checks are very dated.  Bots are much cleaner now than when the core of BB was written to catch issues and the bots are no clumsy these days.

While scanning one set of logs, I found (3) main reasons to BB actions

1) The main one was "Required header 'Accept' missing".   TBH there are a share of these hits that are incorrect.  Several of checks that flag this issue are so-so at this point.  There are various exceptions for things like  like PlayStation 3 or Yahoo hot jobs Seeker, or Window ME among others, just listing those to show the patina on the code.  I know there are specific issues with mobile devices and this check.   

2) The next major BB block was not really BB, but calls made to the http:BL service provided by Project Honey Pot.  These are logged as "IP address found on http:BL blacklist"  This function could also easily be added without the need for BB

3) The last seemed to be "User-Agent was found on blacklist"  This is detected since BB simply checks IP's with spamhaus.org SBL / XBL lists. That is easy to add without having to rely on BB.

I've also seen a few "URL pattern found on blacklist" which is generally someone trying to inject sql via the url, that would not go anywhere w/o BB anyway.

So for those of you using BB,  useful or not?  What detections are you seeing?
Be safe, Be kind, Happy Programing

Re: Bad Behavior Feature

Reply #1
It looks like it got some maintenance love in 2019 but that was the last trace of it (and it didn't make it to the 'official'(?) repo on GitHub. Post: https://bad-behavior.ioerror.us/2019/10/22/bad-behavior-2-2-24/

The accept header being missing used to be more relevant back in the day but I'm genuinely torn on the value of this now - it's been a while since I had any real honeypot data to look at to see if this is still relevant. For the mainstream browsers as far as I know they all still send it. Would agree there are many more fringe clients that don't use it now.

If you're relying on BB for just the blocklist integrations, you're probably better implementing those yourself at this point. I remember when Wedge did BB integration, we cut out a whole bunch of tests that simply weren't relevant. I think for the few that might still be relevant, they can be added as-is without the fluff that is the rest of BB's architecture.

Re: Bad Behavior Feature

Reply #2
Checked this sites log (it does not have spamhaus enabled), had about 2K entries.

Honey Pot -  Logged as:  IP address found on http:BL blacklist ~50%
BB -  Logged as:  Required header 'Accept' missing ~25%
BB (SQL in url check) -  Logged as: URL pattern found on blacklist ~ 23%
BB (Blacklisted user agents) -  Logged as: User-Agent was found on blacklist ~ 2%

So just some data.   Could reject the missing accept at the server level as well.  I think for Nginx its would simply be
Code: [Select]
if ($http_accept = false) {
    return 403;
}
or maybe return 406 ... anyway easier then all of BB
Be safe, Be kind, Happy Programing

Re: Bad Behavior Feature

Reply #3
Huh, interesting stats.

I'd also be curious which user agents you're blocking because what I'm finding is that there are a host of 'search engine bots' that are pretty awful and unlikely to return much traffic, e.g. Aspiegel/Petalbot, Mauibot, Ahrefsbot, to the point where I block them entirely from even getting to the server if the user agent mentions that.

But I think we can agree that you don't need to integrate BB as is at this stage, and just do the integrations you do care about.

I will say that the measures I added to SMF on the CAPTCHA validations (especially registration) have proven reasonably effective at keeping things at the door without too much effort - time gating and a mandatory-empty field.

Re: Bad Behavior Feature

Reply #4
Kind of says its not really doing much, its those outside services that are doing the bulk of the lifting.  The rest could be done with a couple of simple checks -or- blocked at the server level for those that can make changes there.

I think I have data access to another site, so I'll take a look at its logs and see what/why of the blocking.

I've tended to use some nginx items "conn_limit_per_ip",  "req_limit",  "php_limit"  to throttle requests / activity.  The only things that seem to trip those limits are bots.  Then I have a couple of fail2ban rules, so if you trip those breakers a few times in a given period, you get blocked for some period of time.
Be safe, Be kind, Happy Programing

Re: Bad Behavior Feature

Reply #5
As I read through this, and my own logs, I'm glad I have the forum running in a protected directory that nobody can access unless they request it (requires two steps - email validation, and then personal justifying information for access credentials).  The process is homespun using simple html coding.

Email validation is just "enter your email address and submit".  Upon submit, the applicant gets an html page that says to check their email, and when received, click the link on that page that goes to the next step).  My server sends an email response to that addy that includes a random 6-digit number (along with apologies if the addy is not the right one for the submitter) , and I get a copy that goes into my "access pending" folder.

The 2nd step is a series of questions on a form - includes a space to enter their email address and the number that was sent, along with their answers to the questions.  When the applicant submits, they get another page that says to watch their email for credentials (could be a day or two), and that form comes to me in an email - I then can match the email address and number from the first and second emails to verify the supplied addy is real, and decide whether the applicant gets access to the forum area. If so, I email their credentials to the directory (unique for each - username and password related to their personal information)  to them, and the link to the forum home page where they can register for the forum...

As a result I have virtually no forum moderation tasks to perform, subsequently - no "bad behavior' to concern myself with.

And for the record, my questions themselves screen out a lot of undesirable folks, the questions pertain to the subject of the forum and if they aren't familiar with the subject they can't answer them appropriately - I rarely get a 2nd step response from someone who is subsequently denied access,  and my ratio of first email to second email is about 5 to one.   :)

My logs show quite a few "gets" on the menu page that has the application link - maybe 200 a day from unique IPs - about 30% being spiders - and I'll get maybe 5 emails a week from the 1st application step.

My process (for the particulars of *my* forum) weeds out the bad actors and 'lookie-loos', before they ever get to the forum - or even complete the process.

So @Spuds, I posted this in response to your query "Since 1.0 we have provided "Bad Behavior" anti spam functionality included.  I'm curious if this is still useful to anyone.",  so you would understand why I wasn't responding to it..  :D 

[if anyone is curious to see this in action, PM me for a link]
__________________________________________________________________________
// Deep inside every dilemma lies a solution that involves explosives //

Re: Bad Behavior Feature

Reply #6
@Steeley thanks for all those details, I bet that does keep everything running smooth.

ElkArte also provides for a Q&A capability during registration, its under security and moderation, anti spam.  Its a very effective tool in preventing bogus registrations.  Eventually the answers are discovered (a real person/spammer answers it and sometimes enters it into the bot DB) but then you just change the question.

Another effective tool is setting the first post to be approved for new members, so should a spammer successfully register and spam, it goes no where since it needs to be approved.  That helps keep things clean for the users.

BB is really intended to protect the forum before a spammer/bot even gets to see a page.   I think it had its place, and may still for some, but given what its catching, there are lighter weight solutions to do the same.  If you can do blocking  at the web server level is even better.

Ah spam, so much fun and effort on all sides.
Be safe, Be kind, Happy Programing

Re: Bad Behavior Feature

Reply #7
I think my first post in the forum here, @Spuds, was about EA running in a protected directory structure (which is what I had established for a significant portion of my site when I went looking for a suitable forum package). Nobody responded, which I presume was because nobody who read the post was doing it.  So, I just installed it and discovered that happily, it works fine. But my deployment, therefore, is likely unique among the user-base.

And, I suppose, my post also give a heads up just in case you thought it might be a good idea to make some configurable security features defaults or hard-coded - I have one compiled software package with an anti-spam feature I can't disable that requires my intervention to approve certain user-activities it flags that only serves to frustrate my ('pre-screened') users, makes them wonder if I'm being anal, and increases my workload to boot.  I asked the author to allow me to relax or disable the spam-check criteria and his frustrated response was "I'm not changing it AGAIN!"  :'(   (What? He's not going to customize his open source software just for me? I guess I should be offended...)

Anyway, if I am silent on these security types of feature questions in the future, just ignore me if it appears I'm ignoring you.
(Feel free to be annoyed with everyone else that ignores you, however - they get what they deserve, right?   :D ).
__________________________________________________________________________
// Deep inside every dilemma lies a solution that involves explosives //

Re: Bad Behavior Feature

Reply #8
@radu81 sent me a couple of logs.  The quick look shows

Required header 'Accept' missing 78%
IP address found on http:BL blacklist 12%
URL pattern found on blacklist + User-Agent was found on blacklist (.01%)

I looked at bit at the Accept headers missing logs and it breaks down as:
50% Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0 (80% of it from via 13.36.166.xx)
22% SemrushBot
14% Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0 (204.15.208.xx)
5% linkdexbot
3% Firefox/16 (116.202.32.xxx)
1 hit Pinterestbot
1 hit Nimbostratus
3 hits duckduckbot

The remaining 6% all a random, mixture of mobile, chrome, fx, redirects, etc
Be safe, Be kind, Happy Programing

Re: Bad Behavior Feature

Reply #9
What are the odds that that Firefox 52 header is *really* FF 52? I’d suggest not high since that isn’t how FF normally operates, but I’m not sure this is good evidence for keeping this filter… it might have worked back in the day for WP bots but I don’t believe it’s that significant now.

Re: Bad Behavior Feature

Reply #10
Agree ... I'm sure its whatever was coded in to the program,  the Fx16 one was pretty funny as well.   I'm feeling that I'll be removing BB and just replace it with a couple of easy settings in the ACP.  It will be just as effective and cleaner.
Be safe, Be kind, Happy Programing

Re: Bad Behavior Feature

Reply #11
Sounds like Spuds is having fun these days. :D
Bugs creator.
Features destroyer.
Template killer.