ElkArte Community

Extending Elk => Addons => Topic started by: Keiro on December 30, 2015, 04:23:08 pm

Title: StopForumSpam
Post by: Keiro on December 30, 2015, 04:23:08 pm
I'm creating this thread mostly for myself and for others to help contribute with porting the SFS mod to ElkArte.

My Github for porting SFS to ElkArte is here (https://github.com/KeiroD/SFS).

Feel free to post and such about the module here.
Title: Re: StopForumSpam
Post by: Keiro on December 30, 2015, 04:45:24 pm
OK, so here's my initial analysis of the code that SFS for SMF uses.

If you check out the original branch I set up (https://github.com/KeiroD/SFS/tree/original)... you'll notice that the hand-edited code that was posted as per the SMF mod's parse link for said mod... it requires modifying several files that ElkArte does not use anymore.

I'm thinking that some of this will have to be moved to a new file. And then either an integrated hook via the hooks function to pull in SFS or also hand-editing index.php as it seems that index.php may potentially have some registration handling code.

Keep in mind this is just my initial analysis and I may be hilariously wrong. But these are my opening thoughts on the matter of porting this over to ElkArte.
Title: Re: StopForumSpam
Post by: emanuele on December 31, 2015, 02:21:56 am
First hint: you can put the function SpammerCheck into its own file, so it's one edit less already.
The edits to $sourcedir/ManageRegistration.php can be replaced using a hook (integrate_modify_registration_settings).
The edits to $themedir/Who.template.php can be avoided by using the <credits> tag in package-info.xml or using a hook again (there is a discussion going on about that).
What's left is the actual check in $sourcedir/Register.php that can of course be applied using a hook, but at the moment I'm not sure which one, this evening I'll have a look (unless someone else beats me :P).
Title: Re: StopForumSpam
Post by: Keiro on December 31, 2015, 03:10:00 am
Quote from: emanuele – First hint: you can put the function SpammerCheck into its own file, so it's one edit less already.
The edits to $sourcedir/ManageRegistration.php can be replaced using a hook (integrate_modify_registration_settings).
The edits to $themedir/Who.template.php can be avoided by using the <credits> tag in package-info.xml or using a hook again (there is a discussion going on about that).
What's left is the actual check in $sourcedir/Register.php that can of course be applied using a hook, but at the moment I'm not sure which one, this evening I'll have a look (unless someone else beats me :P).

Ah, good, so I was thinking correctly in that the function SpammerCheck can be placed into its own file and called for by another, like say the integration hooks.

I'll toss out the Who.template.php code as I'll be using package-info.xml to add the proper credits. There's no reason for me to rewrite the code when that can be achieved by using package-info.xml.

I was mostly working on analyzing Register.php and where it'd go. It looks like I can go either way, e.g., either as an integration hook or as a direct edit to Members.sub.php as I was able to track down where ElkArte handles its registration stuff.

I tend to typically analyze the code and where and what can be safely ported. It's nice to get confirmation that my initial thoughts're more or less on the right track before I go and blow things up... lol
Title: Re: StopForumSpam
Post by: emanuele on December 31, 2015, 09:54:31 am
Of course code edits are always possible, though if an hook is available, it's much much better to use it. Hooks are usually "portable" among versions and have less chances of conflicts installing multiple addons.
Title: Re: StopForumSpam
Post by: Keiro on December 31, 2015, 10:18:44 am
Quote from: emanuele – Of course code edits are always possible, though if an hook is available, it's much much better to use it. Hooks are usually "portable" among versions and have less chances of conflicts installing multiple addons.

What's the best recommendation as far as an integration hook? You'd briefly mentioned you'd look into which one was the best one.

I think I'll go with integration hook over that of direct code edits... for a fairly obvious reason. I don't really want to modify core unless I absolutely have to... and in this case, I don't think I need to make such direct modifications.
Title: Re: StopForumSpam
Post by: emanuele on December 31, 2015, 12:19:55 pm
I would try to use integrate_register_check.
I think it could be handy to check if the registration is actually a guest or an admin trying to register an account:
Code: [Select]
if ($regOptions['interface'] == 'guest')
{
    // do checks
}

I'm not sure it is worth using $reg_errors to send the guest back to the registration page instead of sending a fatal_lang_error.
Title: Re: StopForumSpam
Post by: Keiro on January 03, 2016, 02:29:01 pm
Quote from: emanuele – I would try to use integrate_register_check.
I think it could be handy to check if the registration is actually a guest or an admin trying to register an account:
Code: [Select]
if ($regOptions['interface'] == 'guest')
{
    // do checks
}/code]

I'm not sure it is worth using $reg_errors to send the guest back to the registration page instead of sending a fatal_lang_error.

Why's that? Also, just moved the SpammerCheck function into SpamCheck.php and made a minor edit to the function to check for Subs.php instead of Subs-Package.php. I've not tested the code yet as I'm still in the process of setting up a development environment to make sure this actually works as intended.

That commit is 99be59a (https://github.com/KeiroD/SFS/commit/99be59a7054684ec2a1f74b43c44502c45336600)

ETA: Whoops, noticed we have Package.subs.php within the subs dir inside sources/. Obviously, I've corrected the code.
Title: Re: StopForumSpam
Post by: emanuele on January 03, 2016, 04:11:20 pm
The function to create a member is shared between the two different possibilities:
1) a guest registers,
2) an admin creates a member from the admin panel.
I guess admin do not create accounts that may look like spammers, or, if they do, it is intended.
Let's assume an IP is banned on StopForumSpam, and by chance (like happens more often than it should to me) that IP is a dynamic one previously used by an infected computer. If I get assigned that IP and your addon applies to the admin registration process as well, I will not be able to create new users until I disconnect and re-connect the modem (and hopefully get a new IP).
Title: Re: StopForumSpam
Post by: Keiro on January 03, 2016, 05:48:42 pm
Quote from: emanuele –
Quote from: Keiro – Why's that?
The function to create a member is shared between the two different possibilities:
1) a guest registers,
2) an admin creates a member from the admin panel.
I guess admin do not create accounts that may look like spammers, or, if they do, it is intended.
Let's assume an IP is banned on StopForumSpam, and by chance (like happens more often than it should to me) that IP is a dynamic one previously used by an infected computer. If I get assigned that IP and your addon applies to the admin registration process as well, I will not be able to create new users until I disconnect and re-connect the modem (and hopefully get a new IP).

The latter case of dynamic IPs're definitely more likely.

I will most likely try integrating it with the guest registration process as that is the most likely source of spammers trying to get in... because 2. as per your thought if an admin creates a spammer, they're doing it for a specific purpose. So we won't go for the admin end... well, other than providing an interface in the admin backend to check and confirm, "Yup, them thar be spammers, boot 'em out!"

Though to be honest, I don't think that'd be quite necessary as we have http:BL and bad behavior. But still, it's nice to have additional options for the rare cases where they slip through.
Title: Re: StopForumSpam
Post by: emanuele on January 03, 2016, 06:17:16 pm
Just to be clear, I didn't mean to add a new option, The "interface" part is already provided by Elk itself, so it's just a matter to do a check in your the code, really just the line I wrote above. ;)
Title: Re: StopForumSpam
Post by: Keiro on January 03, 2016, 06:45:30 pm
Quote from: emanuele – Just to be clear, I didn't mean to add a new option, The "interface" part is already provided by Elk itself, so it's just a matter to do a check in your the code, really just the line I wrote above. ;)

... Oh. Well, then! :P
Title: Re: StopForumSpam
Post by: Keiro on January 22, 2016, 01:05:08 am
Hmm. What're the equivalents of the following:

Code: [Select]
1.	Execute Modification	./$sourcedir/Register.php	File not found
2. Execute Modification ./$sourcedir/ManageRegistration.php File not found
3. Execute Modification ./$themedir/Register.template.php File not found
4. Execute Modification ./$themedir/Who.template.php File not found

for ElkArte? I think once we change this, SFS should work... but I'd need to test this further.

Edit: I'm kind of an idjit, forgive me. I'm still working on wrapping my head about SFS... after not having touched it for a while.

Interestingly, the mod can be added to ElkArte's Package Center with little to no issue... at least, right up until you try to install it. Which causes the above error, naturally. So, for the most part, the code seems to work without too much of an issue... with the obvious need to modify the SFS code to point at ElkArte functions and such.

Edit2: Ah, discovered where Register.template.php is. It's in
Code: [Select]
themes/default/Register.template.php

Edit3: Making progress. Kind of. I've now got it to correctly detect the location of Register.template.php... we'll have to obviously correct the code to work with ElkArte, though.
Title: Re: StopForumSpam
Post by: Frenzie on March 24, 2016, 10:32:52 am
Unfortunately you may have focused on the "worse" modification. The one called Stop Forum Spam (http://custom.simplemachines.org/mods/index.php?mod=1519) hasn't been updated in years and doesn't use hooks. The mod called Stop Spammer (http://custom.simplemachines.org/mods/index.php?mod=1547) relies primarily on hooks and offers other desired features such as reporting spammers to the SFS database. I'd convert the latter for reasons of both functionality and practicality.
Title: Re: StopForumSpam
Post by: Mrs. Chaos on May 01, 2018, 04:47:50 pm
Is this the same addon?
http://addons.elkarte.net/security/StopForumSpam.html

I have just installed the recently updated addon.
What exactly does "Threshold level for spammer detection" mean? Which value/number is recommended?
Title: Re: StopForumSpam
Post by: radu81 on May 01, 2018, 04:54:18 pm
It's not the same add-on, this topic is related to this add-on https://github.com/KeiroD/SFS
Title: Re: StopForumSpam
Post by: Frenzie on May 01, 2018, 04:58:05 pm
@Mrs. Chaos No, that looks like an independent development.

Yet another one here: https://www.elkarte.net/community/index.php?topic=4277.msg30732#msg30732

Threshold is about confidence. The default in the add-on you linked is 50.
Title: Re: StopForumSpam
Post by: tino on May 01, 2018, 05:11:18 pm
I should search the forums before just looking at the add on site, it’s as described the threshold for how confident you are the person is a spammer.

My mod just out right rejects them if it’s met, frenzies will register them but require admin approval. Not sure what’s best, mine met my needs so just published that.

Title: Re: StopForumSpam
Post by: Frenzie on May 01, 2018, 06:05:07 pm
@tino It's partially my fault for writing things that are functional but never quite getting around to wrapping them up and/or submitting them to the add-on site. ;)

That being said, in this particular case it was also about waiting for Elk 1.1 to finalize before wrapping it up. I know,, I know it's been released a little while ago… ^_^

QuoteMy mod just out right rejects them if it’s met, frenzies will register them but require admin approval. Not sure what’s best, mine met my needs so just published that.
A reasonably important part of the mod is also contributing back to the SFS DB. Both in the form of extra e-mail addresses/spammer nicks and submitting spammers that somehow slipped through the cracks.

Anyway, as I wrote in my README "The first and best defense against spam available in Elkarte is to enable Bad Behavior coupled with the Project Honey Pot http:BL. […] Stop Spammer jumps in when a spammer manages to get past this enormously effective line of defense by checking their registration attempt against the Stop Forum Spam database. Only malicious bots that try to spam either through posts or profiles will do this in the first place, so enabling Bad Behavior is quite important." If you do that the amount of spammers trying to register drops to a mere trickle of what it would otherwise be, besides all the bandwidth/CPU savings.

If you disable http:BL for a week or two while you enable Stop Spammer then you can quickly get in hundreds of submissions on the Stop Forum Spam website. I actually did that as an experiment a couple of years ago and I ranked up thousands in mere weeks. I then turned http:BL back on because I had my fun and properly stress tested the system. For normal use you just want to have it there without even thinking about it. I seem to have disappeared from the list since, weird. Maybe it's an only in the last few months kind of thing.
Title: Re: StopForumSpam
Post by: tino on May 02, 2018, 02:19:43 pm
Yeh, I was going to extend mine to have a drop down on the member page to report someone as a spammer.

Either way, was playing with how to do that with just hooks. I found one of most actions tbf, it’s easier to code with out changes to the core. I don’t want to step on your toes though so happy to remove my mod.
Title: Re: StopForumSpam
Post by: Frenzie on May 02, 2018, 04:08:54 pm
To be clear, anything non-hook is not a feature. But at the same time I don't really know what you're talking about considering this (https://github.com/Frenzie/elk-stop-spammer/blob/af0b7a530698a6ed2f72065cbdd836bb989f1daa/install_1.1.xml) is the full extent of modifications.
Title: Re: StopForumSpam
Post by: tino on May 02, 2018, 05:04:52 pm
Quote from: Frenzie – To be clear, anything non-hook is not a feature. But at the same time I don't really know what you're talking about considering this (https://github.com/Frenzie/elk-stop-spammer/blob/af0b7a530698a6ed2f72065cbdd836bb989f1daa/install_1.1.xml) is the full extent of modifications.

Sorry, wasn’t referring to your mod. Was just expressing I was trying to do mine with just modifications.

Either way, If you would prefer me to remove mine im happy with doing that.
Title: Re: StopForumSpam
Post by: Frenzie on May 02, 2018, 05:32:48 pm
I would prefer if yours didn't exist but not in some hurt pride kind of way, just that I think the duplicate effort is a pity. :)

You want to team up? I don't really have time to work on it atm and I'm happy giving you write access to the repo or some such, although of course we should communicate through PRs in that case.

The other way around would work too, but note that my repo has a decade of history in it.
Title: Re: StopForumSpam
Post by: Mrs. Chaos on May 03, 2018, 03:12:49 am
Hey guys, don't argue! :o
I think both addons are good and should stay. Frenzie's is more extensive, but Tino's is enough for me (and maybe for others) until further notice, as I've had little trouble with spammers. So, Tino, please don't remove your addon!

Quote from: Frenzie – That being said, in this particular case it was also about waiting for Elk 1.1 to finalize before wrapping it up. I know,, I know it's been released a little while ago… ^_^
@Frenzie : Are you going to continue working on your addon soon?
Title: Re: StopForumSpam
Post by: Frenzie on May 03, 2018, 03:34:56 am
@tino @Mrs. Chaos I certainly don't want anything to be deleted! Tone can be hard on the Internet sometimes, my apologies if I messed it up. :)

I would personally prefer a bundled effort toward one add-on rather than duplicated effort, which would most logically mean deprecating the other add-on but of course that's entirely up to @tino. Stop Spammer is nearly feature complete so it probably shouldn't be too hard to tell whether the overall intent lines up. I'd like to emphasize that I'm wide open to changing the particular way a feature is implemented in code, as well as how it's presented. It's not like this thing is set in stone.

I don't think there are any real issues with my add-on at this point in time, besides the one that was fixed recently, so I'll just submit it to the add-on site as is. It's just missing one feature that I still wanted to implement and that's about it. (Checking a member from their profile.)
Title: Re: StopForumSpam
Post by: tino on May 03, 2018, 02:35:47 pm
Quote from: Frenzie – It's just missing one feature that I still wanted to implement and that's about it. (Checking a member from their profile.)

That's the one feature my next commit was going to add.

@Frenzie @Mrs. Chaos

I didn't take anything as offensive, was trying not to stand on others toes!

I've had a look at your code @Frenzie and tbh I don't know what half of it is there for or doing, large parts of commented out code don't help that mind!

I'll push my commit for the member search to my code over the weekend, and also do a pull request against your's @Frenzie.
Title: Re: StopForumSpam
Post by: Frenzie on May 03, 2018, 03:16:07 pm
Instead of starting from scratch I converted this from an SMF add-on (with permission). Not necessarily because it's easier but because it already covers the use cases and potential issues that cropped up over the years. That's where the commented out code comes from. Most of it pending deletion, some of it still to convert. Conversion meaning switching from source mods to hooks which was mostly fairly painless thanks to Elk.

But just pretend the comments aren't there (except where they might be relevant as an aid). :P

tl;dr anything between comments is SMF code that hasn't been adapted. It's primarily relevant as a roadmap, to some extent possibly as inspiration.

Anyway, like I said the reverse would work too as in copying bits and pieces from Stop Spammer.  It's already Elk compatible, obviously. ;) What I personally like about the historical approach though is that it dates back a long time and I took a special effort to capture and tag that in Git.
Title: Re: StopForumSpam
Post by: tino on May 03, 2018, 03:41:28 pm
Quote from: Frenzie – But just pretend the comments aren't there (except where they might be relevant as an aid). :P

We all have our own ways of working, none are necessarily the wrong or right approach. I don't store commented out code, thats what Git or SVN is for if I need to refer back I just do it to a commit by looking at the logs.

I was more referring to this https://github.com/Frenzie/elk-stop-spammer/blob/master/sources/StopSpammer.controller.php

I looked at it and couldn't work out what it was doing related to the mod. I need to install it when I get the chance to work it out most likely.
Title: Re: StopForumSpam
Post by: Frenzie on May 03, 2018, 04:59:25 pm
I don't like commented out code around too much either for the most part, but I don't think this is really the referring back kind of situation. :) You'd have to mentally reparse it each and every time, whereas in this form the template is ready. The SMF backend is reasonably similar to Elk and I could've easily converted it badly by simply slightly adjusting the source mods, but to a large extent there was no real match up to the hooks.

A rewrite would've probably been easier and you would get nice clean commits without even trying but it was a conscious choice to do a conversion, as a bit of a fun challenge as well as to retain the wisdom that was hopefully accrued in the old code through the years. I think the only wrong "choice" I made was underestimating that a couple of days (or rather a couple of afternoons) wouldn't be enough time to finish it, and I had no idea just how busy February/March/April/June were going to be.

Anyway, that code is obviously doing nothing at all. :P Unfortunately because I was rather busy last year I completely forgot what I wanted to do with it. Some kind of custom template for something… I'd have to look into it. Now that one should've probably gone into a separate branch.