ElkArte Community

General => OpenImporter => Topic started by: Runic on March 15, 2016, 11:21:24 am

Title: 38k Member Forum
Post by: Runic on March 15, 2016, 11:21:24 am
A friend is wanting to move his forum has 38k active members and he is worried that converting direct from SMF, so I said I would ask this for him, if he would use the 2.0 converter would his members still need to reset passwords, as it is done on 2.1?  If so is their a route you can think off that will save his passwords?
Title: Re: 38k Member Forum
Post by: Flavio93Zena on March 15, 2016, 11:26:50 am
As far as I remember, they have a different encription, so I don't think it's possible at all...
Title: Re: 38k Member Forum
Post by: Runic on March 15, 2016, 11:53:11 am
seen that being said for 2.1 but not 2.0 ( read all topics) also their must be an alternate route somewhere :)
Title: Re: 38k Member Forum
Post by: Joshua Dickerson on March 15, 2016, 12:17:27 pm
I don't think you should have to reset the passwords. They should be updated by ElkArte as people login.
Title: Re: 38k Member Forum
Post by: Flavio93Zena on March 15, 2016, 02:00:04 pm
O.o how can it work at all with the old encryption?
Title: Re: 38k Member Forum
Post by: TE on March 15, 2016, 02:59:16 pm
Quote from: Joshua Dickerson – I don't think you should have to reset the passwords. They should be updated by ElkArte as people login.
This.
Title: Re: 38k Member Forum
Post by: Joshua Dickerson on March 15, 2016, 03:27:32 pm
Quote from: Flavio93Zena (#OpIsis) – O.o how can it work at all with the old encryption?
When you login it first checks the ElkArte hash. If that doesn't match, then it tries other strategies until it gets one that matches. It had a bunch of strategies.

You'll still have to login again.
Title: Re: 38k Member Forum
Post by: Spuds on March 15, 2016, 11:06:43 pm
Be sure "Allow password hash conversion" is checked, its under Admin->Configuration->Security/Moderation->General  Its intended to help this issue when sites are converted

Then (to recap what others already said)

A user will attempt login, it will look liked it failed and ask for their password again, at that point it will log them in (assuming of course the password was right). 

After they successfully login, the old password hash will be converted to the new (industry standard) hashing.
Title: Re: 38k Member Forum
Post by: vkot on March 16, 2016, 06:30:04 am
Hi, I'm the guy with the 38K registered members. Thanks Runic for the iniciative to post my question!

Quote from: Spuds – A user will attempt login, it will look liked it failed and ask for their password again, at that point it will log them in (assuming of course the password was right). 
In this case, it seems simple enough.
Is it also that simple for the people that have stored their password on the browser? They will just click the "Log in" button a second time and they will enter, or they'll have to type their password again somewhere else?
Title: Re: 38k Member Forum
Post by: Spuds on March 16, 2016, 08:27:12 am
I don't think the stored passwords / autofill will work (not 100% sure though).

Likely they will need to use the keyboard on both screens.   Those autofills look for matching form names/ids and they are likely not the same between your current platform and ElkArte.
Title: Re: 38k Member Forum
Post by: vkot on March 16, 2016, 08:46:12 am
It would be a small nuisance for those that forgot their password, they would have to reset it.
But what about those that have changed their email account and haven't updated their forum info with the new email and don't have access to their old email account anymore? It would be impossible to reset their password. They would have to message me outside the forum and I have to trust that they are who they say they are and then reset their password manually.
Title: Re: 38k Member Forum
Post by: emanuele on March 16, 2016, 09:04:13 am
I suppose the login can be changed to always use the fallback and never update the password.
Of course it has drawbacks: 1) security, 2) security.
1) if the database is stolen, the hackers will be able to log in as any of your members without even knowing the password,
2) if your users do not know their password, they do not even know if they have reused it anywhere else and as such are more vulnerable to social engineering attacks and password theft.

@Spuds wouldn't be possible (theoretically, not that I have any plan to implement it), to attach some javascript to the login form, encrypt the password "SMF-like" and use this hash as the actual password passed to Elk's encryption js code and then to the server?
Let's illustrate it a bit. :P
Currently:

While, what I'm describing is:
Title: Re: 38k Member Forum
Post by: Runic on March 16, 2016, 08:09:15 pm
Quote from: vkot – Hi, I'm the guy with the 38K registered members. Thanks Runic for the iniciative to post my question!

Παρακαλώ (Your welcome for those that dont know Greek ;) )
Title: Re: 38k Member Forum
Post by: Vekseid on March 16, 2016, 08:59:24 pm
Quote from: Spuds – Be sure "Allow password hash conversion" is checked, its under Admin->Configuration->Security/Moderation->General  Its intended to help this issue when sites are converted

Then (to recap what others already said)

A user will attempt login, it will look liked it failed and ask for their password again, at that point it will log them in (assuming of course the password was right). 

After they successfully login, the old password hash will be converted to the new (industry standard) hashing.

Is there a reason it's this clunky? I'm going to have to fix this before converting E.
Title: Re: 38k Member Forum
Post by: Spuds on March 16, 2016, 10:03:16 pm
Because the form sends an encrypted password on the first pass, which the site then does a validation against.  Once the site knows the password is valid, to one of the many hashing algorithms, it resets the form such so that it returns the password to the server in plain text.  The plain text password is (assuming it once again validates) used to create the new style db hash.   You could eliminate the form login encryption and make the conversion less "clunky".
Title: Re: 38k Member Forum
Post by: Vekseid on March 16, 2016, 10:24:18 pm
Or activate the session and require a password confirmation (just to make it clear).

When inflicting change on a hundred thousand people, it is wise to make it go as smoothly as possible >_>
Title: Re: 38k Member Forum
Post by: Flavio93Zena on March 16, 2016, 11:38:02 pm
Just my 0.02, but I think that stressing it so much about re-typing a password is... Really, really overkill.
Title: Re: 38k Member Forum
Post by: emanuele on March 19, 2016, 08:24:05 pm
@Vekseid what do you mean with "activate the session"?
If it is what I think it could be you'd have to basically maintain the two systems in parallel and use the one used by the member.
I think that, if you really want to avoid any hassle, it would be easier to just drop the migration of the passwords to the new encryption and continue to use the old one.
It's probably the only way that will ensure nobody is going to have to type the password again.
You have to decide which is the trade-off: if you are sure members will not know their password and will have a broken email, then rely on the volatility of a cookie is your only option to make all of them happy. :)
Title: Re: 38k Member Forum
Post by: Vekseid on March 25, 2016, 02:03:29 am
I understand why they'll have to type the password in a second time, it's mostly just a wording change.
Title: Re: 38k Member Forum
Post by: vkot on March 28, 2016, 01:47:31 pm
Quote from: emanuele – I think that, if you really want to avoid any hassle, it would be easier to just drop the migration of the passwords to the new encryption and continue to use the old one.
It's probably the only way that will ensure nobody is going to have to type the password again.
You have to decide which is the trade-off: if you are sure members will not know their password and will have a broken email, then rely on the volatility of a cookie is your only option to make all of them happy. :)

So, there is an (easy) way to keep the old passwords and the old encryption? There is an option somewhere in the migration process?
If I choose to do this, can I (easily) change to the new encryption at a later time?

I am thinking if there is really a big and serious reason to change to the new encryption. How unsafe is the old one? I mean, it΄s not like passwords in plain text. And we have been using this system for almost 15 years... (since the YaBBSE days...)
Title: Re: 38k Member Forum
Post by: Vekseid on March 28, 2016, 02:43:26 pm
The old one has issues but I'm not sure if the new one is any better, I haven't looked at it.
Title: Re: 38k Member Forum
Post by: Spuds on March 28, 2016, 07:59:55 pm
The main issue with the old password hash was, basically if a forum database was compromised, the saved hash could be used  to login to the system ... you did not even need to know what the original password was, 

If a someone used the same password on other sites (with the same forum software), they could use the hash to login there as well.  So your site may be secure, but you were still vulnerable to all other similar sites, and although the original user password was "safe" it did not matter anyway.

With some edits I believe you can have the old work on the new, have not really thought about that, but you did give up some level of security as discussed.
Title: Re: 38k Member Forum
Post by: Vekseid on March 30, 2016, 08:38:45 pm
I know how it works in SMF.

What I was doing for my own custom CMS was to have both the client and server perform some of the hashing work - with the client doing most of it. The client would do sha512 256 times, and send the 255th and 256th hash to the server in base64 (88 characters).

The server would verify the client had hashed correctly, then hash the 256th hash a few more times (along with a salt) before storing/checking the database.

This IMO gives the best of all possible worlds

- It can allow any size of password, it only ever arrives at the server as 88 characters
- This is one of those rare situations where we can verify client-side crypto.
- No more username salt nonsense
- Offloads some cpu cycles to the client.
- Still not sent over the wire in plaintext.
- Still hashed on the server, so getting the database doesn't get you any closer.

Is what I would like to see, personally.
Title: Re: 38k Member Forum
Post by: Frenzie on April 01, 2016, 05:32:07 am
Keep in mind that it should always work with JS disabled or absent.
Title: Re: 38k Member Forum
Post by: Vekseid on April 04, 2016, 03:46:48 am
What follows is a comprehensive list of the sorts of people who visit my site without functional javascript:

1) Spammers
2) Scrapers
3) People who have it disabled via noscript and can trivially enable it for my domain
4) More scrapers
5) More spammers

So no. It is perfectly valid to require functional javascript from people who wish to log in.
Title: Re: 38k Member Forum
Post by: Frenzie on April 04, 2016, 06:29:30 am
/insert snarky reply about a comprehensive list of uses for Javascript

I wonder if you actually checked your statistics before making that statement.[1] You just might be surprised. I looked at my statistics, and their sheer variety managed to do just that. On my forum the most popular browsers in March were (in order) Firefox, various Chrome derivatives,[2] Otter, Opera/Presto, K-Meleon, w3m and ELinks. That's trailed by e.g. Internet Explorer (at a still respectable ~9%), while Edge doesn't even make the top 10. Linux use (not including Android) is 16.1% and BSD is at 1.2%. And keep in mind that bots use a user agent that says Windows with either Firefox or Chrome, so in real terms much of that is likely significantly higher.

Yes, my forum is atypical. But most of us care deeply about this kind of thing. It's why I selected SMF and I've had plenty of positive feedback about how well it works with Javascript disabled and how well it works in older, alternative, and text browsers. Sure, we could trivially open another browser instead of Links2 or Netsurf.[3] Yes, we could trivially enable Javascript. So what? We can also trivially close a site that's broken by design. Would your solution even work with screen readers? In short, know your audience. ElkArte's audience is obviously a lot wider than my audience, but it isn't synonymous with your audience either.

PS Regarding using Javascript to keep out bots, for that purpose it wouldn't make much sense to require it on every login. Requiring JS on the registration page would have the same effect — or lack thereof, depending on the sophistication of the bot. And come to think of it, I think my users may well be a lot more sympathetic to "please enable Javascript to register (this helps keep out bots)" than to "please enable Javascript to login."
Not even necessarily on Javascript use, although I bet you do not in fact have any statistics on that, ;) but particularly on how long it took before people enabled Javascript. And I'm not just saying this to be argumentative; I'd be curious to know.
Many of which on Android. Incidentally, this includes the majority of spammers. Let's ban user agents that say Chrome to prevent spam, shall we?  O:-)
That's me speaking. I don't use w3m and ELinks… yet. I have to admit I've become quite curious about their popularity now.
Title: Re: 38k Member Forum
Post by: emanuele on April 04, 2016, 08:49:42 am
Most of user-facing parts of Elk work without javascript. IIRC, the only "big" feature that require JS  by design is the likes[1], anything else should pretty much work.
I the admin panel there are some pages that may have problems if JS is disabled, since the general consensus has been is acceptable to require JS in the admin panel (and some moderation functions).
that anyway does work without JS, just the interface is not exposed i.e. the button doesn't have the appropriate URL, but if you really want that, it should be rather easy to fix.
Title: Re: 38k Member Forum
Post by: Vekseid on April 04, 2016, 08:59:03 am
You could, you know, just ask. I have dozens of blind members using a variety of screen readers. Yes, they parse javascript, have for decades. Can even read the ajax chat. They will be actively testing the Elkarte 1.1 betas for us. One uses JAWS even. : )

JAWS is important because it tries to outsmart the web designer. >_>

And if you want to see what useragents are bots, swapout your login url. Enjoy. Have not done this in years, but long story short I dropped IE6 support for Elliquiy when I realized every single IE6 user attempting to sign into my forums for the past year was a bot. 98% of the remaining bots were IE7-9. A handful of Firefox 3 & 3.5.

True, 'know your audience' is a thing. I had one lynx user for awhile - he was very vocal about it. I actually had a very simular argument with him.

Fact is, if you want to actually make an account and log in, I don't feel it is too much to ask for you to spare a few cycles to help secure your own password. If you are competent enough to know how to selectively disable/enable javascript, that should not be that much of a leap to understand why.

Ultimately, my forums are writing forums. My members have a reasonable level of sophistication - many use Linux, even - but they don't subject themselves to browsing from the console for the most part. They are there to read and write with each other.

And yeah, SMF working without javascript was a part of what drew me back in 2005. Now? Now I want the interface out of everyone's way.  SMF/Elkarte is due for a colossal refactoring, from multiple angles.
Title: Re: 38k Member Forum
Post by: Frenzie on April 04, 2016, 10:01:46 am
Quote from: Vekseid – You could, you know, just ask. I have dozens of blind members using a variety of screen readers. Yes, they parse javascript, have for decades. Can even read the ajax chat. They will be actively testing the ElkArte 1.1 betas for us. One uses JAWS even. : )
Cool.

Quote from: Vekseid – Fact is, if you want to actually make an account and log in, I don't feel it is too much to ask for you to spare a few cycles to help secure your own password. If you are competent enough to know how to selectively disable/enable javascript, that should not be that much of a leap to understand why.
NB I never said I was opposed to adding extra security, merely that I was opposed to purposefully breaking things with JS disabled. Obviously, from a security standpoint the addition of Javascript does eliminate most potential sniffing dangers. From the top of my head, the only attack I can think of is that someone in the position to sniff may also be in the position to alter or block the JS sent to the user so that they get a nice unhashed password, but that would take more effort. Encryption would guard against the latter possibility, but in that scenario you probably wouldn't gain much from hashed passwords in the first place. (Correct me if I'm wrong.[1])

What I meant to imply when I called for us to "keep in mind that it should always work with JS disabled or absent" is a warning message, for example:

Code: [Select]
<strong class="noJS">Warning: with Javascript disabled your password may be at greater risk of being sniffed by some script kiddie and/or the NSA.</strong>

coupled with, e.g.

Code: [Select]
$('.noJS').css('display', 'none'); // or however one might accomplish that in jQuery

And of course in my proposal you'd also flip the switch on a hidden INPUT element indicating whether or not the password comes prehashed.

Note that a warning or error message along those lines is a necessity regardless whether or not JS-less logins are supported. Javascript may have failed to load even when enabled because it was filtered by a proxy/firewall/malicious entity. As such, in a JS-only scenario there should be no forms on the page to prevent any accidental submissions. To prevent the layout from breaking you could just clone the children of a relevant DIV inside a newly created FORM element.
Edit: It occurs to me that it'd potentially mitigate some risk in case of a compromised server, although again there you have the issue that the attacker could simply alter the JS.
Title: Re: 38k Member Forum
Post by: Vekseid on April 04, 2016, 08:27:03 pm
Well, it stops heartbleed-like sniffing, for example, as well as firesheep.

More pertinent is that by having the client do most of the hashing, you reduce some of the attack footprints against you (you still need to hash, of course, but still). No million-character passwords (an attack against Django for a time), no repeated login attempts eating up cpu cycles, etc.

In my Phoenix implementation it loads the login form in Javscript, yes, displaying a notice if not ('You need javascript in order to sign in or register') - the idea there being that a user should not have to even reload the page to login.
Title: Re: 38k Member Forum
Post by: Frenzie on April 05, 2016, 03:55:04 am
True, if the encryption method is somehow compromised you regain the anti-sniffing advantages. But again I repeat, I'm not objecting to improving the current client-side hashing methods. Although I'd add that your message gave me a much graver impression of what's going on (https://github.com/elkarte/Elkarte/blob/f35fd546fa008f27e2137b80a3e6eb4d1a0e9ac4/themes/default/scripts/script.js#L649), meaning I wasn't really aware it'd been improved compared to SMF. Repeating that salt+hash a number of times (such as 256) would improve security, but I don't know if switching to SHA512 would matter. It's barely any slower. And for server-side hashing, from what I understand you want to use something much stronger like Crypt/Blowfish (or Crypt/SHA512 or Crypt/whatever) regardless.

But could you explain how problems caused by the submission, or rather the acceptance of a million-character password submission would be mitigated? Javascript would be no different than input pattern=".{88,88}". The actual check that anything submitted meets the requirements still needs to take place on the server side of the equation.
[1] Loading the form through JS merely ensures that no one can submit their plain text password by accident.

As far as repeated login attempts go, that would be a temporary but welcome relief. In the long term I think that'll still be primarily dealt with by Bad Behavior, which luckily is already built-in.
Title: Re: 38k Member Forum
Post by: emanuele on April 05, 2016, 04:40:54 am
Quote from: Frenzie – But could you explain how problems caused by the submission, or rather the acceptance of a million-character password submission would be mitigated? Javascript would be no different than input pattern=".{88,88}". The actual check that anything submitted meets the requirements still needs to take place on the server side of the equation.
The Django issue was "simply" there was no limit to the password length (server-side), so you could send an arbitrary long password to the server, it would have to hash it requiring CPU-cycles. Of course, the longer the password, the harder is the hashing, so using a password in the realm of millions of characters would lead to a (D)DoS attack.
Using SHA\d+ the password is a fixed length, so you don't have to set a limit on the password length: if the hash passed to the server is not the expected length you send back to the client a NAY answer without even starting the server-side hashing.
That said, Elk enforces a max-length for the password (see your link), so it is less of a problem except for the few that would like to use a 64+ chars password and no javascript.
Title: Re: 38k Member Forum
Post by: Frenzie on April 05, 2016, 06:41:01 am
QuoteUsing SHA\d+ the password is a fixed length, so you don't have to set a limit on the password length: if the hash passed to the server is not the expected length you send back to the client a NAY answer without even starting the server-side hashing.
Yes, but although you repeated so at the end I just want to emphasize that as far as I can see, the difference between <= and == is a distinction without meaning. Either there's the potential for a DoS attack through extremely expensive computations or there isn't. Anything client-side does nothing to obviate that.