ElkArte Community

General => OpenImporter => Topic started by: Nao on September 20, 2013, 10:05:01 am

Title: Convert SMF to ElkArte
Post by: Nao on September 20, 2013, 10:05:01 am
Quote from: kucing – Thanks @Antechinus and @emanuele !  Now I'm playing with my new toy :DToo bad there is no converter from SMF2 yet, but it's to be expected from a project that still in alpha phase. There are more important things to do than a converter for now.

Cheers! :)
Doesn't OpenImporter have a smf2elk module? Iirc it has modules to import from and to SMF 1, 2 and even Wedge... (which is what TE wrote it for originally.)
The advantage of an importer is that you can easily revert and/or test whether will work on the new system.
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 20, 2013, 10:29:23 am
@Nao : Thanks for the information! I will give it a try later. For anyone that looking the tool mentioned, here is the link for @TE OpenImporter : https://github.com/eurich/openimporter
Title: Re: Convert SMF to ElkArte
Post by: TE on September 20, 2013, 11:09:00 am
The SMF importer isn't published yet.  I have one for Wedge, that file needs some slightly modifications and will be ready for testing soon  ;)
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 20, 2013, 11:27:09 am
@TE : The current OpenImporter is unusable then? I will wait for your updates. :)
Title: Re: Convert SMF to ElkArte
Post by: emanuele on September 20, 2013, 12:46:04 pm
Quote from: kucing – @emanuele : Now you tempt me to try to convert my (smallish) forum. Is there any caveat for converting my forum now? Want to try to live on bleeding edge  8)
I would do some testing before, it's a while I don't test the upgrade lol
Title: Re: Convert SMF to ElkArte
Post by: Nao on September 20, 2013, 03:14:26 pm
Quote from: kucing – @TE : The current OpenImporter is unusable then? I will wait for your updates. :)
It's perfectly usable. I used it a few weeks ago to try and import noisen.com on a local test site.
The SMF module, though, I don't know, never used it ;) anything that still uses so many globals, no thank you :P
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 20, 2013, 07:51:30 pm
Haha... Then I will proceed with caution in my localhost :) Thanks for the information and warning. I will wait patiently at least until beta released to convert my forum, just to be safe.

Cheers :D
Title: Re: Convert SMF to ElkArte
Post by: TE on September 21, 2013, 02:57:43 am
Quote from: kucing – @TE : The current OpenImporter is unusable then? I will wait for your updates. :)
Nah, it's usable but not with SMF as source forum..

Currently I have Importers for vBulletin4, phpBB3, MyBB 1.6, Wordpress 3, Burning Board 3.1 and phpBoost3.
https://github.com/eurich/openimporter/tree/master/importer

The SMF to Elk import skeleton is almost ready, I think I can finish it within the next days..
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 21, 2013, 10:13:48 pm
Another related question, let's say I already installed ElkArte Alpha. Then how do I update it to the latest alpha? Just overwrite it?

Thanks!
Title: Re: Convert SMF to ElkArte
Post by: TE on September 22, 2013, 02:07:05 am
Quote from: kucing – Another related question, let's say I already installed ElkArte Alpha. Then how do I update it to the latest alpha? Just overwrite it?
yep, most of the time replacing files should be enough..

Sometimes you'd need the upgrade scripts (ugrade.php and upgrade_elk_1-0_mysql.sql) because we made database changes.  Adding / changing these is the job of the upgrade scripts.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on September 22, 2013, 02:49:34 am
I was thinking... why don't we change the install to work from the install directory? (And upgrade from upgrade?)
Title: Re: Convert SMF to ElkArte
Post by: TE on September 22, 2013, 04:13:58 am
Quote from: emanuele – I was thinking... why don't we change the install to work from the install directory? (And upgrade from upgrade?)
it might work for most installations / upgrades but I'm sure we'll get open_basedir errors here and there..
Title: Re: Convert SMF to ElkArte
Post by: TE on September 22, 2013, 06:46:43 am
ok, here we are  ;D  ;D  ;D
https://github.com/eurich/openimporter/tree/master/importer
you'd need these files in order to import an SMF 2.0.x: import.php, import_en.xml, smf2_importer.xml
Just upload the three files to your elkarte dir, start import.php via browser, fill in the path to your SMF 2.0 and click "start".. Depending on the size of your SMF it may take  a while...

I've successfully imported my 270k posts SMF  :D
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 22, 2013, 07:40:58 am
Thanks TE! I will try it later on my localhost first, my forum is still too small comparing to yours though.

Now you really tempt me to try ElkArte live on my forum :P
Title: Re: Convert SMF to ElkArte
Post by: Nao on September 22, 2013, 07:56:09 am
Just start with a sub folder with elk in it! You can run them side by side...
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 22, 2013, 08:05:55 am
Thanks for the idea Nao! Will keep that in my mind before going all the way :D
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 23, 2013, 07:50:47 am
I failed to import from SMF and unfortunately I don't know what's wrong. Is the importer must be online? I'm trying it in localhost.

I tried to import from my forum database and empty installation in WAMP, both stopped in the same way. Screenshot attached.

The only differences beside the environment that I can think of is the database don't have password, so I removed it from SMF settings already.



If this post better to be in separate topic then I kindly ask the moderators to split it. Thanks!
Title: Re: Convert SMF to ElkArte
Post by: TE on September 23, 2013, 11:08:06 am
QuoteIf this post better to be in separate topic then I kindly ask the moderators to split it. Thanks!
yep, splitted from the original topic.

Importing on localhost is fine, I've updated the git repo with a newer version of import.php, hopefully that one fixes the error..


Title: Re: Convert SMF to ElkArte
Post by: kucing on September 23, 2013, 07:21:33 pm
Thanks! I've successfully converted my SMF forum to ElkArte, after playing around for a while I decided to postponed the conversion maybe after beta version released.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on September 23, 2013, 07:38:42 pm
Did you find many things broken? O:-)
Feel free to report. ;)
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 23, 2013, 08:00:59 pm
The only thing I found to be broken is I can't view my own profile as member and as admin. Both (screenshots attached) is caused by database error.

Is avatar supposed to be moved over? because it's not showing up too.

I'm using ElkArte with pull#826 merged. Unrelated, but how do I identify specific ElkArte version from GitHub?

Thanks!
Title: Re: Convert SMF to ElkArte
Post by: Spuds on September 23, 2013, 10:17:29 pm
I can comment on the vieworder error ...

We just added the ability to arrange the custom profile fields (drag/drop) and a new field was required.  Thats not normally the case for the daily changes (meaning db changes are not that common) but it does happen.

To fix that you need to open phpmyadmin and then open the custom_fields table, add at the end a new column vieworder, make it a smallint
Title: Re: Convert SMF to ElkArte
Post by: kucing on September 23, 2013, 11:12:06 pm
No worries Spuds! That's why I wait at least until beta, the features and database structures should be decided already, correct? i.e. no breaking changes.

Thanks!
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 20, 2013, 01:44:17 am
Quote from: TE – ok, here we are  ;D  ;D  ;D
https://github.com/eurich/openimporter/tree/master/importer
you'd need these files in order to import an SMF 2.0.x: import.php, import_en.xml, smf2_importer.xml
Just upload the three files to your elkarte dir, start import.php via browser, fill in the path to your SMF 2.0 and click "start".. Depending on the size of your SMF it may take  a while...

I've successfully imported my 270k posts SMF  :D

A little feedback on the importer and a couple of questions.

I successfully imported my forum with 237K posts (just as a test) and everything went fairly smoothly. The only thing is that attachments and uploaded avatars are not showing afterwards. It did copy over the attachments table data and it copied the files from the SMF site attachment folder to the Elkarte site attachment folder but none of them display. Running the "Attachment integrity check" tells me that I have "9092 attachments are in the wrong directory". When I click the checkbox so it will fix it...it doesn't because if I run it again it tells me the same thing. I've double checked the folder settings for attachments and it is correct in the admin panel. Any ideas on that?

That other minor oddity is after the conversion if you click the button to shrink the header it will not expand when you click it again. Testing on a fresh install of Elkarte and it works fine. I can shrink the header and then expand it but on the imported forum if I shrink it the expand no longer works. Any thoughts on this?

My imported site does have Aeva Media and PortaMx portal installed if that makes any difference. Maybe I should have uninstalled those before doing the import?
Title: Re: Convert SMF to ElkArte
Post by: TE on October 20, 2013, 02:44:13 am
Quote from: b4pjoe – When I click the checkbox so it will fix it...it doesn't because if I run it again it tells me the same thing. I've double checked the folder settings for attachments and it is correct in the admin panel. Any ideas on that?
yep, I think it's my fault.. ElkArte has now filename extensions, the importer needs to be fixed, too.


Quote from: b4pjoe – That other minor oddity is after the conversion if you click the button to shrink the header it will not expand when you click it again. Testing on a fresh install of Elkarte and it works fine. I can shrink the header and then expand it but on the imported forum if I shrink it the expand no longer works. Any thoughts on this?
mhh, IIRC that part was changed in Elk, too.. should be easy to fix.

Quote from: b4pjoe – My imported site does have Aeva Media and PortaMx portal installed if that makes any difference. Maybe I should have uninstalled those before doing the import?
installed mods shouldn't be a problem at all.. 
Title: Re: Convert SMF to ElkArte
Post by: Nao on October 20, 2013, 03:56:33 am
Are you implying that AeMe still works on Elk without changes..?
Title: Re: Convert SMF to ElkArte
Post by: TE on October 20, 2013, 05:05:12 am
Quote from: Nao – Are you implying that AeMe still works on Elk without changes..?
Nope, for sure not. It's simply "ignored" while importing the forum.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 20, 2013, 06:16:36 am
ok, the attachments bug should be fixed within the repo. the new imorter should import the old attachments properly.

Quote from: b4pjoe – That other minor oddity is after the conversion if you click the button to shrink the header it will not expand when you click it again. Testing on a fresh install of Elkarte and it works fine. I can shrink the header and then expand it but on the imported forum if I shrink it the expand no longer works. Any thoughts on this?
I've tested this on my local repo and it's working fine.. IIRC the upshrink header is stored in $_COOKIE, can you please test it with a different browser or clear old cookies?
Title: Re: Convert SMF to ElkArte
Post by: Nao on October 20, 2013, 07:33:48 am
Quote from: TE – Nope, for sure not. It's simply "ignored" while importing the forum.
Like on Wedge then.

But even with empty data, does it still work from Elk?
Thought Elk broke all mods compat long ago...
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 20, 2013, 08:10:18 am
Hooks are still the same, Dispatch is coded in such a way to be backward compatible, $smcFunc is still loaded and populated and should work.
Obviously code is changed, so code edits are likely broken.
The new menu is not (yet) backward compatible with mods (but it's not that difficult to make it so).
Title: Re: Convert SMF to ElkArte
Post by: TE on October 20, 2013, 08:59:51 am
The importer is supposed to import the plain forum structure (attachments, members, posts, topics, messages, polls and such stuff), not to convert a modification from SMF to ElkArte. If AEVA would exist for Elk, then it would be possible to import the data by extending the importer files.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 20, 2013, 04:50:38 pm
Quote from: TE – ok, the attachments bug should be fixed within the repo. the new imorter should import the old attachments properly.

Quote from: b4pjoe – That other minor oddity is after the conversion if you click the button to shrink the header it will not expand when you click it again. Testing on a fresh install of Elkarte and it works fine. I can shrink the header and then expand it but on the imported forum if I shrink it the expand no longer works. Any thoughts on this?
I've tested this on my local repo and it's working fine.. IIRC the upshrink header is stored in $_COOKIE, can you please test it with a different browser or clear old cookies?

Thanks, that did fix the attachment issue but the uploaded avatars still don't show up.

I have tested the upshrink header on Safari, Firefox, and Chrome on my Mac. With all 3, if I shrink the header it shrinks. On all 3 it will not expand after that until I delete the cookies from the browser. Then if I shrink it again it will no longer expand until I delete the cookie again. This doesn't happen with a clean install of Elkarte. Only with a converted SMF forum. Also I checked the behavior on IE 11 under Windows 8.1 and it behaves the same as it does on the Mac. On just noticed under my settings it is set to "Use subdomain independent cookies". I have Elkarte installed on a subdomain if that makes any difference.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 21, 2013, 02:17:37 am
ok, i think I found the related DB setting. Can you please test this via phpMyAdmin:

Code: [Select]
DELETE FROM [icode]elkarte_themes[/icode] WHERE variable = 'minmax_preferences';

and 
Code: [Select]
UPDATE [icode]elkarte_members[/icode] SET id_theme = 1;

Not sure 100% if that helps but I guess it's related to the old id_theme, which doesn't exist in ElkArte.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 09:02:02 am
Doing that does bring the header back but once you click the button to shrink the header it adds the minmax preference back into the database and the header will not expand again until you either clear cookies or delete that row from the database again.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 21, 2013, 09:42:22 am
That's very odd.
I can't think of any reason for the header...
Do you have a bit of time to check if there is any js error?
Using Chrome open a page of your test forum and shrink the header, right click on any point of the page (not important where) and select "inspect element", then select "sources", at the bottom you will see a kind of black circle (I think it's an octagon) with two small vertical lines, click on it until the tooltip says "Pause on uncaught exceptions". At that point try to expand the header (try not to hover any other "sensible" element like menus, etc. in order to avoid the developer tool to stop javascript execution, if that happens (javascript stops) press F8 on your keyboard).
Does it stops anywhere?
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 10:18:00 am
Yes, there is a small yellow box on the screen that says "Paused in debugger" and in the right hand side of the screen there is a box for "Call stack" that is expanded and yellow box in there that says "Paused on exception: 'DOMException'."
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 21, 2013, 10:28:12 am
Ok, on the left, there should be a line highlighted, could you copy and paste it here?
Or post a screenshot if you want.
Thank you very much! :D
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 10:52:25 am
Here is the screenshot.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 10:55:42 am
Hmmm...tried performing that again and got a bit of a different screen. Here is the screenshot of it.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 11:05:30 am
A couple of other theme oddities. Selecting the dark or basic theme shows up like attachment #1. Selecting "Light" shows up as in attachment #2. Which is darker.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 21, 2013, 11:33:52 am
AHA!

The news fader!
Got it, thanks!

ETA: this should fix it: in index.template.php
Code: (find) [Select]
						},
funcOnBeforeExpand: function () {startNewsFader();}

Code: (replace with) [Select]
						}',!empty($settings['enable_news']) && !empty($context['random_news_line']) ? ',
funcOnBeforeExpand: function () {startNewsFader();}' : '', '

The alternative is to move startNewsFader to script.js (or theme.js) and do the checks there or something, dunno.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 21, 2013, 11:37:39 am
Quote from: b4pjoe – A couple of other theme oddities. Selecting the dark or basic theme shows up like attachment #1. Selecting "Light" shows up as in attachment #2. Which is darker.
the variants aren't ready yet (still WIP) the one and only working version is IIRC "light."
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 11:40:04 am
Is there anyway to fix the uploaded avatars not coming over in the conversion?
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 11:44:56 am
Quote from: emanuele – AHA!

The news fader!
Got it, thanks!

ETA: this should fix it: in index.template.php
Code: (find) [Select]
						},
funcOnBeforeExpand: function () {startNewsFader();}

Code: ("replace with") [Select]
						}',!empty($settings['enable_news']) && !empty($context['random_news_line']) ? ',
funcOnBeforeExpand: function () {startNewsFader();}' : '', '

The alternative is to move startNewsFader to script.js (or theme.js) and do the checks there or something, dunno.

Yes! That seems to have fixed it. Thank you.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 21, 2013, 11:46:01 am
Quote from: b4pjoe – Is there anyway to fix the uploaded avatars not coming over in the conversion?
;D oh yes ...  The files need .elk,, not .ext as file extension .. fixed it. 
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 21, 2013, 12:28:05 pm
/me hopes to remember to commit the fix
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 12:31:27 pm
Quote from: TE –
Quote from: b4pjoe – Is there anyway to fix the uploaded avatars not coming over in the conversion?
;D oh yes ...  The files need .elk,, not .ext as file extension .. fixed it.

Has that been fixed in the convertor or is that something I need to do manually?
Title: Re: Convert SMF to ElkArte
Post by: TE on October 21, 2013, 01:01:17 pm
the new smf2_importer.xml file is fixed.. 
but you can do that manually, if you like (maybe yo have got just a few uploaded avatars). Don't know which is more time-consuming for you.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 01:38:31 pm
I have a lot of uploaded avatars... :D

I tried running the importer again on my already converted forum and only selecting "Avatars" but that didn't seem to work. Does that mean I have to do the whole conversion over again?
Title: Re: Convert SMF to ElkArte
Post by: TE on October 21, 2013, 02:22:57 pm

Quote from: b4pjoe – I have a lot of uploaded avatars... :D

I tried running the importer again on my already converted forum and only selecting "Avatars" but that didn't seem to work. Does that mean I have to do the whole conversion over again?
 
To be honest, I haven't tested the single step import that much but in theory it should work . Maybe your ElkArte files are outdated, Spuds fixed some remaining file endings on October, 16th. 
https://github.com/eurich/Elkarte/commit/cd9a4a656094964ea417db0a79b575dc1a54980e (https://github.com/eurich/Elkarte/commit/cd9a4a656094964ea417db0a79b575dc1a54980e)
You could check the file Attachments.subs.php (line 1278).. It should be:

Code: [Select]
$destName = empty($avatar_hash) ? $destName : $path . '/' . $attachID . '_' . $avatar_hash . '.elk';
the important part is the .elk at the end of that line.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 21, 2013, 02:44:32 pm
I downloaded the software on Saturday the 19th. I checked the file though and it has .elk at the end.

I'll attempt the full conversion again and see what happens. Providing that Hostgator doesn't suspend my account again for server overload like they did over the weekend during my first few rounds of conversions. ;D
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 22, 2013, 02:47:41 am
I ran the conversion again and I'm still not getting the uploaded avatars to show up. Uploaded avatars are uploaded to the attachment folder on my SMF site and everything in there has the .elk extension instead of the .ext extension.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 22, 2013, 03:07:04 am

Quote from: b4pjoe – I ran the conversion again and I'm still not getting the uploaded avatars to show up. Uploaded avatars are uploaded to the attachment folder on my SMF site and everything in there has the .elk extension instead of the .ext extension.
That sounds correct.. Perhaps a small bug somewhere else ... I'll check it.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 24, 2013, 02:32:24 am
Quote from: b4pjoe – Uploaded avatars are uploaded to the attachment folder on my SMF site and everything in there has the .elk extension instead of the .ext extension.
Sorry, have to ask for more details.. Your SMF attachments have been named .elk? the importer shouldn't touch these files unless you fill in the wrong pathes during import.

Your current SMF install is: Attachments and Avatars both uploaded to the attachments folder? that's SMF's and Elk's default setting and works fine here. I've tested that with both variants of avatars (encrypted filenames and the older ones from SMF 1.1). 

Do you have multiple attachments directory set in SMF? That's IIRC currently not implemented within the importer..

Sorry, currently no clue what's wrong..
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 24, 2013, 05:16:18 am
Sorry for the confusion. I think I stated it wrong earlier.

I only have the one attachment folder and avatars and attachments both upload to that folder.

No the SMF attachments are named like this:

10001_eb3163e2410c299394add90d039441e301d6c9f7

After the conversion the Elk attachments are named as:

10001_eb3163e2410c299394add90d039441e301d6c9f7.elk

And the attachments now convert over fine after your earlier change.

SMF avatars are named in this manner:

avatar_578.png

Some have .gif, .png, .jpg extensions.

In the converted Elk attachment folder none of the files named starting with "avatar_" get copied over. At least the files named starting with "avatar_" are not in the converted Elk attachments folder.
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 24, 2013, 12:53:47 pm
I got this to import the avatars.

On my SMF forum even though uploaded avatars were set to go to the "Attachments directory" the path below it was not set to that same attachments directory. Once I set the path below to be the same as the SMF "Attachments directory" they imported fine. I would have thought since the setting was to use the "Attachments directory" that it would have ignored the path in line below it. You can't even change that path unless you change the setting above it to "Specific directory...", save it, and then change the setting back to using the "Attachment directory" and saving it back again.
Title: Re: Convert SMF to ElkArte
Post by: TE on October 24, 2013, 01:02:06 pm
ah, thanks .. that's a great help for finding and fixing the bug.. Now I'm able to reproduce and fix. Thank you :)

I think I have a clue what's wrong, will take a look at the code soon...
Title: Re: Convert SMF to ElkArte
Post by: b4pjoe on October 24, 2013, 01:33:39 pm
It appears my ban list didn't import either. I don't see anything in the import list so is this by design?
Title: Re: Convert SMF to ElkArte
Post by: Nao on October 26, 2013, 01:33:40 pm
Quote from: TE –
Quote from: b4pjoe – Is there anyway to fix the uploaded avatars not coming over in the conversion?
;D oh yes ...  The files need .elk,, not .ext as file extension .. fixed it.
The .ext extension is from Wedge, innit?  8)
Title: Re: Convert SMF to ElkArte
Post by: Arantor on October 26, 2013, 07:07:31 pm
You mean another thing I broke?
Title: Re: Convert SMF to ElkArte
Post by: TE on October 27, 2013, 10:44:27 pm

Quote from: Nao – The .ext extension is from Wedge, innit?  8)
yep, that's a "relique" from the Wedge importer file I adapted to Elk..
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 30, 2013, 07:23:51 am
TE, a question about your importer: while giving some support for conversions from IPB to SMF I noticed that frequently (more or less) some installations of IPB have some columns and other installs don't have them (usually is related to aim, icq, and alike), is there already a way to avoid the "missing column" errors thrown by mysql every time a converter is run and one of those columns doesn't exist?
Title: Re: Convert SMF to ElkArte
Post by: TE on October 30, 2013, 11:56:25 pm
Nope, would for sure be a cool feature but it's currently not implemented. I have an idea how it could be integrated though, but that's a lot of work and the xml files would need to be completely reorganized.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on October 31, 2013, 03:59:29 am
I didn't check the code, but guessing it works more or less similarly to the SMF converter, I wonder if a SHOW COLUMNS to get the columns and maybe an array_intersect or something to get only the fields that have a valid column would work?
Title: Re: Convert SMF to ElkArte
Post by: kucing on November 26, 2013, 01:18:28 am
reporting failure when importing SMF 2.0.6 to ElkArte Beta 1 using latest OpenImporter.  screenshots attached.

should I added manually the field requested? what is the field type?

edit1:
so I removed 2 fields from the importer : instant_message and pm_receive_from and then statistic recalculation is failed. It seems I chew more than I can swallow. :D

edit2:
well I can access the imported forum just fine, but I'll wait for any advice.

edit3:
I cannot login using the same password, problem in migration?
Title: Re: Convert SMF to ElkArte
Post by: emanuele on November 26, 2013, 02:45:38 am
instant_message has been renamed to  unread_messages
and
pm_receive_from to receive_from
Title: Re: Convert SMF to ElkArte
Post by: kucing on November 26, 2013, 06:01:27 am
thanks! but what should I do now in the importer? when I renamed them it still throwing error.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on November 26, 2013, 06:08:24 am
ehm...okay I'll wait for TE (can't test it right now, sorry)
Title: Re: Convert SMF to ElkArte
Post by: kucing on November 26, 2013, 06:16:05 am
no problem :D I can use the imported forum just fine and briefly checking nothing is missing. The importing process is partially successful by deleting both fields, if the members not using those fields then no data is lost.
Title: Re: Convert SMF to ElkArte
Post by: TE on November 26, 2013, 07:52:21 am
I'll check the db changes later, will keep you updated..
Title: Re: Convert SMF to ElkArte
Post by: kucing on November 26, 2013, 07:53:22 am
Quote from: TE – I'll check the db changes later, will keep you updated..

Thanks TE! :)
Title: Re: Convert SMF to ElkArte
Post by: TE on November 26, 2013, 10:48:36 pm
Sorry, due to some trouble with my internet connection I wasn't able to check..

Good news nonetheless: Eman sent a Pull request with the fixes included, so you just have to re-download the smf_importer.xml and restart the importer.
https://github.com/OpenImporter/openimporter
Title: Re: Convert SMF to ElkArte
Post by: kucing on November 27, 2013, 07:24:56 am
Thanks TE and emanuele, I have redownloaded then upload smf2_importer.xml and restarting the process but it failed me in receive_from field. screenshot attached.
Title: Re: Convert SMF to ElkArte
Post by: TE on November 28, 2013, 12:50:07 pm
fixed in the new version.
Title: Re: Convert SMF to ElkArte
Post by: kucing on November 28, 2013, 08:41:25 pm
Thanks TE! Now the error only in recalculating statistic phase.
Title: Re: Convert SMF to ElkArte
Post by: Nao on December 14, 2013, 04:38:14 pm
Thorsten, does the OpenImporter version of your script fix the infamous path checking problem in the Wedge importer? You know, the error message that shows up even when the path is alright, and that can only be avoided by directly clicking Next after entering the path name..? (i.e. skipping the path check altogether.)
Title: Re: Convert SMF to ElkArte
Post by: TE on December 15, 2013, 06:31:38 am
Quote from: Nao – Thorsten, does the OpenImporter version of your script fix the infamous path checking problem in the Wedge importer? You know, the error message that shows up even when the path is alright, and that can only be avoided by directly clicking Next after entering the path name..? (i.e. skipping the path check altogether.)
TBH, I don't remember that bug, and I also don't know if it's fixed or not.. :P
Title: Re: Convert SMF to ElkArte
Post by: Runic on March 29, 2014, 10:45:05 pm
wanna bump this as I cant see a new topic anywhere about this ... any updates will current one work for beta 2?
Title: Re: Convert SMF to ElkArte
Post by: emanuele on April 03, 2014, 08:03:14 am
Hi TrayBake, this would probably be @TE field, though I can guess that if it worked with beta 1, it will work with anything up to the final 1.0 version. :D
Of course this is a guess about the future, but for now, since Elk 1.0 beta 1 was released, the database schema has not been changed and as such if the converter worked with beta 1, it will work with beta 2 as well! ;D
Title: Re: Convert SMF to ElkArte
Post by: Nao on April 06, 2014, 11:01:39 am
Quote from: TE –
Quote from: Nao – Thorsten, does the OpenImporter version of your script fix the infamous path checking problem in the Wedge importer? You know, the error message that shows up even when the path is alright, and that can only be avoided by directly clicking Next after entering the path name..? (i.e. skipping the path check altogether.)
TBH, I don't remember that bug, and I also don't know if it's fixed or not.. :P
It was half-fixed. There was a bug in the JavaScript testing the path, I fixed that one around the time I discussed this. But there's also another bug left, and I just looked into it and understood what was going on, but I haven't found a fix yet.
The problem is that when the importer script is first loaded, $this->xml is empty, and thus $this->xml->general->settings returns nothing. Because of that, your script replaces it with 'null', and does its test on /foldernull instead of /folder/Settings.php...
I noticed that it only happens on FIRST load, that is, if you reload the page, then it works.

Because of the first load thing, I figured it might be a session problem, and indeed it seems to be.
From what I could gather, you're initializing $_SESSION variables, then determining if a script was passed in the URL, in which case it initializes it. But if there isn't, it doesn't seem to initialize anything.
I'm not 100% sure, I'll need to test, but it looks like if you have just ONE *_to_wedge.xml file in your root folder, then this might trigger the bug.

I'll spend some more time on this.
Title: Re: Convert SMF to ElkArte
Post by: Nao on April 06, 2014, 11:59:28 am
Okay... Found it.

NOT A FUNNY BUG.
OOP is cool, but it can end up a mess, as is the case here.

Now, when it comes to a situation where only one script is in the folder, here's how the importer detects them...

- Importer object is created. Does the session variable hold a script? If yes, decode it, and fill $this->xml. (Currently: none)
- $template->header(), where $import->xml is tested against. (Currently: empty. The 'from' header is initialized as 'null'.)
- step0. Here's where we look for scripts... If more than one is found, ask for which to use. After this point, we go back to the beginning, and we have a session variable. If only one is found, the session variable is directly initialized with the script path, and... We skip to the next step. At that point, the importer script has already shown the header, which has 'null' in it, and we have a valid script with its data, etc in it.

Possible ways to fix:
- Skip $template->header in case $import->xml is empty, and execute it in doStep0 instead.
- Redirect to the same script once the session variable is set.

What do you prefer, @Thorsten..?
Title: Re: Convert SMF to ElkArte
Post by: Nao on April 06, 2014, 04:53:25 pm
I've decided to go with the redirect. (Header location.)
It's just slightly cleaner, and more in line with the multi-script part of the code.
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 06, 2014, 05:55:43 pm
@emanuele yep I tried and got it working:

http://convert.bryandeakin.com/forum/

Now just working on a new look that fits my site better :P
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:00:43 pm
Was your SMF test site in ISO or UTF-8? I'm just wondering how well the converter handles going from SMF/ISO to Elk/UTF-8.

Also, has anyone tested 1.1.x/ISO to Elk/UTF-8 yet?
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 06, 2014, 06:02:46 pm
test site is a direct clone of main site and thats in UTF-8
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:03:59 pm
K. Gonna have to see what disasters happen with ISO/UTF-8.
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 06, 2014, 06:10:27 pm
only one way to find out ant :P
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:11:40 pm
Has Spuds gone and changed the censor strings again? :D
Title: Re: Convert SMF to ElkArte
Post by: scripple on April 06, 2014, 06:12:24 pm
I ran into a bug in the openimporter when I let the Elk install script create the Settings.php file from scratch.  (i.e. I didn't copy the one from the install directory into the main directory.)  Elk works ok but openimporter dies because a variable wasn't set in Settings.php.  Sadly I don't recall what variable it was though.
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 06, 2014, 06:14:38 pm
Is spuds the new grudge?
Title: Re: Convert SMF to ElkArte
Post by: emanuele on April 06, 2014, 06:15:46 pm
Quote from: Antechinus – K. Gonna have to see what disasters happen with ISO/UTF-8.
OpenImporter should convert it automatically (while the upgrader kills itself if it detects a non-utf8 system).
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:16:51 pm
Dunno what Grudge used to get up to. Spuds had my username set to render as Fart Knocker. I was wondering if you typed my username or if you actually typed Wombat.
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:18:11 pm
Quote from: emanuele –
Quote from: Antechinus – K. Gonna have to see what disasters happen with ISO/UTF-8.
OpenImporter should convert it automatically (while the upgrader kills itself if it detects a non-utf8 system).
Ok cool. Might give it a bash with a test 1.1.x db. I have one handy that is fairly sizable, and full of all sorts of mod nastiness.
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 06, 2014, 06:25:53 pm
If you call you the 3 worded insect usually from the Formicidae family it says Wombat.
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:27:48 pm
Ok, so Spuds has been fornicating around again.
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 06, 2014, 06:38:44 pm
pretty much :)
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:50:39 pm
He's like a toddler playing with a new set of blocks. Keeps him happy and out of trouble. :D
Title: Re: Convert SMF to ElkArte
Post by: Antechinus on April 06, 2014, 06:57:47 pm
Spuds with his word censor.

(http://williamsburg-montessori.org/files/2013/01/toddler-blocks.jpg)
Title: Re: Convert SMF to ElkArte
Post by: emanuele on April 07, 2014, 04:25:00 am
The censor table is rather funny. :P
Title: Re: Convert SMF to ElkArte
Post by: Nao on April 08, 2014, 06:00:10 am
Quote from: TrayBake – @emanuele yep I tried and got it working:

http://convert.bryandeakin.com/forum/

Now just working on a new look that fits my site better :P
And you still have no Wedge test site?  O:-)
Title: Re: Convert SMF to ElkArte
Post by: Spuds on April 08, 2014, 08:37:01 pm
LOL, that does look like me :P

Yeah I changed Fart Knocker to Wombat .. thought most would recognize Wombat as you, although Fart Knocker does a certain air about it :P  Be glad to change it back or to something different as well.  

here are only a handful of expletives in the filter and I changed them to odd things just to make them stand out, as well as to entertain myself with my blocks.
Title: Re: Convert SMF to ElkArte
Post by: Runic on April 15, 2014, 08:27:54 am
Quote from: Nao –
Quote from: TrayBake – @emanuele yep I tried and got it working:

http://convert.bryandeakin.com/forum/

Now just working on a new look that fits my site better :P
And you still have no Wedge test site?  O:-)

When I was working on the new site I had everything wedge, elkarte and a fork I was working on that died like 3 days after I forked as I couldnt be bothered ... I went to transfer my new site to the main folder and decided to clear up so selected every folder (including the development folder but left the forum folder) and deleted them all so everything but the forum was deleted ... just not had chance to put Wedge back up ... but give me time :)
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 04, 2014, 09:26:15 am
So after reading all this, where would obtain a converter from SMF to ElkArte?
Title: Re: Convert SMF to ElkArte
Post by: Spuds on May 04, 2014, 09:53:56 am
Use TE's open importer https://github.com/eurich/openimporter It does a fantastic job!
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 04, 2014, 10:10:25 am
Thank you for the link I hate to sound like a newb but are there instructions on where to upload and what to url is need. Something like xxxxxx.com/importer.
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 04, 2014, 11:27:00 am
Got all this figured out but I can't get past step 1 I get this error
Title: Re: Convert SMF to ElkArte
Post by: Spuds on May 04, 2014, 11:51:42 am
Shared hosting I'd guess ... what that means, or should mean, is that in your php.ini or your hosts, they have
Code: [Select]
disable_functions set_time_limit
which prevents the script from asking for more time to run.   Not sure if @TE can update (or if it already has) paused loop execution as a way around that.
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 04, 2014, 11:53:50 am
it's shared and I don't have a php.ini file in that directory, I will wait i can do thanks.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on May 04, 2014, 01:18:29 pm
Anyway that's a bit of an odd configuration, if it is disabled through php.ini on my localhost, the @ silences the error. But in Allan's case case the execution is stopped anyway... O_o

What host are you using?

I think as workaround the line can just be removed, because anyway the importer is paused after a certain time.
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 04, 2014, 01:21:15 pm
I have a reseller account, it was something server side since they had to disable it for me. On a side note everything went great after I got that straightened out.
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 15, 2014, 04:25:30 pm
Has anyone tried converting a SMF 1.1.9 to Elkarte?
Title: Re: Convert SMF to ElkArte
Post by: emanuele on May 15, 2014, 06:10:25 pm
Last time I tried it was something like...2 or three months ago.
It worked AFAIR. Though I can't find it any more, so most likely I deleted everything.
I have to try again...
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 16, 2014, 12:58:24 pm
My employer uses SMF 1.1.9 and was talking about upgrading to 2.0. I been talking to him about Elkarte I was wondering how successful anyone was.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on May 16, 2014, 02:52:49 pm
1.1.9 or 1.1.19? (Asking just to be sure.)

In theory, the upgrade (that has not been packaged in any release), works the same way as SMF, from YABB up to SMF 2.0.
Of course is theory, because in fact it has not been extensively tested.
What I can say is: I tested it during... March I think and I found some bugs that I fixed. Now I'd have to test it again (maybe to find new bugs LOL).
Title: Re: Convert SMF to ElkArte
Post by: Allan on May 16, 2014, 03:52:46 pm
Sorry it is 1.1.19 version, I informed them that that version would be not supported once 2.1 went live. But I have been talking to them about elkarte.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on June 16, 2014, 10:14:47 am
A small update on the subject: I just tried an upgrade (a test, not live) from SMF 1.1.19 and apart few bugs I fixed in the meantime, everything seems to work without problems! :D
The only big problem is that the admin should convert the database to UTF8 before upload the upgrade files, that is a weak point, maybe in the time between the release of the RC and the final version I'll try to extract the converter and make it standalone.
Title: Re: Convert SMF to ElkArte
Post by: Eliana Tamerin on June 18, 2014, 05:20:42 pm
So, does this work with the RC1 release? I have a candidate board I'd like to upgrade to Elk, but I do want to keep the data.
Title: Re: Convert SMF to ElkArte
Post by: Allan on June 19, 2014, 11:06:49 am
Eliana I would give it try, I really had no problems except the share hosting thing, which spuds posted a quick fix for.
Title: Re: Convert SMF to ElkArte
Post by: Jorin on June 19, 2014, 11:10:02 am
Converting my actual SMF 2 to RC1 worked like a charme. Give it a try.
Title: Re: Convert SMF to ElkArte
Post by: Eliana Tamerin on June 22, 2014, 01:45:15 pm
Okay, got it running (the importer does not like to be in a different folder right now, it tries to call install.php in its parent folder, even if the "install path" is pointed somewhere else). But I've got an error:

Code: [Select]
Message: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead
Trace: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead
Line: 2972
File: /path/to/new/elk/import.php

Some other things:
Importer believes Elk version is only Beta 2.
import_en.xml thinks that Wedge is the only software its importing from (line 12).
Title: Re: Convert SMF to ElkArte
Post by: emanuele on June 22, 2014, 03:21:34 pm
The attached is an upgrade package, try to use it like a normal SMF upgrade.

Of course keep a backup, because this is a conversion of the "current" database and not an import.
Title: Re: Convert SMF to ElkArte
Post by: Eliana Tamerin on June 22, 2014, 03:29:57 pm
Quote from: emanuele – The attached is an upgrade package, try to use it like a normal SMF upgrade.

Of course keep a backup, because this is a conversion of the "current" database and not an import.

I live dangerously. :P

Nah, actually I took a backup before I ran the importer anyway.
Title: Re: Convert SMF to ElkArte
Post by: Eliana Tamerin on June 22, 2014, 03:34:57 pm
Actually, @emanuele there's no php upgrade script. Just the SQL files. ;)
Title: Re: Convert SMF to ElkArte
Post by: emanuele on June 22, 2014, 04:39:59 pm
HEH! :P

Here it is.
Title: Re: Convert SMF to ElkArte
Post by: Eliana Tamerin on June 22, 2014, 05:29:29 pm
Silly ema, trix are for kids.

You forgot the index.php file in your upgrade zip as well. I had to dig in upgrade.php to figure out why the script thought I had old or outdated files. You should probably include another check to make sure there is an index.php file. ;)

Also, the old SMF settings.php file was left mostly untouched, so that even changing $maintenance = 2; to 0 didn't undo maintenance mode. I had to import the settings to the Elk settings.php file and then upload that in place of it, before my board would load.

But that's okay, it's still RC. Should probably fix those up before final, though.
Title: Re: Convert SMF to ElkArte
Post by: emanuele on June 22, 2014, 05:34:33 pm
Quote from: Eliana Tamerin – You forgot the index.php file in your upgrade zip as well. I had to dig in upgrade.php to figure out why the script thought I had old or outdated files. You should probably include another check to make sure there is an index.php file. ;)
/me blames... oh heck, I'm alone in the room... darn.

Quote from: Eliana Tamerin – Also, the old SMF settings.php file was left mostly untouched, so that even changing $maintenance = 2; to 0 didn't undo maintenance mode. I had to import the settings to the Elk settings.php file and then upload that in place of it, before my board would load.
O_o
That's pretty odd...
I'll test it again tomorrow.

Quote from: Eliana Tamerin – But that's okay, it's still RC. Should probably fix those up before final, though.
Nahhh... :P