Doesn't OpenImporter have a smf2elk module? Iirc it has modules to import from and to SMF 1, 2 and even Wedge... (which is what TE wrote it for originally.)
The advantage of an importer is that you can easily revert and/or test whether will work on the new system.
@Nao : Thanks for the information! I will give it a try later. For anyone that looking the tool mentioned, here is the link for @TE OpenImporter : https://github.com/eurich/openimporter
The SMF importer isn't published yet. I have one for Wedge, that file needs some slightly modifications and will be ready for testing soon ;)
@TE : The current OpenImporter is unusable then? I will wait for your updates. :)
I would do some testing before, it's a while I don't test the upgrade lol
It's perfectly usable. I used it a few weeks ago to try and import noisen.com on a local test site.
The SMF module, though, I don't know, never used it ;) anything that still uses so many globals, no thank you :P
Haha... Then I will proceed with caution in my localhost :) Thanks for the information and warning. I will wait patiently at least until beta released to convert my forum, just to be safe.
Cheers :D
Nah, it's usable but not with SMF as source forum..
Currently I have Importers for vBulletin4, phpBB3, MyBB 1.6, Wordpress 3, Burning Board 3.1 and phpBoost3.
https://github.com/eurich/openimporter/tree/master/importer
The SMF to Elk import skeleton is almost ready, I think I can finish it within the next days..
Another related question, let's say I already installed ElkArte Alpha. Then how do I update it to the latest alpha? Just overwrite it?
Thanks!
yep, most of the time replacing files should be enough..
Sometimes you'd need the upgrade scripts (ugrade.php and upgrade_elk_1-0_mysql.sql) because we made database changes. Adding / changing these is the job of the upgrade scripts.
I was thinking... why don't we change the install to work from the install directory? (And upgrade from upgrade?)
it might work for most installations / upgrades but I'm sure we'll get open_basedir errors here and there..
ok, here we are ;D ;D ;D
https://github.com/eurich/openimporter/tree/master/importer
you'd need these files in order to import an SMF 2.0.x: import.php, import_en.xml, smf2_importer.xml
Just upload the three files to your elkarte dir, start import.php via browser, fill in the path to your SMF 2.0 and click "start".. Depending on the size of your SMF it may take a while...
I've successfully imported my 270k posts SMF :D
Thanks TE! I will try it later on my localhost first, my forum is still too small comparing to yours though.
Now you really tempt me to try ElkArte live on my forum :P
Just start with a sub folder with elk in it! You can run them side by side...
Thanks for the idea Nao! Will keep that in my mind before going all the way :D
I failed to import from SMF and unfortunately I don't know what's wrong. Is the importer must be online? I'm trying it in localhost.
I tried to import from my forum database and empty installation in WAMP, both stopped in the same way. Screenshot attached.
The only differences beside the environment that I can think of is the database don't have password, so I removed it from SMF settings already.
If this post better to be in separate topic then I kindly ask the moderators to split it. Thanks!
yep, splitted from the original topic.
Importing on localhost is fine, I've updated the git repo with a newer version of import.php, hopefully that one fixes the error..
Thanks! I've successfully converted my SMF forum to ElkArte, after playing around for a while I decided to postponed the conversion maybe after beta version released.
Did you find many things broken? O:-)
Feel free to report. ;)
The only thing I found to be broken is I can't view my own profile as member and as admin. Both (screenshots attached) is caused by database error.
Is avatar supposed to be moved over? because it's not showing up too.
I'm using ElkArte with pull#826 merged. Unrelated, but how do I identify specific ElkArte version from GitHub?
Thanks!
I can comment on the vieworder error ...
We just added the ability to arrange the custom profile fields (drag/drop) and a new field was required. Thats not normally the case for the daily changes (meaning db changes are not that common) but it does happen.
To fix that you need to open phpmyadmin and then open the custom_fields table, add at the end a new column vieworder, make it a smallint
No worries Spuds! That's why I wait at least until beta, the features and database structures should be decided already, correct? i.e. no breaking changes.
Thanks!
A little feedback on the importer and a couple of questions.
I successfully imported my forum with 237K posts (just as a test) and everything went fairly smoothly. The only thing is that attachments and uploaded avatars are not showing afterwards. It did copy over the attachments table data and it copied the files from the SMF site attachment folder to the Elkarte site attachment folder but none of them display. Running the "Attachment integrity check" tells me that I have "9092 attachments are in the wrong directory". When I click the checkbox so it will fix it...it doesn't because if I run it again it tells me the same thing. I've double checked the folder settings for attachments and it is correct in the admin panel. Any ideas on that?
That other minor oddity is after the conversion if you click the button to shrink the header it will not expand when you click it again. Testing on a fresh install of Elkarte and it works fine. I can shrink the header and then expand it but on the imported forum if I shrink it the expand no longer works. Any thoughts on this?
My imported site does have Aeva Media and PortaMx portal installed if that makes any difference. Maybe I should have uninstalled those before doing the import?
yep, I think it's my fault.. ElkArte has now filename extensions, the importer needs to be fixed, too.
mhh, IIRC that part was changed in Elk, too.. should be easy to fix.
installed mods shouldn't be a problem at all..
Are you implying that AeMe still works on Elk without changes..?
Nope, for sure not. It's simply "ignored" while importing the forum.
ok, the attachments bug should be fixed within the repo. the new imorter should import the old attachments properly.
I've tested this on my local repo and it's working fine.. IIRC the upshrink header is stored in $_COOKIE, can you please test it with a different browser or clear old cookies?
Like on Wedge then.
But even with empty data, does it still work from Elk?
Thought Elk broke all mods compat long ago...
Hooks are still the same, Dispatch is coded in such a way to be backward compatible, $smcFunc is still loaded and populated and should work.
Obviously code is changed, so code edits are likely broken.
The new menu is not (yet) backward compatible with mods (but it's not that difficult to make it so).
The importer is supposed to import the plain forum structure (attachments, members, posts, topics, messages, polls and such stuff), not to convert a modification from SMF to ElkArte. If AEVA would exist for Elk, then it would be possible to import the data by extending the importer files.
Thanks, that did fix the attachment issue but the uploaded avatars still don't show up.
I have tested the upshrink header on Safari, Firefox, and Chrome on my Mac. With all 3, if I shrink the header it shrinks. On all 3 it will not expand after that until I delete the cookies from the browser. Then if I shrink it again it will no longer expand until I delete the cookie again. This doesn't happen with a clean install of Elkarte. Only with a converted SMF forum. Also I checked the behavior on IE 11 under Windows 8.1 and it behaves the same as it does on the Mac. On just noticed under my settings it is set to "Use subdomain independent cookies". I have Elkarte installed on a subdomain if that makes any difference.
ok, i think I found the related DB setting. Can you please test this via phpMyAdmin:
DELETE FROM [icode]elkarte_themes[/icode] WHERE variable = 'minmax_preferences';
and
UPDATE [icode]elkarte_members[/icode] SET id_theme = 1;
Not sure 100% if that helps but I guess it's related to the old id_theme, which doesn't exist in ElkArte.
Doing that does bring the header back but once you click the button to shrink the header it adds the minmax preference back into the database and the header will not expand again until you either clear cookies or delete that row from the database again.
That's very odd.
I can't think of any reason for the header...
Do you have a bit of time to check if there is any js error?
Using Chrome open a page of your test forum and shrink the header, right click on any point of the page (not important where) and select "inspect element", then select "sources", at the bottom you will see a kind of black circle (I think it's an octagon) with two small vertical lines, click on it until the tooltip says "Pause on uncaught exceptions". At that point try to expand the header (try not to hover any other "sensible" element like menus, etc. in order to avoid the developer tool to stop javascript execution, if that happens (javascript stops) press F8 on your keyboard).
Does it stops anywhere?
Yes, there is a small yellow box on the screen that says "Paused in debugger" and in the right hand side of the screen there is a box for "Call stack" that is expanded and yellow box in there that says "Paused on exception: 'DOMException'."
Ok, on the left, there should be a line highlighted, could you copy and paste it here?
Or post a screenshot if you want.
Thank you very much! :D
Here is the screenshot.
Hmmm...tried performing that again and got a bit of a different screen. Here is the screenshot of it.
A couple of other theme oddities. Selecting the dark or basic theme shows up like attachment #1. Selecting "Light" shows up as in attachment #2. Which is darker.
AHA!
The news fader!
Got it, thanks!
ETA: this should fix it: in index.template.php
},
funcOnBeforeExpand: function () {startNewsFader();}
}',!empty($settings['enable_news']) && !empty($context['random_news_line']) ? ',
funcOnBeforeExpand: function () {startNewsFader();}' : '', '
The alternative is to move startNewsFader to script.js (or theme.js) and do the checks there or something, dunno.
the variants aren't ready yet (still WIP) the one and only working version is IIRC "light."
Is there anyway to fix the uploaded avatars not coming over in the conversion?
Yes! That seems to have fixed it. Thank you.
;D oh yes ... The files need .elk,, not .ext as file extension .. fixed it.
/me hopes to remember to commit the fix
Has that been fixed in the convertor or is that something I need to do manually?
the new smf2_importer.xml file is fixed..
but you can do that manually, if you like (maybe yo have got just a few uploaded avatars). Don't know which is more time-consuming for you.
I have a lot of uploaded avatars... :D
I tried running the importer again on my already converted forum and only selecting "Avatars" but that didn't seem to work. Does that mean I have to do the whole conversion over again?
I downloaded the software on Saturday the 19th. I checked the file though and it has .elk at the end.
I'll attempt the full conversion again and see what happens. Providing that Hostgator doesn't suspend my account again for server overload like they did over the weekend during my first few rounds of conversions. ;D
I ran the conversion again and I'm still not getting the uploaded avatars to show up. Uploaded avatars are uploaded to the attachment folder on my SMF site and everything in there has the .elk extension instead of the .ext extension.
That sounds correct.. Perhaps a small bug somewhere else ... I'll check it.
Sorry, have to ask for more details.. Your
SMF attachments have been named .elk? the importer shouldn't touch these files unless you fill in the wrong pathes during import.
Your current SMF install is: Attachments and Avatars both uploaded to the attachments folder? that's SMF's and Elk's default setting and works fine here. I've tested that with both variants of avatars (encrypted filenames and the older ones from SMF 1.1).
Do you have multiple attachments directory set in SMF? That's IIRC currently not implemented within the importer..
Sorry, currently no clue what's wrong..
Sorry for the confusion. I think I stated it wrong earlier.
I only have the one attachment folder and avatars and attachments both upload to that folder.
No the SMF attachments are named like this:
10001_eb3163e2410c299394add90d039441e301d6c9f7
After the conversion the Elk attachments are named as:
10001_eb3163e2410c299394add90d039441e301d6c9f7.elk
And the attachments now convert over fine after your earlier change.
SMF avatars are named in this manner:
avatar_578.png
Some have .gif, .png, .jpg extensions.
In the converted Elk attachment folder none of the files named starting with "avatar_" get copied over. At least the files named starting with "avatar_" are not in the converted Elk attachments folder.
I got this to import the avatars.
On my SMF forum even though uploaded avatars were set to go to the "Attachments directory" the path below it was not set to that same attachments directory. Once I set the path below to be the same as the SMF "Attachments directory" they imported fine. I would have thought since the setting was to use the "Attachments directory" that it would have ignored the path in line below it. You can't even change that path unless you change the setting above it to "Specific directory...", save it, and then change the setting back to using the "Attachment directory" and saving it back again.
ah, thanks .. that's a great help for finding and fixing the bug.. Now I'm able to reproduce and fix. Thank you :)
I think I have a clue what's wrong, will take a look at the code soon...
It appears my ban list didn't import either. I don't see anything in the import list so is this by design?
The .ext extension is from Wedge, innit? 8)
You mean another thing I broke?
yep, that's a "relique" from the Wedge importer file I adapted to Elk..
TE, a question about your importer: while giving some support for conversions from IPB to SMF I noticed that frequently (more or less) some installations of IPB have some columns and other installs don't have them (usually is related to aim, icq, and alike), is there already a way to avoid the "missing column" errors thrown by mysql every time a converter is run and one of those columns doesn't exist?
Nope, would for sure be a cool feature but it's currently not implemented. I have an idea how it could be integrated though, but that's a lot of work and the xml files would need to be completely reorganized.
I didn't check the code, but guessing it works more or less similarly to the SMF converter, I wonder if a SHOW COLUMNS to get the columns and maybe an array_intersect or something to get only the fields that have a valid column would work?
reporting failure when importing SMF 2.0.6 to ElkArte Beta 1 using latest OpenImporter. screenshots attached.
should I added manually the field requested? what is the field type?
edit1:
so I removed 2 fields from the importer : instant_message and pm_receive_from and then statistic recalculation is failed. It seems I chew more than I can swallow. :D
edit2:
well I can access the imported forum just fine, but I'll wait for any advice.
edit3:
I cannot login using the same password, problem in migration?
instant_message has been renamed to unread_messages
and
pm_receive_from to receive_from
thanks! but what should I do now in the importer? when I renamed them it still throwing error.
ehm...okay I'll wait for TE (can't test it right now, sorry)
no problem :D I can use the imported forum just fine and briefly checking nothing is missing. The importing process is partially successful by deleting both fields, if the members not using those fields then no data is lost.
I'll check the db changes later, will keep you updated..
Sorry, due to some trouble with my internet connection I wasn't able to check..
Good news nonetheless: Eman sent a Pull request with the fixes included, so you just have to re-download the smf_importer.xml and restart the importer.
https://github.com/OpenImporter/openimporter
Thanks TE and emanuele, I have redownloaded then upload smf2_importer.xml and restarting the process but it failed me in receive_from field. screenshot attached.
fixed in the new version.
Thanks TE! Now the error only in recalculating statistic phase.
Thorsten, does the OpenImporter version of your script fix the infamous path checking problem in the Wedge importer? You know, the error message that shows up even when the path is alright, and that can only be avoided by directly clicking Next after entering the path name..? (i.e. skipping the path check altogether.)
TBH, I don't remember that bug, and I also don't know if it's fixed or not.. :P
wanna bump this as I cant see a new topic anywhere about this ... any updates will current one work for beta 2?
Hi TrayBake, this would probably be
@TE field, though I can guess that if it worked with beta 1, it will work with anything up to the final 1.0 version. :D
Of course this is a guess about the future, but for now, since Elk 1.0 beta 1 was released, the database schema has not been changed and as such if the converter worked with beta 1, it will work with beta 2 as well! ;D
It was half-fixed. There was a bug in the JavaScript testing the path, I fixed that one around the time I discussed this. But there's also another bug left, and I just looked into it and understood what was going on, but I haven't found a fix yet.
The problem is that when the importer script is first loaded, $this->xml is empty, and thus $this->xml->general->settings returns nothing. Because of that, your script replaces it with 'null', and does its test on /foldernull instead of /folder/Settings.php...
I noticed that it only happens on FIRST load, that is, if you reload the page, then it works.
Because of the first load thing, I figured it might be a session problem, and indeed it seems to be.
From what I could gather, you're initializing $_SESSION variables, then determining if a script was passed in the URL, in which case it initializes it. But if there isn't, it doesn't seem to initialize anything.
I'm not 100% sure, I'll need to test, but it looks like if you have just ONE *_to_wedge.xml file in your root folder, then this might trigger the bug.
I'll spend some more time on this.
Okay... Found it.
NOT A FUNNY BUG.
OOP is cool, but it can end up a mess, as is the case here.
Now, when it comes to a situation where only one script is in the folder, here's how the importer detects them...
- Importer object is created. Does the session variable hold a script? If yes, decode it, and fill $this->xml. (Currently: none)
- $template->header(), where $import->xml is tested against. (Currently: empty. The 'from' header is initialized as 'null'.)
- step0. Here's where we look for scripts... If more than one is found, ask for which to use. After this point, we go back to the beginning, and we have a session variable. If only one is found, the session variable is directly initialized with the script path, and... We skip to the next step. At that point, the importer script has already shown the header, which has 'null' in it, and we have a valid script with its data, etc in it.
Possible ways to fix:
- Skip $template->header in case $import->xml is empty, and execute it in doStep0 instead.
- Redirect to the same script once the session variable is set.
What do you prefer,
@Thorsten..?
I've decided to go with the redirect. (Header location.)
It's just slightly cleaner, and more in line with the multi-script part of the code.
@emanuele yep I tried and got it working:
http://convert.bryandeakin.com/forum/
Now just working on a new look that fits my site better :P
Was your SMF test site in ISO or UTF-8? I'm just wondering how well the converter handles going from SMF/ISO to Elk/UTF-8.
Also, has anyone tested 1.1.x/ISO to Elk/UTF-8 yet?
test site is a direct clone of main site and thats in UTF-8
K. Gonna have to see what disasters happen with ISO/UTF-8.
only one way to find out ant :P
Has Spuds gone and changed the censor strings again? :D
I ran into a bug in the openimporter when I let the Elk install script create the Settings.php file from scratch. (i.e. I didn't copy the one from the install directory into the main directory.) Elk works ok but openimporter dies because a variable wasn't set in Settings.php. Sadly I don't recall what variable it was though.
Is spuds the new grudge?
OpenImporter should convert it automatically (while the upgrader kills itself if it detects a non-utf8 system).
Dunno what Grudge used to get up to. Spuds had my username set to render as Fart Knocker. I was wondering if you typed my username or if you actually typed Wombat.
Ok cool. Might give it a bash with a test 1.1.x db. I have one handy that is fairly sizable, and full of all sorts of mod nastiness.
If you call you the 3 worded insect usually from the Formicidae family it says Wombat.
Ok, so Spuds has been fornicating around again.
pretty much :)
He's like a toddler playing with a new set of blocks. Keeps him happy and out of trouble. :D
Spuds with his word censor.
The censor table is rather funny. :P
And you still have no Wedge test site? O:-)
LOL, that does look like me :P
Yeah I changed Fart Knocker to Wombat .. thought most would recognize Wombat as you, although Fart Knocker does a certain air about it :P Be glad to change it back or to something different as well.
here are only a handful of expletives in the filter and I changed them to odd things just to make them stand out, as well as to entertain myself with my blocks.
When I was working on the new site I had everything wedge, elkarte and a fork I was working on that died like 3 days after I forked as I couldnt be bothered ... I went to transfer my new site to the main folder and decided to clear up so selected every folder (including the development folder but left the forum folder) and deleted them all so everything but the forum was deleted ... just not had chance to put Wedge back up ... but give me time :)
So after reading all this, where would obtain a converter from SMF to ElkArte?
Use TE's open importer https://github.com/eurich/openimporter It does a fantastic job!
Thank you for the link I hate to sound like a newb but are there instructions on where to upload and what to url is need. Something like xxxxxx.com/importer.
Got all this figured out but I can't get past step 1 I get this error
Shared hosting I'd guess ... what that means, or should mean, is that in your php.ini or your hosts, they have
disable_functions set_time_limit
which prevents the script from asking for more time to run. Not sure if
@TE can update (or if it already has) paused loop execution as a way around that.
it's shared and I don't have a php.ini file in that directory, I will wait i can do thanks.
Anyway that's a bit of an odd configuration, if it is disabled through php.ini on my localhost, the @ silences the error. But in Allan's case case the execution is stopped anyway... O_o
What host are you using?
I think as workaround the line can just be removed, because anyway the importer is paused after a certain time.
I have a reseller account, it was something server side since they had to disable it for me. On a side note everything went great after I got that straightened out.
Has anyone tried converting a SMF 1.1.9 to Elkarte?
Last time I tried it was something like...2 or three months ago.
It worked AFAIR. Though I can't find it any more, so most likely I deleted everything.
I have to try again...
My employer uses SMF 1.1.9 and was talking about upgrading to 2.0. I been talking to him about Elkarte I was wondering how successful anyone was.
1.1.9 or 1.1.19? (Asking just to be sure.)
In theory, the upgrade (that has not been packaged in any release), works the same way as SMF, from YABB up to SMF 2.0.
Of course is theory, because in fact it has not been extensively tested.
What I can say is: I tested it during... March I think and I found some bugs that I fixed. Now I'd have to test it again (maybe to find new bugs LOL).
Sorry it is 1.1.19 version, I informed them that that version would be not supported once 2.1 went live. But I have been talking to them about elkarte.
A small update on the subject: I just tried an upgrade (a test, not live) from SMF 1.1.19 and apart few bugs I fixed in the meantime, everything seems to work without problems! :D
The only big problem is that the admin should convert the database to UTF8 before upload the upgrade files, that is a weak point, maybe in the time between the release of the RC and the final version I'll try to extract the converter and make it standalone.
So, does this work with the RC1 release? I have a candidate board I'd like to upgrade to Elk, but I do want to keep the data.
Eliana I would give it try, I really had no problems except the share hosting thing, which spuds posted a quick fix for.
Converting my actual SMF 2 to RC1 worked like a charme. Give it a try.
Okay, got it running (the importer does
not like to be in a different folder right now, it tries to call install.php in its parent folder, even if the "install path" is pointed somewhere else). But I've got an error:
Message: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead
Trace: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead
Line: 2972
File: /path/to/new/elk/import.php
Some other things:
Importer believes Elk version is only Beta 2.
import_en.xml thinks that Wedge is the only software its importing from (line 12).
The attached is an upgrade package, try to use it like a normal SMF upgrade.
Of course keep a backup, because this is a conversion of the "current" database and not an import.
I live dangerously. :P
Nah, actually I took a backup before I ran the importer anyway.
Actually,
@emanuele there's no php upgrade script. Just the SQL files. ;)
HEH! :P
Here it is.
Silly ema, trix are for kids.
You forgot the index.php file in your upgrade zip as well. I had to dig in upgrade.php to figure out why the script thought I had old or outdated files. You should probably include another check to make sure there is an index.php file. ;)
Also, the old SMF settings.php file was left mostly untouched, so that even changing $maintenance = 2; to 0 didn't undo maintenance mode. I had to import the settings to the Elk settings.php file and then upload that in place of it, before my board would load.
But that's okay, it's still RC. Should probably fix those up before final, though.
/me blames... oh heck, I'm alone in the room... darn.
O_o
That's pretty odd...
I'll test it again tomorrow.
Nahhh... :P