OpenImporter

Topic: OpenImporter (Read 30971 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

OpenImporter

October 14, 2014, 01:01:07 am

Since I can't seem to find a dedicated topic, so here it is.

Yesterday I went a bit mad (...no, wait scratch that I'm already mad, so it was probably it was just I was writing bugs as usual

) and started breaking that one apart too... lol

https://github.com/emanuele45/openimporter/compare/separation?expand=1

I started splitting the code into multiple files, easier to read and maintain.
Then I removed some static classes in favour of objects.
I also tried to start the splitting of the computing and the response (this is currently badly broken).
And then I re-organized a bit also the files.

Since the project is called Open Importer I think it is meant to be able to convert between any two systems, or at least I would love to see something like that, so I started "moving" the code towards that goal.
First step I moved the xml importers into a subdirectory (Importers) and into that one I created another subdir that represents the "destination" platform (for now there is only ElkArte 1.0...).

For now that's all, the first page (script selection) works, but I'm pretty sure everything else is broken and doesn't work (because of the response/template code). lol

ETA: The attached file is a development version, pay attention and always take a backup before trying anything!!

Re: OpenImporter

Reply #1 – October 14, 2014, 02:30:44 am

I think this is great, reminds me alot of the merge system MyBB has, which is actually very good.

Re: OpenImporter

Reply #2 – October 14, 2014, 08:02:58 am

Quote from: emanuele - October 14, 2014, 01:01:07 am

Since the project is called Open Importer I think it is meant to be able to convert between any two systems,

Yep, that's why the name was choosen... Basically it's the same for all systems.. move data from one database structure to another..

I'm fine with merging, even if it is totally broken for now.. Maybe we should create a "developement" branch until it's working?

Re: OpenImporter

Reply #3 – October 14, 2014, 12:17:35 pm

A development branch makes the most sense to me

Re: OpenImporter

Reply #4 – October 14, 2014, 12:35:59 pm

The dev branch could be an idea.
I wouldn't merge that one now, because is quite a mess... lol
It would be better to clean it up a bit and have it at least somehow working.

Re: OpenImporter

Reply #5 – October 14, 2014, 04:47:56 pm

Today some more stuff related to steps 2 and 3, in particular I moved the implementation out the the main import class, into a platform specific file (part into the xml and part into a php).
Since step 2 is common to all the importers for a certain "destination platform" so I moved the code to another class and the xml now just instantiates the class and runs the proper step (this is a bit redundant and I would prefer something less verbose, but for the moment it could work). The same considerations apply to step3.

ETA: I moved a bit further and removed the all the substeps of the step 2 from the xml and delegated everything to the class. The xml now is used to specify the name of the class.

Re: OpenImporter

Reply #6 – October 16, 2014, 02:34:47 pm

Still changing my mind quite frequently.
In the last commits I created pairs of xml and php files, moving part of the <general> tag to php fles.
Steps 2 and 3 are now into a class file, along with some special methods that may be necessary in step 1 (see elkarte1.0_importer.php).
All the needed files are automatically loaded when the importer is chosen and the classes are named following a convention (the ones for the destination platform, ElkArte for now) while the source-specific class names are specified in the <general> section of the xml file.

Re: OpenImporter

Reply #7 – October 20, 2014, 09:14:52 pm

Back here!

One thing that would be terribly useful, would be to "collect" at least one dummy (with fake data) database for each system of the supported platforms. This would allow two things: 1) at some point set up proper testing, and 2) give a base to let people help in refining existing databases.

If you have a database of any platform and you want to help, feel free to post here or get in contact by PM.
Of course the database should be clean from any personal data and any "real-life" reference (so for example the messages should be just dummy messages, even better if a set of "standard" messages are agreed, so that testing the conversions will be much easier and straightforward).

At the moment there are converters from:
* UBB 7.5
* mybb 1.6
* phpBoost 3
* phpBB 3
* smf 2.0
* vBulletin4
* WBB 3
* WordPress 3
* XenForo 1.1
to ElkArte.
I plan on porting the ones available for the Wedge version in the next few days, so there will be two destination platforms.

If you want to help, an easy way is to provide any of these dummy (remember do not post databases with real data in public) databases. If you have a real one and you don't know how to clean it up, feel free to ask.

Databases of open source (including ElkArte) or freeware projects (like Wedge) should be easy to create: just install create a user, do some posts and at least the basics are covered. Of course it would be wonderful to be able to convert also likes, custom fields, etc., so have them in the system would help.

Todo:
* define a list of standard messages and users for all the tests.

Re: OpenImporter

Reply #8 – October 21, 2014, 11:04:46 am

Wordpress? Is there a way to migrate Wordpress blog entries into ElkArte forum posts?

Re: OpenImporter

Reply #9 – October 21, 2014, 11:09:23 am

I didn't test it, so I guess only @TE can tell how "complete" the conversion can be.

Re: OpenImporter

Reply #10 – October 22, 2014, 02:59:06 am

I'd assume that'd work via RSS export/import easily.

Re: OpenImporter

Reply #11 – October 23, 2014, 10:47:40 am

Quote from: Jorin - October 21, 2014, 11:04:46 am

Wordpress? Is there a way to migrate Wordpress blog entries into ElkArte forum posts?

Not just blog entries to Forum Posts .. users and some other things will also be migrated to ElkArte.

Re: OpenImporter

Reply #12 – October 23, 2014, 09:16:26 pm

Sorry if this is not the right topic, but converting from smf with multiple folders for attachments it did not work with this version of open importer

Re: OpenImporter

Reply #13 – October 23, 2014, 09:41:55 pm

The topic it's okay.

Yep, attachments seem a bit broken at the moment.

Re: OpenImporter

Reply #14 – October 24, 2014, 01:09:26 am

https://travis-ci.org/emanuele45/openimporter/builds/38876157

Of course it doesn't mean the conversions are working, just that I added the skeleton of integration testing.