Skip to main content ↑ ↓
OpenImporter Started by emanuele · · Read 100399 times 0 Members and 2 Guests are viewing this topic. previous topic - next topic

Re: OpenImporter

Reply #15 –

I know, I know, just numbers. O:-)
https://scrutinizer-ci.com/g/emanuele45/openimporter/?branch=master
https://scrutinizer-ci.com/g/emanuele45/openimporter/?branch=separation

:D

Well, not really that important, it only means I spend a few days splitting up methods and multiplying files.
Funny thing is that the importer is still badly broken (and I'm not yet sure how to fix it ROFL), but the code apparently looks good! LOL
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #16 –

Well it may not work, but it looks damn good doing it :P

All kidding aside,  that's a pretty awesome change !

Re: OpenImporter

Reply #17 –

Spuds, Mr. Regex, any idea how to convert the following /e to a preg_replace_callback? O:-)
https://github.com/emanuele45/openimporter/blob/separation/importer/OpenImporter/Utils.php#L252
I found something similar in Elk code, but I'm too scared to change a preg_replace involving charsets without tests... O_O
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #18 –

Quote from: Spuds – All kidding aside,  that's a pretty awesome change !
Let's add some more kidding: I went from 3.67 to 7.83 in 15 days and 150 commits.
Elk is at 3.23 how many commits to 3.67? :P
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #19 –

That is pretty much what the function replaceEntities__callback will do ... I replaced those hideous nested ternary create functions with permanent functions since the former hurt my brain :P

So just copy that function to the importer and do
Code: [Select]
preg_replace_callback('~(&#(\d{1,7}|x[0-9a-fA-F]{1,6});)~', 'replaceEntities__callback', $buf);

Now that second regex one seems a bit mysterious to me, I'm trying to see a pattern that that the first does not cover ... well it would capture &#xx123; but thats not a valid code ... maybe it was added for some damaged input?  I'm tempted to say just the first one is enough as the function checks for hex numbers.

Re: OpenImporter

Reply #20 –

Quote from: emanuele –
Quote from: Spuds – All kidding aside,  that's a pretty awesome change !
Let's add some more kidding: I went from 3.67 to 7.83 in 15 days and 150 commits.
Elk is at 3.23 how many commits to 3.67? :P
Well it took 10K to move it from 0 to 3.23, so if my math is right, its a crap load more !

Re: OpenImporter

Reply #21 –

preg_replace_callback replaced: https://github.com/emanuele45/openimporter/commit/e33e8a39004e823265cff8172ef6d7f53b7c2edd
Thanks! ;D

I think I almost finished splitting up and fixing everything:
https://github.com/emanuele45/openimporter/tree/separation
I was able to use it to properly (as far as I can tell) import an SMF database into ElkArte. This is at the moment the only import I'm reasonably sure that works.

There is still some code Wedge-specific around, but I completely isolated it, so that move it out when time comes should be pretty easy.

Now step 2: test all the conversions present so far and write tests for them... actually, before that generic tests for ManageImporter, Importer and XmlProcessor are more important... provided these classes are already testable.
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #22 –

That is nothing short of awesome sauce !

Re: OpenImporter

Reply #23 –

Quote from: emanuele – https://github.com/emanuele45/openimporter/tree/separation
I was able to use it to properly (as far as I can tell) import an SMF database into ElkArte. This is at the moment the only import I'm reasonably sure that works.

Do you want me to test this importer with my SMF?

Re: OpenImporter

Reply #24 –

Are there any  improvements with multiple folders for attachments?  The last importer I test does not import attachment from smf with multiple folders
sorry for my bad english

Re: OpenImporter

Reply #25 –

I was re-checking the code and there is still an edge case with avatars, I think I fixed it in MyBB, but if you have the avatars stored as attachments it may not be able to import them...

@Jorin of course any testing is very welcome!

@radu81 in theory (as usual lol) it should work, it should also move the attachments to the corresponding directory they were before (and create such directory if it doesn't exist), though at the moment it's all untested code...
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #26 –

radu81 did you get the mention for the post above that one?
I see a strange behaviour I'm not sure about.
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #28 –

Quote from: emanuele – ...but if you have the avatars stored as attachments it may not be able to import them...

I will try and test migrate tonight, at the same time with building nice houses on the lake in Minecraft on my PS4.  O:-)
 

Re: OpenImporter

Reply #29 –

I would like to try the new importer, but is completely different than the old one

Can I have some basic indications on how to proceed? thanks in advance
sorry for my bad english