Skip to main content
Topic: OpenImporter (Read 59353 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: OpenImporter

Reply #105

Regex question of the day.
I have added a dummy class to mimic a database.
The class takes a query string, parses the SELECT and returns an array with the appropriate indexes (sort of).
To do so I used the following code:
Code: [Select]
		// clean tabs
$string = trim($string);
// remove SELECT
$string = trim(substr($string, 6));
$from = strpos($string, 'FROM {');
$string = substr($string, 0, $from);
$chunks = array_map('trim', preg_split('~,(?! [^(]*\))~i', str_replace("\n", '', $string)));

$array = array();
foreach ($chunks as $chunk)
{
$res = explode(' as ', strtolower($chunk));
if (count($res) == 2)
$array[$res[1]] = '';
else
$array[$res[0]] = '';
}
The idea looked simple: strip anything that is not necessary, split on the commas and do something about the "as".
It works on things like:
Code: [Select]
 SELECT something, something2 FROM whatever
and:
Code: [Select]
 SELECT something, IFNULL(something2, 0) FROM whatever
But of course we have weirder things like:
Code: [Select]
 SELECT something, '1, 0' as something2 FROM whatever
and here it fails.
I could add the single quotes to the regex I already used, something like:
Code: [Select]
preg_split('~,(?! [^(\']*[\)\'])~i', 
though we have even more weird things like:
Code: [Select]
 SELECT usepostcounts != 'yes' AS count_posts, '-1,0' AS member_groups FROM whatever
and that regex would fail.

Is there any way to split a similar string simply based on a regexp?
Or do I have to write a parser?
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #106

Scratch the previous message, since I was bored, I wrote a basic "parser".
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #107

Quote from: emanuele – Actually, now I have to understand if keep the two set of tests separated is acceptable.
Then, if it is, I have to find a way to merge the two coverage reports, while if it not acceptable, I have to find a way to merge them...

Maybe @Spuds has some idea about it. O:-)
Scruit will combine coverage sets, you just have to tell it to expect X number of coverage result files for a run.  https://scrutinizer-ci.com/blog/support-for-merging-code-coverage-data

Re: OpenImporter

Reply #108

Any idea if this works with IPB 3.4.x or plans to add it very soon like today?   ;)

Re: OpenImporter

Reply #109

Nope, IPB is not among the alternatives at the moment and I don't even have a way to write such an importer... (i.e. I don't have an IBP 3.4.x database to test it.)
Of course, if you have one and you are willing to let me play with it I can try, but it's very unlikely it will become ready for today.
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #110

Oh I wasn't expecting it today just a little humor. Would do you need I can hook you up.

Re: OpenImporter

Reply #111

lol

Yep, would be useful to have the importer available.
What is needed is of course the database (do not post it here, if you have any space to securely (i.e. no file sharing services, and a password protected zip) upload it, it's better, if not drop me a PM we can find a way), and is possible the configuration file (cleaned up of the database passwords and usenames at least, but do not remove the row, just put some xxxxxx where the password and usename is).
With that I should be able to have a look and write the necessary importer.

Of course I will not distribute the database nor part of it, and I will not read the content except few random bits to verify for example the proper importing of the messages in the different format. What else? Nothing I guess.
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #112

I just sent you a pm.

Re: OpenImporter

Reply #113

FWIS, this evening I did the members conversion. There are three todo's pending:
Code: [Select]
			// @todo check avatars conversion
// @todo find a way to port titles (post-based groups)
// @todo warn_level what is the value?
The first will be "fixed" as soon as I try a run, the other two may took a bit more.
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #114

@emanuele  thank you and please take your time there is no rush.


Re: OpenImporter

Reply #116

It could be  ;D, really turned off by IPB, I thought it would be better then it is. Thankfully I got it for free.

 

Re: OpenImporter

Reply #117

I decided to do one or two steps a day, at the moment the following are completed (at least theoretically):
members
attachments (no idea if it works unfortunately)
I skipped avatars, maybe I'll do it later
categories
boards
topics
* messages
pending: polls, PMs, several logs, member groups, permissions, custom fields, and likes (and I see the converter doesn't have (yet) a mentions/alerts/notifications/whatever section.

It's a good exercise this converter! :D
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #118

Quote from: emanuele – I decided to do one or two steps a day,
Unless of course I have two/three/five days of mess in RL... LOL
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #119

That damn RL  gets in the way of everything  :P