Re: OpenImporter Reply #91 – December 16, 2014, 12:26:07 pm I just posted a "technical" thought at github:https://github.com/OpenImporter/openimporter/issues/38
Re: OpenImporter Reply #92 – December 28, 2014, 05:22:13 pm Quick question:https://scrutinizer-ci.com/g/emanuele45/openimporter/inspections/50596641-ced4-4df1-85ed-ff45d174d2e9/issues/files/importer/Importers/sources/phpbb3_importer.php?status=existing&orderField=path&order=aschttps://github.com/emanuele45/openimporter/blob/abstraction/importer/Importers/sources/phpbb3_importer.php#L48Am I dumb of the method fixBbc is defined 16 lines above the point is used?
Re: OpenImporter Reply #94 – December 28, 2014, 06:38:38 pm Probably just because it's used only from the inside of the class.I can try with a public and see if something changes.
Re: OpenImporter Reply #95 – December 28, 2014, 10:55:13 pm May help or you may have to tweak it to the parent ... I always get confused by where the protect methods are available when you extend like that.
Re: OpenImporter Reply #96 – December 29, 2014, 03:50:23 am Yeah, but in theory it's in the very same class, not inherited, not extended, etc.Dunno, from time to time Scrutinizer reports odd things. LOLAnyway, I'm about 30 blocks of xml away from converting all the "exporters", that means a couple of days away from lots of testing! Darn! xDBut also a couple of days away from the (first step of the) original goal! YAY!In other words, when I have finished converting these blocks, OpenImporter should be able to provide a really "open" platform for any kind of conversion!I expect (still) quite a lot of limitations because in fact it has been tested only with the SMF-family of scripts as "importers" and it may have some tricky part to iron out (see comments to issue 38), but still a good base for further development.
Re: OpenImporter Reply #97 – December 29, 2014, 05:37:25 pm https://github.com/emanuele45/openimporter/tree/abstractionMostly done!! I found that in this new setup three converters (WBB 3.0, WP and phpBB) are broken due to the strategy of using the destination database to query for certain info [1].I hope there is a solution...At the moment I need some sleep, so I can't figure out, but I'm pretty sure that using some additional query should be enough to get rid of the cross-queries.1 More info here.↵
Re: OpenImporter Reply #98 – December 29, 2014, 09:02:53 pm Oh my .. you have been busy !!Its looking really nice, I'm just going through the SMF importer to understand how you have done the separation, good stuff, lots to understand yet Not sure what to say about the issue you found, needs to be a solution but I don't yet understand everything you have done to even make a suggestion, not even one of my dumb ones
Re: OpenImporter Reply #99 – December 30, 2014, 04:31:58 am That gives me the opportunity to explain a bit. The general idea is that: a "source" importer/converter[1] takes the data from the db you want to convert from (with a query or some code), creates a "standard" array for a certain type of data and passes it back to OpenImporter OI sends the array to a "destination" importer/converter that "does something" if needed and passes it back again to OI, OI inserts the data in the destination database.How this is achieved technically?Step 1 consists of 2 files: * an XML file that contains all the steps a certain "exporter" is able to understand, each step has a specific name[2], and at least a couple of childs: a title (string text that is displayed during the conversion), a detect (it's a table name, OI checks if the table exists or not, if not it sips the step) and a code or a query (one or the other, they cannot coexist). I'm considering moving away from the xml to a yaml or json file. * a php file with a class[3]. The class contains some generic methods to describe the class itself and can contain several other methods, sort of "hook" into the conversion process. These methods' names are in the format prefixStepname, where "prefix" is the "hooking point", for example can be preparse or code, while "Stepname" is the name of the step (as defined in the XML file), these methods are called during the process and allow to change the results of the queries and/or of the code blocks.Step 2 consists of 1 file:* a php file containing a class[4]. Similarly to the php file in step 1, it may contain methods with names that follow the prefixStepname schema and that will be called at each step. OI passes to these methods the result of step 1. The return may be an array of rows, or nothing.Step 3 is really an implementation detail, the only thing to know is that it may expect two types of arguments coming from step 2:an "empty" (false, array(), null, 0, whatever as long as it evaluates "empty"),an array of rows, so array(array('id' => 'val'), array('id' => 'val2')) where every row has always the same keys, that will be used as column names in the query.I think this summarize what happens during the conversion... I think.The main point is that all the system should be based on the assumption that the "exporter" doesn't know anything about the "importer" and the "importer" doesn't know anything about the "exporter"... I hope. 1 I'd be tempted to talk about exporter for the "source to intermediate" and importer for the "intermediate to destination", seems easier, but terminology consistency is not yet the main problem.↵2 The known names are defined in the file Importer_skeleton.xml. This file is not yet used, but in the future it may be used to verify the data types and fill up the gaps, for example entries in each array that doesn't have a value.↵3 The name of the class is specified in the generic section of the XML file and extends an abstract with some methods common to all the exporters↵4 Actually, because I'm lazy it contains three of four classes all necessary for the "importer" during the process, but let's not be too picky. ↵
Re: OpenImporter Reply #100 – January 01, 2015, 05:42:51 pm I'm playing with tests now:https://github.com/emanuele45/openimporter/tree/testing/importer/Importers/sources/Teststhough at the moment travis doesn't want to collaborate much.Well, I'll call it a day and go sleep for the moment.
Re: OpenImporter Reply #101 – January 02, 2015, 12:04:41 pm Interesting stuff: https://github.com/internetbrands/vbimpex
Re: OpenImporter Reply #102 – January 02, 2015, 03:22:16 pm YAY!https://travis-ci.org/emanuele45/openimporter/jobs/45716134
Re: OpenImporter Reply #103 – January 02, 2015, 03:35:16 pm Quote from: emanuele – January 02, 2015, 03:22:16 pmYAY!https://travis-ci.org/emanuele45/openimporter/jobs/45716134Great
Re: OpenImporter Reply #104 – January 02, 2015, 04:01:44 pm Actually, now I have to understand if keep the two set of tests separated is acceptable.Then, if it is, I have to find a way to merge the two coverage reports, while if it not acceptable, I have to find a way to merge them...Maybe @Spuds has some idea about it.
Modern, Free, Powerful