Skip to main content
Topic: OpenImporter (Read 59394 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: OpenImporter

Reply #45

I will wait for a new package of the importer with the latest fixes, because I use custom profile fields.  :D

Re: OpenImporter

Reply #46

Quote from: emanuele –
What configuration do you use for the avatars?
As attachment? Or did you specify a custom directory?
I have some custom avatars uploaded in my folder /avatars (not imported).
The uploaded avatars are saved as attachments, I think in /attachments folder because in /avatars there are only my custom avatars.

Quote from: emanuele – Yes, custom fields are easy:
added to /Importers/elkarte1.0/smf2-0_importer.xml following the XML structure. ;)

Less easy may be the instant message fields (ICQ, AIM, MSN, ???), but... honestly... is there anyone still using any of them? O:-)
that worked like a charm. Custom fields imported, thanks again emanuele!
sorry for my bad english

Re: OpenImporter

Reply #47

Quote from: radu81 – I have some custom avatars uploaded in my folder /avatars (not imported).
hmm... I didn't thought about that.
Avatars are still a bit messy because of the multiple options (from gallery, uploaded, stored as attachments, stored in a directory, etc.), I have to set up a proper testing environment... :-\

Quote from: radu81 – The uploaded avatars are saved as attachments, I think in /attachments folder because in /avatars there are only my custom avatars.
And also these ones are not imported?
Just to know exactly were I should look. ;)

Quote from: radu81 – that worked like a charm. Custom fields imported, thanks again emanuele!
Great! :D

I tried to change a bit the first page. O:-)
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #48

Okay, and with one of the last commits I removed entirely any system-specific code I was aware of from the "core". Now it should be ready to go beyond ElkArte, SMF and Wedge as "destination" platforms.

If I have time, tomorrow I'll try to write a converter from Elk to WordPress to see if it is really ready. :)
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #49

@emanuele not that I want you to convert my forum for me but if could be useful to you I can send you a copy of my files and db to make some tests
sorry for my bad english

Re: OpenImporter

Reply #50

That's an option, thanks! :)

But, maybe, it's possible to set up a matrix of all the combinations so that we can get all the cases imported without troubles.
Let's see.

We are now talking about SMF 2.0.
In that software avatars can be:
from a gallery
from an url (either simply linked or "uploaded", see next point)
* uploaded (either as attachment or in a custom directory).
I hope I didn't miss anything.
In that set of possibilities, we know that:
  • from gallery are just stored in the members table as relative path
  • from an url and uploaded with a custom dir is stored in members as a full url
  • uploaded as attachment is stored... as an attachment. :P

As far as I remember, the same applies to ElkArte as well. That makes things easier.
Case 1) data in the members table is moved with members, so nothing to do. <= done! Then there is just to move the entire gallery to the new location (that actually we can just guess I think, but it's better than nothing), so a "move this directory to the new place" should be enough. <= todo.
Case 2) data in the members table is moved with members, so nothing to do. <= done! What's left is move the avatars from the custom dir to the new one. <= I think it's already done somehow, but I'm not entirely sure. To check.
Case 3) requires 2 things: a) move the attachment to the new attachment/s dir, b) re-create the attachment in the database. I wonder if we can just take care of the substep b in the importing of the attachments, it is in fact the same thing. Then, if b is taken care of, substep a would just be a matter of copying files where they should be.
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #51

@Spuds I have a question for you, master of phpunit. 8)

I would really like to add tests for the converters, and I'm trying to figure out how to do it.
My idea is:
have a "forum" (database + files)
install the forum
run the converter
compare with the expected result (database + files)
Let's assume I'm going to use Travis-ci.
Do you think it is possible to do the loop install/convert/compare for each test (so that only one forum is installed at a time) in script, or it is necessary to install all of them in the "before script" and then run the tests and compares in the script section?
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #52

I think you could do something like that in the setup/teardown functions in phpunit.

Maybe create an abstract class that extends to implement the before/after each testClass file you have defined.  Just include your abstract class in bootstrp or in the .yaml file.  There are also methods that run before and after test method of the class but I don't think those will help with this.

Code: [Select]
abstract class OpenImp extends PHPUnit_Framework_TestCase
{
// called before the first test of a testClass
public static function setUpBeforeClass()
{
}

// called after the final test of a testClass
public static function tearDownAfterClass()
{
}

// other functions you may want to use in your testcases
    public function other_stuff_i_need(){};
}

Re: OpenImporter

Reply #53

All that stuff is now in "master". :D

@radu81 thank you very much for the data, I was able to squash some other bugs and make the code simpler! :D
Now it imports should import ( :P ) all the avatar irrespective of where they are, unless of some very odd problems.

@Spuds thanks for the hint! I'll give it a try and see what I can come up with! (A mess. LOL)
Last Edit: November 03, 2014, 04:44:03 pm by emanuele
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #54

And here it is an experiment.

Instructions:
unzip
upload
* point to import.php

That's an experiment to understand if it is worth exploring the route of the phar archive.
A phar archive is a "special" php file that includes the entire application into a single file (actually it needs two: the phar and a php to run the phar, meh). I thought this may be easier to handle for the "final users" in that specific case, since it would be a "one shot" and have lots of files and directories to deal with could be a little pain, while two files are much easier to track in the process. Also in theory the released package doesn't require any change to the files, etc.

Anyway, I tried this solution, though reading around there are some configurations that do not like phar (e.g. suhosin) and if it doesn't work on the majority of systems, it may not be worth to maintain a double system of distribution (that would create confusion I think).

Let's see.

ETA: package removed.
Last Edit: December 08, 2014, 07:39:01 am by emanuele
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #55

phar-out baby, I'm going to give it a try !

I should have mentioned that after you create your abstract class then your tests extend that instead of extends PHPUnit_Framework_TestCase so myTest extends OpenImp  yada yada ... Just a way to call those before and after things at the top level of each test class.

Re: OpenImporter

Reply #56

Hi @emanuele ant thanks for the importer, but I have a problem. Have a look

Code: [Select]
Importing...

Importing settings... ✔
Importing members... ✔
Importing attachments...

( ! ) Fatal error: Maximum function nesting level of '100' reached, aborting! in phar://C:/wamp/www/skodaelk/openimporter.phar/OpenImporter/Utils.php on line 88
Call Stack
# Time Memory Function Location
1 0.0003 690488 {main}( ) ..\import.php:0
2 0.0058 951360 require_once( 'C:\wamp\www\skodaelk\openimporter.phar' ) ..\import.php:4
3 0.0059 945168 include( 'phar://C:/wamp/www/skodaelk/openimporter.phar/index.php' ) ..\openimporter.phar:9
4 0.0063 987952 require_once( 'phar://C:/wamp/www/skodaelk/openimporter.phar/import.php' ) ..(null):1
5 1.3333 2557488 ImportManager->process( ) ..(null):77
6 1.3334 2558624 call_user_func ( ) ..(null):236
7 1.3334 2558624 ImportManager->doStep1( ) ..(null):236
8 1.4119 2725448 Importer->doStep1( ) ..(null):471
9 2.7881 2739568 XmlProcessor->processSteps( ) ..(null):516
10 2.7935 2741112 XmlProcessor->doSql( ) ..(null):99
11 2.7990 2810544 XmlProcessor->prepareRow( ) ..(null):157
12 2.7991 2809736 eval( ' moveAttachment($row, $this->db, $this->config->from_prefix, $this->step1_importer->getAttachDir($row)); ' ) ..(null):212
13 2.8001 2809736 moveAttachment( ) ..(null):2
14 2.8003 2813328 copy_file( ) ..(null):109
15 2.8003 2813536 create_folders_recursive( ) ..(null):58
16 2.8003 2813696 create_folders_recursive( ) ..(null):91
17 2.8003 2813848 create_folders_recursive( ) ..(null):91
18 2.8004 2813992 create_folders_recursive( ) ..(null):91
19 2.8004 2814096 create_folders_recursive( ) ..(null):91
20 2.8004 2814176 create_folders_recursive( ) ..(null):91
21 2.8004 2814256 create_folders_recursive( ) ..(null):91
22 2.8004 2814336 create_folders_recursive( ) ..(null):91
23 2.8004 2814416 create_folders_recursive( ) ..(null):91
24 2.8004 2814496 create_folders_recursive( ) ..(null):91
25 2.8004 2814576 create_folders_recursive( ) ..(null):91
26 2.8004 2814656 create_folders_recursive( ) ..(null):91
27 2.8004 2814736 create_folders_recursive( ) ..(null):91
28 2.8005 2814816 create_folders_recursive( ) ..(null):91
29 2.8005 2814896 create_folders_recursive( ) ..(null):91
30 2.8005 2814976 create_folders_recursive( ) ..(null):91
31 2.8005 2815056 create_folders_recursive( ) ..(null):91
32 2.8005 2815136 create_folders_recursive( ) ..(null):91
33 2.8005 2815216 create_folders_recursive( ) ..(null):91
34 2.8005 2815296 create_folders_recursive( ) ..(null):91
35 2.8005 2815376 create_folders_recursive( ) ..(null):91
36 2.8005 2815456 create_folders_recursive( ) ..(null):91
37 2.8005 2815536 create_folders_recursive( ) ..(null):91
38 2.8005 2815616 create_folders_recursive( ) ..(null):91
39 2.8006 2815704 create_folders_recursive( ) ..(null):91
40 2.8006 2815792 create_folders_recursive( ) ..(null):91
41 2.8006 2815880 create_folders_recursive( ) ..(null):91
42 2.8006 2815968 create_folders_recursive( ) ..(null):91
43 2.8006 2816056 create_folders_recursive( ) ..(null):91
44 2.8006 2816152 create_folders_recursive( ) ..(null):91
45 2.8006 2816248 create_folders_recursive( ) ..(null):91
46 2.8006 2816344 create_folders_recursive( ) ..(null):91
47 2.8006 2816440 create_folders_recursive( ) ..(null):91
48 2.8006 2816536 create_folders_recursive( ) ..(null):91
49 2.8007 2816632 create_folders_recursive( ) ..(null):91
50 2.8007 2816728 create_folders_recursive( ) ..(null):91
51 2.8007 2816824 create_folders_recursive( ) ..(null):91
52 2.8007 2816920 create_folders_recursive( ) ..(null):91
53 2.8007 2817016 create_folders_recursive( ) ..(null):91
54 2.8007 2817112 create_folders_recursive( ) ..(null):91
55 2.8007 2817208 create_folders_recursive( ) ..(null):91
56 2.8007 2817304 create_folders_recursive( ) ..(null):91
57 2.8007 2817400 create_folders_recursive( ) ..(null):91
58 2.8007 2817496 create_folders_recursive( ) ..(null):91
59 2.8007 2817592 create_folders_recursive( ) ..(null):91
60 2.8008 2817688 create_folders_recursive( ) ..(null):91
61 2.8008 2817784 create_folders_recursive( ) ..(null):91
62 2.8008 2817880 create_folders_recursive( ) ..(null):91
63 2.8010 2817976 create_folders_recursive( ) ..(null):91
64 2.8010 2818072 create_folders_recursive( ) ..(null):91
65 2.8010 2818168 create_folders_recursive( ) ..(null):91
66 2.8011 2818264 create_folders_recursive( ) ..(null):91
67 2.8011 2818360 create_folders_recursive( ) ..(null):91
68 2.8011 2818456 create_folders_recursive( ) ..(null):91
69 2.8011 2818552 create_folders_recursive( ) ..(null):91
70 2.8011 2818648 create_folders_recursive( ) ..(null):91
71 2.8011 2818744 create_folders_recursive( ) ..(null):91
72 2.8011 2818840 create_folders_recursive( ) ..(null):91
73 2.8011 2818936 create_folders_recursive( ) ..(null):91
74 2.8011 2819032 create_folders_recursive( ) ..(null):91
75 2.8011 2819128 create_folders_recursive( ) ..(null):91
76 2.8011 2819224 create_folders_recursive( ) ..(null):91
77 2.8012 2819320 create_folders_recursive( ) ..(null):91
78 2.8012 2819416 create_folders_recursive( ) ..(null):91
79 2.8012 2819512 create_folders_recursive( ) ..(null):91
80 2.8012 2819608 create_folders_recursive( ) ..(null):91
81 2.8012 2819704 create_folders_recursive( ) ..(null):91
82 2.8012 2819800 create_folders_recursive( ) ..(null):91
83 2.8012 2819896 create_folders_recursive( ) ..(null):91
84 2.8012 2819992 create_folders_recursive( ) ..(null):91
85 2.8012 2820088 create_folders_recursive( ) ..(null):91
86 2.8013 2820184 create_folders_recursive( ) ..(null):91
87 2.8013 2820280 create_folders_recursive( ) ..(null):91
88 2.8013 2820376 create_folders_recursive( ) ..(null):91
89 2.8013 2820472 create_folders_recursive( ) ..(null):91
90 2.8013 2820568 create_folders_recursive( ) ..(null):91
91 2.8013 2820664 create_folders_recursive( ) ..(null):91
92 2.8013 2820760 create_folders_recursive( ) ..(null):91
93 2.8013 2820856 create_folders_recursive( ) ..(null):91
94 2.8013 2820952 create_folders_recursive( ) ..(null):91
95 2.8013 2821048 create_folders_recursive( ) ..(null):91
96 2.8014 2821144 create_folders_recursive( ) ..(null):91
97 2.8014 2821240 create_folders_recursive( ) ..(null):91
98 2.8014 2821336 create_folders_recursive( ) ..(null):91
99 2.8014 2821432 create_folders_recursive( ) ..(null):91
sorry for my bad english

Re: OpenImporter

Reply #57

Just to be sure (I saw it in the backup you sent me, so it should be fine), did you change the paths to the attachments to the local ones? (C:\wamp\www\...)

Could you also try putting the following code in a php file, execute it and report output?
Thank you very much!

Code: [Select]
<?php
error_reporting(E_ALL);

create_folders_recursive('C:\\wamp\\www\\skodaelk\\attachments');

function create_folders_recursive($path)
{
$parent = dirname($path);

if (!file_exists($parent))
{
echo 'Nesting in: ' . $parent . '<br />';
create_folders_recursive($parent);
}

if (!file_exists($path))
{
echo 'Creating: ' . $path . '<br />';
mkdir($path, 0755);
}
}
Bugs creator.
Features destroyer.
Template killer.

Re: OpenImporter

Reply #58

I did not change anything, the files are the same that I sent you.

where should I put the php file? in the root of my smf forum? Itried but I see a blank page
sorry for my bad english

Re: OpenImporter

Reply #59

Never mind, phar archives and directory creation in windows are not friends.
I'll try to investigate the reason, but not now.

I attached here a "normal" package, try using this one.
And sorry. ;)

ETA: package removed.
Last Edit: December 08, 2014, 07:50:13 am by emanuele
Bugs creator.
Features destroyer.
Template killer.