Skip to main content
Topic: Backups (Read 11844 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: Backups

Reply #15


lol Yep, it is. Then let's erase the backup functionality we have at the moment. Better no backup at all than an uncomplete one. Oh, and let's erase the file edit functionality in the backend of ElkArte too. There are better tools for this kind of work.  ;)

Sorry for the sarcasm.  :-[  But yep, I think kind of this way. I don't think it is good to offer functionalities that don't work in a full, complete way. It can lead to serious problems if users trust in the functionalities and they don't work perfect in the end. An incomplete backup can crash a board.

If we have a backup function inside ElkArte, why can't it work like the database maintenance and offer scheduled backups? I understand the technical difficulties (at least some of them), and if there isn't a way after all or you think it is much too complicated, okay, I can respect and live with that. I can't change this then.

All I wanted is to share my thoughts about things that maybe make ElkArte better. Better in handling and/or better in UI. I don't demand things. I know you appreciate that.  ;)  So no bad feelings, it's just an exchange of ideas and thoughts.

Re: Backups

Reply #16

I agree with @Jorin on this. Besides, we can still attempt to create an addon for automatic database backup though it may not be a feature.

Re: Backups

Reply #17

i agree, it should be an addon too.
192.MY.ID: Forum ISP Indonesia.

Re: Backups

Reply #18

I said that we should get rid of it before. I am all for that. Though, it was shot down because it would take more to get rid of it than it is just to keep it. I guess in a future version it will disappear.

I totally agree with not implementing half assed features and users' expectations.

Re: Backups

Reply #19

Just to clear things up a bit and not mix stuff.
1) The database back in ElkArte is perfectly functioning.
2) To reduce at a minimum the risk of broken backups, Elk tries to "guess" (based on resources available) if the backup will complete before the backup is started. And if the server config looks problematic, is not even possible to start the backup.
Under optimal conditions I was able to perfectly backup a 2 GB database.
So, as far as I can tell it's far from being an half assed feature.

That said, of course any php script may suffer from many kind of problems, so it may be possible the database backup is not entirely reliable.
Unfortunately it's difficult to account for all the possible combinations.

I personally wouldn't use an email to store the backups for many reasons (from the limits that may prevent the sending or the receiving of the email, making the backup as useless as a broken one, to the fact that emails are not really the safes place to store things, see the many cases of account breaches and passwords leaks).
Bugs creator.
Features destroyer.
Template killer.

Re: Backups

Reply #20

Quote from: emanuele – Just to clear things up a bit and not mix stuff.
1) The database back in ElkArte is perfectly functioning.
2) To reduce at a minimum the risk of broken backups, Elk tries to "guess" (based on resources available) if the backup will complete before the backup is started. And if the server config looks problematic, is not even possible to start the backup.
Under optimal conditions I was able to perfectly backup a 2 GB database.
So, as far as I can tell it's far from being an half assed feature.

That said, of course any php script may suffer from many kind of problems, so it may be possible the database backup is not entirely reliable.
Unfortunately it's difficult to account for all the possible combinations.

I personally wouldn't use an email to store the backups for many reasons (from the limits that may prevent the sending or the receiving of the email, making the backup as useless as a broken one, to the fact that emails are not really the safes place to store things, see the many cases of account breaches and passwords leaks).

My Gmail has 2FA security, starting with a 15 character password with varying case, numbers and symbols. I'd like to see someone hack into my email.

Re: Backups

Reply #21

Part 2 (sorry, I had to quickly stop because I noticed I was going to be late for work lol).

First: I'm not saying that because I think the idea is bad, but just to put on the table all the difficulties to overcome before having a fully-100%-working-without-any-doubt feature/addon/whatever.

After what I wrote, there is the little bit of "problems" due to php.
The usual reason why SMF backup was used to fail, is because of timeouts and/or memory limits.
Even though php tries to give you ways to workaround these limits, many hosts do not allow you to override default settings, so you are bound to a system that is limited and doesn't allow proper ways of backing up data.

If you have to backup few megabyes of data, it may take "some" time (difficult to estimate) to build up the stream, pack it and prepare for further handling. So, if you do that from a fake-cron like the Elk (or SMF) scheduled tasks, you are going to have bad surprises because sooner or later the backup my fail due to timeouts.
In order to avoid timeout issues you should be able to run a php script as a proper cronjob, of course not all the hosts allow to manage cronjobs. So there is a limit.

Assuming timeouts are not a problem, if your php memory is limited, you'll be able to backup only so much of your database before php hits the wall of the "out of memory" error.
And work around the memory limit is a bit trickier.
Not impossible, just tricky.

There may be a part 3, but likely only Sunday. :P
Bugs creator.
Features destroyer.
Template killer.

Re: Backups

Reply #22

2FA is a fake sense of security. Someone with a duplicate SIM card can easily get through the defence.

Re: Backups

Reply #23

Quote from: meetdilip – 2FA is a fake sense of security. Someone with a duplicate SIM card can easily get through the defence.

No they can't. Google Authenticator app, not text messages.

Re: Backups

Reply #24

Tricky indeed. I find one of the script that deals with large database here but you may need to register to download this. Since it's GPL and redistributable, I attached a copy of it here.

And for the gmail and gDrive, I think, sending the file as attachement should be safe, but to be safer, you can ftp it.

Edited, again: It seems that google drive can be access via web even to upload and download files. May be useful as backup tools and storage.

Re: Backups

Reply #25

Quote from: ahrasis – It seems that google drive can be access via web even to upload and download files. May be useful as backup tools and storage.
I found this old post and was thinking on using it as backup tool. I managed to get the quickstart working but I am not sure to proceed further from there.

I guess I have to create a folder most probably outside the website folder and link it to my gDrive, but I am not sure about the link part. I will keep testing this for a while and see how it goes.

Re: Backups

Reply #26

What I do is use something like this and mount gdrive on a directory, then a cronjob to do the backups.
I always wanted to document it... but as usual time is the limiting factor (along with the mess I did before getting it right LOL).
Bugs creator.
Features destroyer.
Template killer.

Re: Backups

Reply #27

There is one I just found that can use ssh / terminal base for gDrive which is called grive2. I think this one is good and I will attempt to use this one for database, ssl and other important backups. One will need to run grive manually to upload / download / sync or alternatively create a script and set up cron job to run it automatically. Better instructions seem to be in the same website mentioned by @emanuele.

Re: Backups

Reply #28

Actually... scrap what I said, I use http://www.webupd8.org/2014/09/gdrive-simple-google-drive-cli-client.html
Basically I have an sh script that looks something like:
Code: [Select]
now=$(date +"%Y_%m_%d")
file_name="elkarte_$now"
full_path="/safe/path/$file_name"

mysqldump -u root --password=mypwd mydb > "$full_path.mydump.sql"
zip -q -r "$full_path.dump.zip" "$full_path.mydump.sql"
/usr/local/bin/drive upload -f "$full_path.dump.zip" -t "$file_name.dump.zip" -p secret_token

rm -f "$full_path.mydump.sql"
rm -f "$full_path.dump.zip"

zip -q -P apwd -r "$full_path.forum.files.zip" /path/to/root/htdocs/forum/ -x "*/packages/*"
/usr/local/bin/drive upload -f "$full_path.forum.files.zip" -t "$file_name.forum.files.zip" -p secret_token

rm -f "$full_path.forum.files.zip"


DATE=$(date +%y-%m-%d)
/usr/sbin/sendmail myemail@gmail.com <<EOF
subject: Daily backup - $DATE

Saved files:
* $file_name.dump.sql
* $file_name.forum.files.zip
EOF

There are some variables playing around to set
Bugs creator.
Features destroyer.
Template killer.

Re: Backups

Reply #29

I already setup grive2 and only need a script to backup my database to its folder.

I was considering to use jorin's php script yesterday but yours seems better and simpler @emanuele, since it covers both forum database and files.

However I'd prefer smallest compression, so I think I'll change the zip command to tar -cjvf to turn it to a tar.bz2 file.

I am not sure whether I need to backup the whole forum though I think it is good and easy. At least I think packages/backups can be opted out by adding --exclude=/path/to/root/htdocs/forum/packages/backups.

Let's see how this goes then.