Support/Comments for BkupMODx

177 Posts

Chuck Reply #91, 17 years, 11 months ago

Website Backup and Restore: A Disaster Recovery Tool

v2.1 just released!

This release has significant refactoring which concedes to the increasing complexity of the scripts. It continues to follow the M-V-C design pattern with a new twist. The script also introduces what I call "object containers" for the model, view and control functions. The major functions have been separated into separate classes and each class is in a separate file.

The major feature changes include addition of an upload feature to the Archive screen. This required a new upload/ directory with a controlling .htaccess file to enable really huge uploads. Unfortunately if your web server doesn’t allow overrides with .htaccess then your uploads will be restricted to the settings in your php.ini file.

The config process replaces the 3rd party ConfigTool class with a custom class that reads config files. That’s the last 3rd party class so it’s all custom code now! The config changed the primary config file name from bkup_config.php to config.inc.php and also eliminated the "a_" prefix that the ConfigTool class required for arrays. Beware that the new config file is not plug-and-play reusable and you’ll need to transfer the values to the new config.inc.php if you are upgrading. There is also a new config style file lang.inc.php config used for the description in the Config screen.

For a change of pace I added a few icons in the Backup and Archive screens that contain the action links to the respective functions. They resize on mouseover using CSS in FF but IE6 doesn’t seem to do anything yet. I still need to test the CSS tricks in more browsers.

Here are the official changes from the README.txt file:

* Added upload function

Feature available in Archive
Added upload/ directory
Requires upload/.htaccess for large files

* Improved Config process

Replaced ConfigTool library with custom class
Renamed config file and some variables
Added language file for Config function

* Added action link icons in Backup and Archive
* Refactored code

Added M-V-C object containers
Standardized one class per file

Enjoy!

4,205 Posts

dev_cw Reply #92, 17 years, 11 months ago

Looks like a significant new release. Thanks grin

[font=Verdana]Shane Sponagle | [wiki] Snippet Call Anatomy | MODx Developer Blog | [nettuts] Working With a Content Management Framework: MODx

Something is happening here, but you don't know what it is.
Do you, Mr. Jones? - [bob dylan]

☆ A M B ☆
24,524 Posts

sottwell Reply #93, 17 years, 11 months ago

I’ve just put it on two sites I’m responsible for, and it was fast and painless. Hopefully I’ll never see how equally fast and painless I’m sure it is at restoring!

Studying MODX in the desert - http://sottwell.com
Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
Join the Slack Community - http://modx.org

177 Posts

Chuck Reply #94, 17 years, 11 months ago

Website Backup and Restore: A Disaster Recovery Tool

v2.2 just released!

I didn’t expect this was going to be released so soon. All I started to do was move from XHTML 1.0 Transitional to Strict and I found myself deeper into the code than I expected. Ouch! The biggest validation change was the conversion from iframes to objects. HTML objects don’t respond/refresh the same way as iframes so the hole got deeper. UI changes to the rescue! I found the upload form needed conversion to frameset tags. That was so much fun that I thought why not do the config edit function I had been thinking about. Change around the config object code and mix in a few more UI changes and bang, Version 2.2 is born.

Here are the official changes from the README.txt file:

* Reduced menu to Tools and Exit
* Added Tools Home object
* Added edit functions to Config Tool

Removed config.inc.example.php
Help included in edit mode

* Separated Upload Tool object from Archive Tool object
* Refactored code

Replaced iframe tags with object tags and iframs tags with object tags
Reduced separate screens into tool objects
Validates as W3C XHTML 1.0 Strict

Enjoy!

21 Posts

BlueIndigo Reply #95, 17 years, 11 months ago

First, this is a great tool for modx. Really !

But I was having some problem when trying download the entire .zip backup file (approx. 70 MB). Download ended suddenly around 50% (36 MB).

I’ve noticed that the download link was a php script. After a little search in the code, I’ve seen that you use header() function to make the download.
I’ve tried to download directly the backup file without the php script : it works perfectly.

It was the code with header and so that was my problem. Well I think this is a waste of time using php script sending headers : I just want a simple link, as usual, to download my file.

So I’ve modified the function archiveFrameZip(), in file view_archive.class.php, line 22-23 :

<td class = "cell"> <a class = "link" href = "' . $config['script']
          . '?nav=afd&file=' . $file['file'] . '">

By this , using some of your var to make a direct download link :

<td class = "cell"> <a class = "link" href = "'. $config['site_url'] . $config['archive_dir']. $file['basename'] . '">

It works great now ! grin

177 Posts

Chuck Reply #96, 17 years, 11 months ago

Quote from: BlueIndigo at Oct 30, 2008, 02:50 AM

...
But I was having some problem when trying download the entire .zip backup file (approx. 70 MB). Download ended suddenly around 50% (36 MB).
...

I’m glad you were able to find a fix for your situation! The Archive Tool is designed to handle all files types. For example you could upload a pdf, txt or html file and download it later. Your fix will break the ability to download those type of files and may cause the browser to display known file types instead.

Forcing downloads using the HTTP header method is not uncommon. I’m curious what conditions caused the method to fail for you. Do you have specific errors from your web server logs you can share?

I’ve read that Safari (Mac) sometimes doesn’t respond well to this method but I have no way to test that. I’ve also read that others add additional header commands to control browser caching. Your report is the first where the method in BkupMODx didn’t work. I would like to get to the bottom of the issue if we can.

21 Posts

BlueIndigo Reply #97, 17 years, 11 months ago

Quote from: Chuck at Oct 30, 2008, 12:18 PM

...
The Archive Tool is designed to handle all files types. For example you could upload a pdf, txt or html file and download it later. Your fix will break the ability to download those type of files and may cause the browser to display known file types instead.
...

I understand that. But the modified function is for ZIP archive (function archiveFrameZip()).
And even if it’s a PDF, I just right click and select Download target file.

But sorry that’s not the point.

Do you have specific errors from your web server logs you can share?

After a little search, here is what I’ve found in the apache web log (there is nothing in the apache error log) :
(note that I’ve replaced some of the site name by "My-Site", "My Host", ect...)

Line 1564 : MyHost.net www.my-site.fr - [29/Oct/2008:14:06:25 +0100] "GET /backup/index.php?nav=afd&file=/homez.49/login/www/MySite/v2/backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 39452672 "http://www.my-site.fr/backup/index.php?nav=af" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)"
Line 1829 : MyHost.net www.my-site.fr - [29/Oct/2008:14:25:00 +0100] "GET /backup/index.php?nav=afd&file=/homez.49/login/www/MySite/v2/backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 37789696 "http://www.my-site.fr/backup/index_.php?nav=af" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)"
Line 3004 : MyHost.net www.my-site.fr - [29/Oct/2008:17:45:30 +0100] "GET /backup/index.php?nav=afd&file=/homez.49/login/www/MySite/v2/backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 38420480 "http://www.my-site.fr/backup/index_.php?nav=af" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)"
Line 3108 : MyHost.net www.my-site.fr - [29/Oct/2008:17:51:48 +0100] "GET /backup/index.php?nav=afd&file=/homez.49/login/www/MySite/v2/backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 42336256 "http://www.my-site.fr/backup/index_.php?nav=af" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
Line 3112 : MyHost.net www.my-site.fr - [29/Oct/2008:17:59:27 +0100] "GET /backup/index.php?nav=afd&file=/homez.49/login/www/MySite/v2/backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 15106048 "http://www.my-site.fr/backup/index_.php?nav=af" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
Line 3113 : MyHost.net www.my-site.fr - [29/Oct/2008:17:59:47 +0100] "GET /backup/index.php?nav=afd&file=/homez.49/login/www/MySite/v2/backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 8282112 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
Line 3120 : MyHost.net www.my-site.fr - [29/Oct/2008:17:59:56 +0100] "GET /backup/archive/adeis_bkup_20081029_140252.zip HTTP/1.1" 200 75366928 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"

Well, we easely see that I’ve tried multiple times to download the file with various web browser (Firefox 3 and IE7). With the GET method by php header, the server answer around 39452672 bytes (~40mb), this is a wrong file size ! That’s why the download end dramatically around 50%.

But when I download the file directly (see the last line), the lenght is 75366928 bytes (~76mb), wich is correct ! (and the download works).

So maybe the php headers send a wrong file size ... I’m not sure about that, because I’ve tried to echo() the $file[’size_bytes’] variable in the function archiveFrameDnld() (with the php header) and the size seems to be correct !

I hope it could help you ...

(oh, and sorry if my english is ... a little weird).

177 Posts

Chuck Reply #98, 17 years, 10 months ago

Quote from: BlueIndigo at Oct 31, 2008, 03:16 AM

...
I understand that. But the modified function is for ZIP archive (function archiveFrameZip()).
And even if it’s a PDF, I just right click and select Download target file.

But sorry that’s not the point.
...

You’re right and thanks for the log output. After doing some more research I suspect the problem is compounded by the readfile() command used just after the header() commands. readfile() is affected by the memory limit, time limit and probably a few more settings on the server.

I tested your changes and they work great. It’s a much simpler method so I’ll use this instead of the headers() and readfile() commands starting in the next release.

Thanks again!

21 Posts

BlueIndigo Reply #99, 17 years, 10 months ago

Quote from: Chuck at Oct 31, 2008, 09:40 AM

...
After doing some more research I suspect the problem is compounded by the readfile() command used just after the header() commands. readfile() is affected by the memory limit, time limit and probably a few more settings on the server.
...

I’ve show a phpinfo() and see that the option memory_limit is at 32MB. Could it be this ?
Oh and I’m on PHP Version 5.2.6.

Well I’ve another problem : the PHP option max_execution_time is set to 30 seconds. I can’t modify this setting (it’s my host who choice this value).
So my problem : when the Backup take less than 30 secs : everything is perfect, I’m lucky.
When it’s higher ... Well it stop dramatically and I have a corrupted ZIP archive (and a .sql file) undecided

I was dreaming of an option in BckModx like starting a backup, and if the backup is not complete, can continue where it was (and not at the beginning).
But I know it’s a dream rolleyes

I can’t imagine the complexity of a script like that...

My solution : doing big backup the morning, very very early ! It seems to work.

I tested your changes and they work great. It’s a much simpler method so I’ll use this instead of the headers() and readfile() commands starting in the next release.

I’m really glad that I have been able to help you !

177 Posts

Chuck Reply #100, 17 years, 10 months ago

Quote from: BlueIndigo at Oct 31, 2008, 11:02 AM

I’ve show a phpinfo() and see that the option memory_limit is at 32MB. Could it be this ?
Oh and I’m on PHP Version 5.2.6.

Well I’ve another problem : the PHP option max_execution_time is set to 30 seconds. I can’t modify this setting (it’s my host who choice this value).

The scripts use set_time_limit() and ini_set(’memory_limit’, ...) commands to override the host settings. They can be set via the Config Tool.

The memory is obviously not an issue for you. You can verify this since you’re using PHP version > 5.2.0. The script outputs "Peak memory usage" just before the "Backup process complete" message. You’ll see that your peak probably greater than 70Mb based on your zip file size.

The time limit might be a problem. According to php.net "This function has no effect when PHP is running in safe mode. There is no workaround other than turning off safe mode or changing the time limit in the php.ini." An alternate way to control server resource usage is to change your BkupMODx config directories and/or MySQL dbases. Backup a limited portion of your site then change the values to backup to remaining portions.