forum.coppermine-gallery.net

No Support => Feature requests => Topic started by: nova on October 21, 2003, 03:32:47 pm

Title: [help wanted]:MySQL backup system
Post by: nova on October 21, 2003, 03:32:47 pm
I would join Laugesen in his request for a backup/restore function. It would be nice, not to have to use the phpMyAdmin part. In my oppinion the function should create a single file you could download (or ftp) to your HD and/or your CD-writer.

If your database goes down the drain, there should be a restore function. I know that there can be some glitches if you have a working database with dublet entries, but the starting point should be a "clean" mySQL database, so you don't have to think about that.

It sounds simple, but I'm not a php expert, so I turn to this forum. :-)

Yours sincerly

Rasmus
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on October 21, 2003, 06:22:36 pm
let's make some kind of "quest" out of this: phpMyAdmin is open source released under GNU GPL as well as Coppermine; I have seen other free software using the phpMyAdmin function inside their software (for example phpBB or some typo3 backup modules).
It'd be a nice opportunity for some coder who'd like to contribute to coppermine to code this into a standalone piece of code (one file), taking into account to Coppermine structure, build some front-end bits and voila - we have a backup routine.
Who'd like to try and take the "quest" (and get all the "fame"; sorry, no "money" or "girls" come with this :wink: )?

GauGau
Title: [help wanted]:MySQL backup system
Post by: Laugesen on October 21, 2003, 06:31:29 pm
My idea was not to use a whole PHPMySQL system, I know that, but a simple way to save all coppermine related MySQL data in a file, stored local on pc.

And in the Install process, a function like a choice to use the xx.sql files in the reinstall

Hopefully to make it simple, not complicated like phpMYSQL is for "noobs"
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on October 21, 2003, 06:37:02 pm
yeah, I know that - the coder has to use some of phpMyAdmin functions, but his mod has to do two things (or buttons):
- backup the complete Coppermine gallery mySQL data
- restore it from a previous backup

To get an idea, check phpBB documentation: backup (http://phpbb.com/guide/phpBB_Users_Guide.html#section3_2_1)

GauGau
Title: [help wanted]:MySQL backup system
Post by: azrael on October 23, 2003, 01:24:24 pm
Might be a bit outside the meaning of the thread but just adding my
2 cents of worth into the discussion.
Isnt system backup something which should be outside the application?
Although it is a nice thing to integrate, it is sometimes the best thing to
keep an application for what it is and not try to integrate every know
feature known to man into it, since result often will be a bloated code and
a slow application.

Personally i wrote shell scripts for backing up the 30 or so mysql bases
i got running that i put into the cron.daily folder.
Basics is that i run mysqldump on the bases, take the output, tars and
gunzips it and gives it a filenane with todays name in and puts it in a
backupdir, runs daily and seamless in the background each night.
If you do regular backups of your server to tape, files will then be
included for offsite backup too.

If theres any interest to it i can post the scripts here.
Title: [help wanted]:MySQL backup system
Post by: Tarique Sani on October 23, 2003, 02:14:41 pm
I have been thinking about the bloat in CPG code... The first fact is that it is inevitable BUT the question is do we need to encumber everyone with the entire code?

I propose a system of modules / plugins - the API of which should be such that if the module files are placed within a directory of their own within the modules folder the Config / admin menu on top should be able to link to it directly and automagically.

I guess the back-up can be one such utility. The themes already work on this principal - BUT - that is being simplistic.
Title: [help wanted]:MySQL backup system
Post by: Laugesen on October 23, 2003, 02:22:15 pm
As a plug-in it would be great, then the users got a choice to DL it and use it or not. :wink:
Title: [help wanted]:MySQL backup system
Post by: jasendorf on October 23, 2003, 04:03:32 pm
Quote
Isnt system backup something which should be outside the application?


In a utopia where everyone owned their own server and had a shell account, yes... I suppose so.  But, remember, many people are stuck with far less access than that.
Title: [help wanted]:MySQL backup system
Post by: azrael on October 23, 2003, 05:21:57 pm
Ok, guess i am a bit spoiled accesswise having root access to the
production servers.
Although, the module idea that was proposed sounds like a rather good
idea, since everyone stands a bit freer to install what parts of the
application they can use. I can have my stripped down core version of
coppermine, small and optimized for speed due to heavy load, other
people get all the nice features they want.

Customizing would be easier in a set standard, and installation of named
plugins would hopefully go easier.
Hope you will follow this path.

--
Az.
Title: [help wanted]:MySQL backup system
Post by: nova on October 23, 2003, 06:39:53 pm
A plugin you could download and use (just like the templates and/or the languages) sounds like a really good solution. In that way you don't have to make the 'core' of Coppermine bigger or more complicated.

Yours sincerly

Rasmus
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on October 23, 2003, 11:04:06 pm
I agree that we don't have to add each and every feature one could think of into the core - the module idea seems to be tempting. But I've seen other projects trying this and fail...
An application like coppermine is mainly for those who actually can't code (non-geeks), because that's what they can do: put pics on their site to have some content after all.
Especially a backup of coppermine could be done in a separate file (only one more link in the admin menu) with just the look of coppermine - no big deal... Call it a module or whatever.
There are other parts of coppermine I'd like to drop (making them add-ons that can be downloaded and added separately): there are already to many themes  in the package, and way to many languages to maintain.

GauGau
Title: [help wanted]:MySQL backup system
Post by: Tarique Sani on October 24, 2003, 05:58:11 am
The idea of module is also tempting for me because as I see it the lang arrays associated with that module will be in that module directory and not clutter the lang files of the core application also an API will be provided so that the module can default to english if the selected lang is not available

Right now I envisage just two category of modules
1) Those which integrate with the main admin (top menu) like Resize and Backup
2) Those which integrate with the image editing like edit description, crop and rotate image, add multimedia etc (will appear as drop-down below image in admin mode)

We can also have guidelines for designing modules and self imposed rating of difficulty of integration, though ideally a module should never require to edit a core CPG file
Title: [help wanted]:MySQL backup system
Post by: Oxy on November 30, 2003, 07:17:23 am
I just have a question for those who are hosted with few rights or others who know the situation.
Do you have right to remotely execute sqldump ?
I mean, do most host allow that or not ?

Because even without getting a shell access maybe the function is available. I activated a free account on a public host but I could only try later they need 24h to activate my DB access (cron task :( ) so maybe some of you have the answer right now and it is always good to have several feedback.


ps: Is someone working on that ?
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on November 30, 2003, 08:11:13 am
I doubt that there's a workaround if you don't have shell access.

GauGau
Title: [help wanted]:MySQL backup system
Post by: Oxy on November 30, 2003, 03:45:19 pm
??????
If you do not have shell access this is not preventing you from executing the imagemagick converter tool for example so if servers propose access to sqldump why not : exec "/usr/bin/sqldump" .. ?
Title: backup and restore sql database
Post by: jason on December 31, 2003, 05:06:03 pm
backup (mysqldump (http://www.mysql.com/doc/en/mysqldump.html))

Code: [Select]
#!/usr/bin/perl
open(STDERR, ">&STDOUT");
$|=1;
print "Content-type: text/html\n\n";
print "<pre>\n";
$t = `date`;
print "$t\n\n";
system("mysqldump -u USERNAME -pPASSWORD --opt DATABSE_NAME > /path/to/database.sql");
$t = `date`;
print "\n\n$t";
print "</pre>";


restore (mysql (http://www.mysql.com/doc/en/mysql.html))

Code: [Select]
#!/usr/bin/perl
open(STDERR, ">&STDOUT");
$|=1;
print "Content-type: text/html\n\n";
print "<pre>\n";
$t = `date`;
print "$t\n\n";
system("mysql -u USERNAME -pPASSWORD DATABASE_NAME < /path/to/database.sql");
$t = `date`;
print "\n\n$t";
print "</pre>";


place perl scripts inside your cgi-bin directory and call them from your web browser, no need to split or have shell access.  delete perl scripts and database file from your site when you are done.

you could take this further by compressing the file and setup a date archival format that could be utilized from cron jobs.

mysql backup restore split chunks large database cron
Title: cron jobs for backup of database
Post by: daniel.duarte on January 10, 2004, 01:12:44 pm
Quote from: "azrael"
If theres any interest to it i can post the scripts here.


Hello,

I've been using phpMyAdmin to do regular backups of the db to a file in the coppermine directory, and then I simply do regular backups of all the directory.

However I wanted to automate this process, and the idea of a daily cron job to backup is very tempting.

Could you please post the scripts, and some insctrutions? And restoring the backup, can be done with phpmyadmin?

Thanks!
Title: [help wanted]:MySQL backup system
Post by: jason on January 13, 2004, 03:17:42 am
look up.

other than adding in a simple recursive copy command for your cpg files, what else are you looking for?
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on January 13, 2004, 04:37:20 am
those in the know don't have need for a built-in backup....
Coppermine is being used by a lot of beginners who simply don't know how to use such a command. What we're looking for is an integration into the user interface of coppermine (similar to the backup function of phpBB) that has two buttons: "backup coppermine database" and "restore coppermine database" with some bells and whirrs starting when someone is about to restore the database (alerting him that all db content will be replaced with the stuff from the backup).

GauGau
Title: [help wanted]:MySQL backup system
Post by: jason on January 13, 2004, 02:19:02 pm
gaugau, the problem with integrating a backup/restore from a php page is it get's restricted by the php upload limitations.  while this only affects restores, it still hinders a "cute and simple" solution that other php based projects (like phpBB) have attempted (without splitting into chunks, who wants to do that!)

i supposed if the php backup/restore page created a local file on the webserver instead of downloading through the browser and then allowing the user to pick and choose which backup to restore from list of backups made....then it would work fine.

i *think* i can set this up.
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on January 14, 2004, 12:06:46 am
would be great - better than what we have now (nothing except smart-ass geek instructions on how to use phpMyAdmin's backup/dump). If you could come up with a solution that will work for newbies I'd appreciate it a lot (remember that half of it will be a newbie-proof documentation on how to use such a tool).

I'm looking forward to your contribution.

GauGau
Title: [help wanted]:MySQL backup system
Post by: Casper on January 14, 2004, 12:13:01 am
:lol: There's no such thing as 'newbie proof'.
 Just look at half the questions you get here, would not be asked if the newbies had read the docs, and that includes me when I first came here. :oops:  :wink:
Title: [help wanted]:MySQL backup system
Post by: Joachim Müller on January 14, 2004, 12:23:37 am
yeah, I (sadly) have to agree - but a good documentation will keep the other half from asking :wink: (hopefully)...

GauGau
Title: [help wanted]:MySQL backup system
Post by: jason on January 14, 2004, 02:27:31 am
Quote from: "gaugau"
I'm looking forward to your contribution.


ok, keep squinting....i'll post back when im ready for some outside testing.
Title: [help wanted]:MySQL backup system
Post by: mikis on January 16, 2004, 12:43:03 am
Maybe this can be used as a start... I picked it up on a forum some time ago. It is a simple script for restoring MySQL dump files. Restore (dump) file should be in the same dir as the script, or you can change its name and path; you can for example upload dump files with FTP.

Code: [Select]

<?
/* change this to fit your configuration: */
$hostname = "localhost";
$username = "myusername";
$password = "mypassword";
$dbname   = "mydbname";
$filename = "mydbname_db.sql";

/* Code begins here */

$connect = mysql_pconnect($hostname, $username, $password) or die(mysql_error());

echo "Import Started...<br>\n";
$handle = fopen ($filename, "rb");
$contents = fread ($handle, filesize ($filename));
$contents .= "\n\n";
fclose ($handle);
mysql_select_db($dbname);
$queries = explode(";\r\n", $contents); // if dump file is in Unix format, change ";\r\n" to ";\n"
$querycount = count($queries)-1;
for($i=0; $i < $querycount; $i++) {
    $result = mysql_query($queries[$i]);
    if(mysql_errno() != 0)
        echo '<br>' . mysql_errno() . ": " . mysql_error(). "\n";
}
echo "Import finished!<br>\n";
?>


From here, you can easily add file uploading via browser, or something like that. Creating dump file is much harder though. I tried using code form phpMyAdmin, but got lost in the way :)
Title: Re: [help wanted]:MySQL backup system
Post by: Einstein on April 28, 2004, 06:41:16 pm
Check out these (it's a good begining):
http://www.phpbb.com/phpBB/viewtopic.php?t=134983
http://www.phpbb.com/phpBB/viewtopic.php?t=164873
http://www.phpbb.com/phpBB/viewtopic.php?t=17528

Take a look at these also:
http://www.hotscripts.com/PHP/Scripts_and_Programs/Database_Tools/index.html
Title: Re: [help wanted]:MySQL backup system
Post by: loboclerk on November 08, 2004, 08:42:36 am
I used to code my own forum. And have implemented the backup this way:

1) Use mysqldump and zip with the date appended to the file name. Backup includes table structures and data.

2) Option of download or storage of the zip file into a specific backup folder.

3) User can manage (download, delete) all the stored zip files. Deletion is done for older archives to save space.

4) To restore database, just choose the zip file identified by date, or upload the zip file.

I chose zip as it compresses text files very well. Saving the zip file to a backup folder is helpful especially if HTTP/PHP timeout occurs when backing up large databases. For those limited by host space, they would choose the normal download and upload method to backup and restore respectively.
Title: Re: [help wanted]:MySQL backup system
Post by: Joachim Müller on November 08, 2004, 09:58:59 am
Sounds promising. Do you have a script ready that accomplishes this?

Joachim
Title: Re: [help wanted]:MySQL backup system
Post by: loboclerk on November 10, 2004, 07:39:17 am
Lost my scripts long ago when I moved to phpBB. :P

Perhaps could code it again, should not be a big problem.

Will keep you updated :)
Title: Re: [help wanted]:MySQL backup system
Post by: donnoman on March 06, 2005, 10:04:24 pm
This project looks like a possibility for some to use, makes no mention of the specific license it is released as.

http://www.absoft-my.com/pondok/backup.php
Title: Re: [help wanted]:MySQL backup system
Post by: wildwalker on April 04, 2005, 11:02:42 pm
This idea is really great.

Like everyone else on this forum, I would like to be able to backup my databases. However, I would like to be able to backup directly to my backup web server, so if one server is down, the other is still running (although I would not allow uploads on the backup server)

A backup system on the next version of coppermine would be really really handy though.
Title: Re: [help wanted]:MySQL backup system
Post by: havoc615 on June 14, 2005, 02:43:14 am
I can do a little php coding, so if u guys needa ny help lemmie know
Title: Re: [help wanted]:MySQL backup system
Post by: Stramm on June 27, 2005, 03:44:57 pm
I'm using http://www.mysqldumper.de/en/index.php

from the webpage:
multilingual
Backup and restore of MySQL-Databases
Automatic sending of backups via Email or FTP
Automatic deleting of backups on the server following your own rules
Using a Perlscript as a Cronjob
Administration and manipulation of databases
Backup of database-structures (without data)
Backup and restore of tables
establishing of .htaccess to protect your backups
many details about your PHP-Version, MySQL-Version and more are shown
easy installation
easy handling
Multipart-Backup for very big databases
Multidatabase-Backup!
 
Title: Re: [help wanted]:MySQL backup system
Post by: François Keller on January 21, 2007, 03:18:13 pm
Hi all,

I have adapted a backup script to make a coppermine plugin.
see mi post here: http://forum.coppermine-gallery.net/index.php?topic=40439.0 (http://forum.coppermine-gallery.net/index.php?topic=40439.0)