Advanced search  

News:

cpg1.5.48 Security release - upgrade mandatory!
The Coppermine development team is releasing a security update for Coppermine in order to counter a recently discovered vulnerability. It is important that all users who run version cpg1.5.46 or older update to this latest version as soon as possible.
[more]

Pages: [1] 2 3 4 5 6   Go Down

Author Topic: Batch add with automatic album/cat structure from directory tree.  (Read 142701 times)

0 Members and 1 Guest are viewing this topic.

Nibbler

  • Guest

Here we go, upload this into your coppermine directory and call it timer.php (I hardcoded it, so sue me :P) then go there, you know the drill.

It will keep reloading until all the pics are added if you set it to, so you can start it and leave it unattended for hours until it finishes, even with php set with a very low max_execution_time (ignore the fatal error). Not recommended for shared hosting environments unless you are vewy vewy careful. :)

Note: Auto reload needs js support.

[Note: Tagged as 1.4, see later posts in thread for updated versions of this mod]

« Last Edit: January 04, 2006, 06:26:13 pm by Nibbler »
Logged

mrplow

  • Coppermine newbie
  • Offline Offline
  • Posts: 1
    • techangel
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #1 on: February 08, 2005, 10:13:33 pm »

You, sir, are a legend.

 ;D

I thank you for your sterling work.




I can't express how awesomely happy I am with this hack!  :D :D :D


edit (heh..)
EXCEPT... it's a shame it can't resume a folder/cat... if I bung a load more files into the same folder then ask your amazing script to do its work again, it starts a new catagory and populates it with new albums, all of the same names as before, and starts from scratch.
Shame about that :(
« Last Edit: February 09, 2005, 12:19:46 am by mrplow »
Logged

thumeau

  • Coppermine newbie
  • Offline Offline
  • Posts: 3
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #2 on: February 10, 2005, 05:09:38 pm »

Thank you so much Nibbler,

So glad to be able to batch input more than 1 album at a time.

I can't get to have the autorun / auto reload working eventhough JS is definitely on.
I tried both on Mozilla Firefox and IE...
Any ideas?

Thanks,
Thomas
Logged

oshelef

  • Coppermine newbie
  • Offline Offline
  • Posts: 12
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #3 on: March 08, 2005, 06:19:46 pm »

I get the following error when I try to run this:


"Created root category

Warning: Invalid argument supplied for foreach() in /var/www/html/coppermine/timer.php on line 200
structure created, 0 files to add."

The category is created, but no albums or pictures are included. (I've gotten more complicated errors before, but I got the above trying to use it add a single image, in a single directory)

I.e. directiory albums/test contains test.jpg and nothing else. permissions are all 775.

Any suggestions as to what might cause this?

Thanks,

Orie

Logged

Nibbler

  • Guest
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #4 on: March 08, 2005, 06:34:49 pm »

OK,  you make a folder in albums -> that becomes the root category.
You cannot then have any pics directly in that folder since you cannot have pics directly in a category.
You need to make folders inside the root folder - these become your albums, or more categories.

Bottom line is - You can only create a directory/file structure that would be a legal category/album/pic structure in coppermine.
Logged

oshelef

  • Coppermine newbie
  • Offline Offline
  • Posts: 12
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #5 on: March 08, 2005, 07:27:46 pm »

Ahh, I think I understand it. I need a folder structure that looks like:

Category/Album/Image.jpg right?

I redid my test and it worked; although it still spit out an error, it did insert my test image as it should.

Edit, it works with one image, but when I have more than one image I get this following error:

"Fatal error: Call to undefined function: array_diff_assoc() in /var/www/html/coppermine/timer.php on line 98
structure created, 700 files to add."

Thanks for your help.

Orie
« Last Edit: March 09, 2005, 07:04:26 am by oshelef »
Logged

Nibbler

  • Guest
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #6 on: March 09, 2005, 11:48:02 am »

Ah, seems the script requires PHP 4.3.0 or better to work. You can try adding the substitute function in the first comment here into the script, or you could upgrade your PHP, or you could try using PHP_Compat.
Logged

donnoman

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 1615
  • From donovanbray.com
    • Donovan Bray
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #7 on: March 14, 2005, 07:05:34 am »

Nibbler - Great script.

"It needed more power; so I rewired it"  (Tool Time Joke, hope somebody gets it)

I modified the script so that #1 you can run it against the album directory itself as the root cat.. There's an exclusion list that eliminates the basic garbage folders you might find there.

#2 It eliminates trying to add thumb_ and normal_ images that are already there from previous batch attempts.

#3 Checks to see if it can use a pre-existing cat/album before creating the new ones.

Which allows you to run this to do your batch adds all the time, successive runs don't hurt anything.

I was using this script against 1.4, but I didn't change anything that would be materially different than 1.3 so I would expect it to work on both platforms.

Also a nice little side effect of running this with 1.4 is that IPTC titles,keywords, and comments are automatically imported if you have iptc turned on in the config.

Logged

donnoman

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 1615
  • From donovanbray.com
    • Donovan Bray
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #8 on: March 14, 2005, 07:12:25 am »

@mrplow With my changes there should be no problem resuming. Do it as many times as you like.
Logged

Juniors

  • Coppermine newbie
  • Offline Offline
  • Posts: 6
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #9 on: March 23, 2005, 09:51:15 pm »

Edit (double Post)
......
« Last Edit: March 23, 2005, 10:14:26 pm by Juniors »
Logged

Juniors

  • Coppermine newbie
  • Offline Offline
  • Posts: 6
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #10 on: March 23, 2005, 09:52:13 pm »

Hello

With both Scripts i get the following Error when i call the timer.php from my Browser:

Quote
Parse error: parse error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING in /abc/def//timer.php on line 174

Any hints?

Thank you...
« Last Edit: March 24, 2005, 04:05:33 pm by Juniors »
Logged

donnoman

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 1615
  • From donovanbray.com
    • Donovan Bray
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #11 on: March 24, 2005, 04:51:29 am »

im sure the line number that errors out isn't the same between the two scripts. make a note of which script gives which line number.
Logged

kegobeer

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 4637
  • Beer - it does a body good!
    • The Kazebeer Family Website
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #12 on: March 24, 2005, 05:15:54 am »

This is probably a problem with the file format.  I've attached a plain text version of Donnoman's file.  Try this one and see if you still get those errors.
Logged
Do not send me a private message unless I ask for one.  Make your post public so everyone can benefit.

There are no stupid questions
But there are a LOT of inquisitive idiots

Juniors

  • Coppermine newbie
  • Offline Offline
  • Posts: 6
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #13 on: March 24, 2005, 12:04:12 pm »

Hi Kegobeer

Thank you for the Script. With this one i recieve the following Error:

Quote
Parse error: parse error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING in /abc/def/timer.php on line 416

Any hints again?

Junior
« Last Edit: March 24, 2005, 04:05:10 pm by Juniors »
Logged

Nibbler

  • Guest
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #14 on: March 24, 2005, 02:13:38 pm »

That file is 266 lines long. Therefore there cannot be an error on line 416. I suggest you try a different ftp client.
Logged

Juniors

  • Coppermine newbie
  • Offline Offline
  • Posts: 6
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #15 on: March 24, 2005, 03:59:02 pm »

Hi

Iam sorry. I fixed the Script. Now its exactly 266 lines long and my Error-Msg is:

Quote
Parse error: parse error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING in /abc/def/timer.php on line 192

Line 192:
Code: [Select]
$auto = (isset($_POST['auto']) && $_POST['auto']) ? 'checked = "checked"' : '';
 ???
Code: [Select]
« Last Edit: March 24, 2005, 04:04:34 pm by Juniors »
Logged

kegobeer

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 4637
  • Beer - it does a body good!
    • The Kazebeer Family Website
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #16 on: March 24, 2005, 06:12:43 pm »

Are you uploading your files as ASCII or binary?  These files need to be uploaded as ASCII, or use the auto switch so your FTP client will do it for you.
Logged
Do not send me a private message unless I ask for one.  Make your post public so everyone can benefit.

There are no stupid questions
But there are a LOT of inquisitive idiots

Juniors

  • Coppermine newbie
  • Offline Offline
  • Posts: 6
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #17 on: March 25, 2005, 01:03:29 pm »

I uploaded the filles in ASCII of corse  ;D
Logged

kegobeer

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 4637
  • Beer - it does a body good!
    • The Kazebeer Family Website
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #18 on: March 26, 2005, 02:25:53 am »

Then perhaps you should try a different FTP client.
Logged
Do not send me a private message unless I ask for one.  Make your post public so everyone can benefit.

There are no stupid questions
But there are a LOT of inquisitive idiots

flux

  • Contributor
  • Coppermine regular visitor
  • ***
  • Offline Offline
  • Posts: 80
    • CPG Remote Hack Demo
Re: Batch add with automatic album/cat structure from directory tree.
« Reply #19 on: April 11, 2005, 03:28:55 pm »

Hi,

here's my small participation to this nice and very usefull script..

Unfortunately as i didn't read the whole topic and especially donnoman's posts i didn't see that some of my modificatons had already been introduced and i just grabbed the file at the top of the topic, expecting it to be the latest version.. i was quite in a rush and started to make my modifications right away, silly me :)

Here's how the script works :
The script expects a directory name under ./albums/, that directory is supposed to be a root category in your gallery. Bottom directories, that should contain your pictures, will be treated as albums and intermediate directories will be treated as sub categories from the root category of course..
here's a quick example, let's say you have the following root category tree :
./albums/lib/
you should declare "lib" as the root directory to process.
now if you have the following tree under lib :
./albums/lib/A/Aname/pic.jpg

a root category named "lib" will be created, a sub category "lib>A", name "A" will be created, an album "lib>A>Aname" named "Aname" will be created and the picture pic.jpg will be added to the album "lib>A>Aname"..

that's how the original script works..

Now here are the features i've added to the original script :

In the same way donnoman did it, i added some filtering in order to avoid adding the same category/album/picture twice. So if you run the script twice on the aboce dirctory tree, everything should be created/added the first time and nothing will be done the second time..

Beware that there is a "limitation" in my database query on how category and album names are matched. Indeed a side efect of the query will be to limit creation of categories and albums to unique names. Which is not a limitation in the orignal design of CPG but is obviously a limitation of the filesystem (you cannot have the same directory name twice at the same location).. So you cannot have two categories or albums with the same name at the same location.. For example if you manually create, thru cpg interface, a second "Aname" album under "lib>A" and run the script to add new pics located in "/lib/A/Aname/" the script will use the first ID of the albums named "Aname".. maybe it's not worth mentioning, i don't know.. The same thing will happen with Category Names..
Of course if your second "Aname" album is located under "lib>Amisc>" then it's not a problem as the script will always use the reference to the parent when it tries to find if an album or category name already exists.. usually different parents mean different children ;)


Then the pictures filter was already there in the original script but was applied while processinf every pictures found.. I've add to that filter a global pictures filtering that will remove from the pictures lists build on the directory tree, the pictures already inserted in the database.. So for example if you update the directory tree under"./albums/lib", let's say it contains 20k+ pics but only have 150 new pictures, then my script will purge the already added 20k+ pics and add directly the 150 new pics.. quite nice isn'it  :)

A thing you shoul also know it that i've modified the way filenames are build.. In the original script any char not being part of the lists a-z A-Z 0-9 would be replaced by the char "_" (underscore).. I've improved that by removing existing double underscores "__" until there aren't any doubles left..

You'll probably notice that the renaming process will crush any non US chars, so you might want to modify that if you filenames have international chars in it..

Well i think it's all i can say about it..
I've tested it against a quite huge fake directory tree made of 26 subcategories (a-z) and 26x26 albums (26 albums in each 26 subcategories), each album containing 50 times the same picture named from 1.jpg to 50.jpg, which make a total of 33800 pictures..

The script take a few seconds (25sec) to build and filter the directory tree, which is located on a remote nfs server, then it will process the pictures according to the reloading parameters you've chosen.. The first update took a few hours as expected.. Then adding a few pics here and there just take a few seconds, no need to go over the previously added 30k pics right ? ;)

Thx for the coder of the original script..
Enjoy,

flux
Pages: [1] 2 3 4 5 6   Go Up
 

Page created in 0.024 seconds with 19 queries.