forum.coppermine-gallery.net

Support => cpg1.5 plugins => cpg1.5.x Support => cpg1.5 plugin contributions => Topic started by: Joachim Müller on August 19, 2009, 10:27:12 am

Title: Mass Import plugin for cpg1.5.x
Post by: Joachim Müller on August 19, 2009, 10:27:12 am
Mass Import gives the admin the ability to import large numbers of pictures organized by directory structure. The plugin is a port of the Mass Import plugin for cpg1.4.x (http://forum.coppermine-gallery.net/index.php/topic,27566.html).
The mass import works similarly to the batch-add process, but it allows you to add an entire structure of folders, subfolders and files to be added in one go. The plugin will create categories and albums that correspond to the folder names. It will then loop though the files in the structure and batch-add them to the database and create the resized images.
Use this plugin as well if you have issues with the regular batch-add process consuming too many resources.
You can download the tarball from the SVN: http://coppermine.svn.sourceforge.net/viewvc/coppermine/branches/cpg1.5.x/plugins/mass_import.tar.gz?view=tar if you know your way around. To download the plugin as a zip file package, click here: http://sourceforge.net/projects/coppermine/files/Plugins/1.5.x/cpg1.5.x_plugin_mass-import_v3.4.zip/download
Attached is a screenshot of the plugin "in action". Due to the nature of the plugin (it is admin-only) there is no demo available. The plugin will add another admin menu item named "mass import" next to the batch-add button.
Please understand that I will not support you in the usage of this plugin, so it's sort-of "take it or leave it".

Here's the content of the readme file that comes with the plugin:
Code: [Select]
/**************************************************

  Coppermine 1.5.x Plugin - mass_import

  *************************************************

  Copyright (c) 2010 Nibbler

  *************************************************

  This program is free software; you can redistribute it and/or modify

  it under the terms of the GNU General Public License as published by

  the Free Software Foundation; either version 3 of the License, or

  (at your option) any later version.

  ********************************************

  $HeadURL: https://coppermine.svn.sourceforge.net/svnroot/coppermine/branches/cpg1.5.x/plugins/mass_import/readme.txt $

  $Revision: 7149 $

  $LastChangedBy: gaugau $

  $Date: 2010-01-28 10:17:02 +0100 (Do, 28. Jan 2010) $

  **************************************************/

What it does
============
Mass Import gives the admin the ability to import large numbers of pictures organized by directory structure.

Details
=======
The mass import works similarly to the batch-add process, but it allows you to add an entire structure
of folders, subfolders and files to be added in one go.
The plugin will create categories and albums that correspond to the folder names. It will then loop though
the files in the structure and batch-add them to the database and create the resized images.
Use this plugin as well if you have issues with the regular batch-add process consuming too many resources.

Announcement
============
Visit the announcement thread of this plugin for breaking news
http://forum.coppermine-gallery.net/index.php/topic,61281.0.html

SVN checkouts
=============
Use subversion checkout to get the latest and greatest version of this plugin. Refer to the documentation
that comes with Coppermine to find out details on the Subversion repository.
Check out https://coppermine.svn.sourceforge.net/svnroot/coppermine/branches/cpg1.5.x/plugins/mass_import/
or use WebSVN to browse the repository at
http://coppermine.svn.sourceforge.net/viewvc/coppermine/branches/cpg1.5.x/plugins/mass_import/

Credits
=======
Nibbler initially created this as a mod of the batch-add function that featured the delayed execution.
Donnoman converted the mod into a plugin for cpg1.4.x
Flux and Paul Van Rompay contributed code to improve the functionality.
Joachim Müller ported the plugin for cpg1.5.x and spiced up the output a bit.

Support
=======
The scope of support for this tool is limited. Basically, it comes as-is.

Todo
====

The changelog's content:
Code: [Select]
/**************************************************

  Coppermine 1.5.x Plugin - mass_import

  *************************************************

  Copyright (c) 2010 Nibbler

  *************************************************

  This program is free software; you can redistribute it and/or modify

  it under the terms of the GNU General Public License as published by

  the Free Software Foundation; either version 3 of the License, or

  (at your option) any later version.

  ********************************************

  $HeadURL: https://coppermine.svn.sourceforge.net/svnroot/coppermine/branches/cpg1.5.x/plugins/mass_import/changelog.txt $

  $Revision: 7149 $

  $LastChangedBy: gaugau $

  $Date: 2010-01-28 10:17:02 +0100 (Do, 28. Jan 2010) $

  **************************************************/



Changelog

=========

[A] = Added new feature

[B] = Bugfix (fix something that wasn't working as expected)

[C] = Cosmetical fix (layout, typo etc.)

[D] = Documentation improvements

[M] = Maintenance works

[O] = Optimization of code

[S] = Security fix (issues that are related to security)

*********************************************


2010-01-28 [M] Counted plugin version up from 3.3 to 3.4 {GauGau}
2010-01-28 [A] Added Finnish language file {GauGau}

2010-01-28 [M] Added changelog {GauGau}

2009-08-20 [M] First public release of the plugin for cpg1.5.x (plugin version 3.3) {GauGau}
2009-08-20 [D] Added screenshot to install information {GauGau}
2009-08-20 [O] I18n of configuration file {GauGau}
2009-08-19 [A] Addeded spin button {GauGau}
2009-08-19 [A] Addeded German language file {GauGau}
2009-08-19 [A] Added menu icons {GauGau}
2009-08-19 [C] Converted admin menu entry function for cpg1.5.x {GauGau}
2009-08-18 [B] Inspectified form {GauGau}
2009-08-18 [C] Spiced up form {GauGau}
2009-08-18 [M] Added SVN header {GauGau}
2009-08-18 [M] Renamed plugin folder from CPGMassImport to mass_import to respect naming conventions {GauGau}
2009-08-18 [M] Increased plugin version count from 2.0 to 3.0 to reflect the in-depth changes {GauGau}
Title: Re: Mass Import plugin for cpg1.5.x
Post by: papukaija on September 06, 2009, 01:56:27 pm
Great plugin. The Finnish translation is added to this message.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: sammyd109 on June 22, 2010, 09:14:33 pm
When using this plugin to import file structures that contain several level of folders, what sort of structure of albums or categories will be used? Will they follow the folder structure or will they simply import the pictures into a mass album?

http://ec2-75-101-214-77.compute-1.amazonaws.com/login.php
Title: Re: Mass Import plugin for cpg1.5.x
Post by: sammyd109 on June 23, 2010, 03:03:17 pm
The login info for this site is as follows:

email: test@test.com
password: **** [/Edit Joachim] Edited out the password of the admin account you posted [/Edit]
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Joachim Müller on June 25, 2010, 07:30:27 am
That's an admin account. Posting an admin account publicly is silly. I have logged in and changed the password of that account. I have edited your posting above. Remove that account from your gallery.
The mass import plugin is not installed in your gallery, so what should the test user account do? Don't expect to be spoon-fed. Stop wasting our time - all you're doing is causing extra moderation effort.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: panic83 on September 20, 2010, 11:02:02 am
Hi,
 Coppermine Photo Gallery 1.5.8 (stable)

Gallery Configuration - File settings - Auto resize images that are larger than max width or height

When i set this settings to "NO", i got error  in "Mass Import".

settings:
http://img196.imageshack.us/img196/6343/screenshot1vx.png
error:
http://img214.imageshack.us/img214/5506/screenshotitp.png

Please advice.

I just want to add photos form folder without  any convention.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: panic83 on September 20, 2010, 11:14:15 am
without  any convention.

without convertion of course, sorry for typo
Title: Re: Mass Import plugin for cpg1.5.x
Post by: justforthispost on September 23, 2010, 02:00:54 pm
First of all I want to say thanks for this wonderful plugin. I've used it to add more then 250 albums to my gallery!

There is however 1 little thing that is bothering me. When I manually create an album the user permissions on the album are set to: uploads no, comments yes and votes yes. On the support forums here I found a way to chance the upload permissions to yes due to a modification in the delete.php file. This works like a charm when manually creating albums.

But if I add the albums with the Mass Import plugin the permissions aren't set as I want to. Is there a way that you give us the option to chance the permissions in a new release of Mass Import?

Kind regards.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on September 24, 2010, 09:21:05 am
Is there a way that you give us the option to chance the permissions in a new release of Mass Import?
Please understand that I will not support you in the usage of this plugin, so it's sort-of "take it or leave it".

So you have 2 options:
1. Change all settings with an SQL query
2. Modify the plugin code
Title: Re: Mass Import plugin for cpg1.5.x
Post by: justforthispost on September 27, 2010, 03:52:58 pm
Thanks Αndré.

I'll go for option 1 then.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: knym on April 07, 2011, 01:10:29 am
First of all I want to say thanks for this wonderful plugin. I've used it to add more then 250 albums to my gallery!

There is however 1 little thing that is bothering me. When I manually create an album the user permissions on the album are set to: uploads no, comments yes and votes yes. On the support forums here I found a way to chance the upload permissions to yes due to a modification in the delete.php file. This works like a charm when manually creating albums.

But if I add the albums with the Mass Import plugin the permissions aren't set as I want to. Is there a way that you give us the option to chance the permissions in a new release of Mass Import?

Kind regards.

While your there, and anyone who reads after and finds the same delema.. load your servers mysql database editor or install something like phpmyadmin.. locate in the database table albums .. default install being "cpg15x_albums
" and inside there find the field uploads.. it's an anum field that accepts YES or NO.. The default value is NO.. simple change it to YES and from that time forward YES will be selected as default instead of NO.  Note you can change the defaults for the other two fields (comments and votes) mentioned in post as well. Those are default set to YES.

All future problems solved..
Title: Small MySQL insert change suggestion
Post by: ls5302 on May 24, 2011, 01:08:55 am
I have just upgraded to Coppermine version 1.5.12, and installed the Mass Import plugin. It looks like with MySQL 5.5.10 the description field in the categories table cannot have a default value of '', yet a couple of inserts statements in import.php fail to assign an explicit value. I'd like to suggest, if I may, the modification of the following lines in import.php:

Line 168:
$db_query_wrapper("INSERT INTO {$CONFIG['TABLE_CATEGORIES']} (pos, parent, name, description) VALUES ('10000', '$parent', '" . addslashes($name) . "', '')");

Line 360:
$db_query_wrapper("INSERT INTO {$CONFIG['TABLE_CATEGORIES']} (pos, parent, name, description) VALUES ('10000', '0', '{$post_directory}', '')");

Great plugin!!!
Title: Small MySQL insert change suggestion (One more)
Post by: ls5302 on May 24, 2011, 01:29:06 am
Cannot edit post and missed one change:

Line 189:
$db_query_wrapper("INSERT INTO {$CONFIG['TABLE_ALBUMS']} (category, title, pos, description) VALUES ('".(INT)$category."', '" . addslashes($title) . "', '10000', '')");
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on May 24, 2011, 11:40:34 am
I haven't checked the behavior of MySQL 5.5.10, but committed your changes in svn revision 8209 (http://coppermine.svn.sourceforge.net/viewvc/coppermine/branches/cpg1.5.x/plugins/mass_import/import.php?r1=7119&r2=8209). People who have the same issue as ls5302 should use the latest svn version (http://coppermine.svn.sourceforge.net/viewvc/coppermine/branches/cpg1.5.x/plugins/mass_import/?view=tar) of the plugin until no new version is released.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 28, 2012, 07:47:27 am

The mass import works similarly to the batch-add process, but it allows you to add an entire structure of folders, subfolders and files to be added in one go. The plugin will create categories and albums that correspond to the folder names. It will then loop though the files in the structure and batch-add them to the database and create the resized images.


Oh Good Lord!

Have you been reading into my mind ?
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 01:04:24 am
Hallo Joachim, Bonsoir André,

the tool worked very nicely on my gallery and properly imported all subfolders as categories...... However it did not import any one single image......

any idea?
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 05:27:01 am
 :( I even contacted my support and asked them to check the error logs.

Curiously, there's nothing in the server's error_logs either.

You would expect to see something if there were a permissions error.

Is it possible that because I have about 500 subfolders with about 70,000 files in all that the script hit a resource limit (such as a memory limit) while enumerating the files, and simply died before it could complete ?

Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 08:03:17 am
I figured out that memory was not enough. I regret your script did not display clear crash messages (maybe actually that was in the collapsed debug in the footer.... )

Anyway, memory was increased and script is working now.......

One very strange thing though.....

every single picture added belongs to no album.

If you look in the database, all the imported pictures belong to album 0 (aid=0)

Any idea why ?

Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 08:11:29 am
and all albums belong to category 0 .....  :-[ :-[ :-[ :-[
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 08:13:40 am
and while the images are being added in batches of 10, I am regularly disconnected with this message:

Error
You don't have permission to access this page.

But I was logged in as admin and the mass import tool was working........ why am I then still disconnected ? There is no inactivity there at all as the mass import refires itself every 1000 millisecond.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 07:57:37 pm
is there anything I could do to avoid being logged out after a few minutes?

is there anything I could do for images to be assigned correctly to their albums?

is there anything I could do for albums to be properly assigned to their categories?

Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 08:00:54 pm
Can anyone kindly simply point me in the right direction ?

Is it possible that the mass import tool is Not meant to be used in a bridged gallery but should be used in a native one?

I don't mind breaking the bridge and going back to native coppermine. But after the mass import, would I be able to re-bridge without losing the albums and categories?
Title: Re: Mass Import plugin for cpg1.5.x
Post by: pols1337 on February 29, 2012, 08:07:40 pm
Can anyone kindly simply point me in the right direction ?

Is it possible that the mass import tool is Not meant to be used in a bridged gallery but should be used in a native one?

I don't mind breaking the bridge and going back to native coppermine. But after the mass import, would I be able to re-bridge without losing the albums and categories?

Hi S73,

I'm not an expert in Coppermine, but I got the 'mass import tool' to work for me in a bridged gallery (bridged with SMF2.0). 

If you do decide break the bridge and re-bridge, that shouldn't affect the albums or categories at all from what I understand.

And finally, I think you experienced the same problem as me when I first tried to run Mass Import -- the pictures got imported but were imported into a blank category.  As an admin, go to "Categories" and delete the blank category ... hopefully, that will un-hide the albums that were hidden in the blank category. 

And lastly, I would honestly recommend that you do Batch Add Files, simply because it's the safer method to add pictures.  I'm still testing out Mass Import Plug-in, and nobody supports the plug-in so you won't get any 'official' support on it. 

Cheers,
pols1337
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 08:17:31 pm
thank you loads pols and I highly appreciate your kind help.

Mine is bridged to vbulletin4 (the bridge says it supports vbulletin3).

I would have used the batch tool ...... but for 70,000 pics in 500 subfolders (albums) in 30 categories......... that is a loooooooooot of work to do manually.

I still thank you for your great feedback. I am breaking the bridge and trying now.

You know pols, I many times heard this " official support".

Well I never asked for official support. Any kind advice that helps me move a bit closer to my target is very welcome.

Title: Re: Mass Import plugin for cpg1.5.x
Post by: pols1337 on February 29, 2012, 08:25:40 pm
I have 50,000 pictures in 400 folders.  Trust me, batch adding is not that bad  :-\

Also, breaking the bridge won't do anything.  I hope you understand that.  All that will do is modify the users and permission settings, but won't change how you add photos. 
Title: Re: Mass Import plugin for cpg1.5.x
Post by: S73 on February 29, 2012, 08:32:40 pm
I have 50,000 pictures in 400 folders.  Trust me, batch adding is not that bad  :-\

Also, breaking the bridge won't do anything.  I hope you understand that.  All that will do is modify the users and permission settings, but won't change how you add photos.

Wow. you must be a very patient man. Or that you added those over some period of time.

I followed your advice, broke the bridge, and guess what...... things look much better now.

Categories were created properly, albums too, albums belong to their categories, and all the pics belong to their owner....... that is much better indeed.

I will let you know how it goes, but for the moment, 1000 pics were added to the right albums in the right categories........ hopefully. I will have to check thoroughly.

Title: Re: Mass Import plugin for cpg1.5.x
Post by: DirtRider on March 29, 2012, 01:13:54 pm
Ok I installed it but I just cannot get it to work at all. I create a folder under my albums folder called ftp. Now when I try and do the import it in the option to enter what folder it is or leave blank I put in ftp and I also tried leaving it blank. I then get this error

Critical error
There was an error while processing a database query
 
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on March 29, 2012, 02:11:56 pm
Enable debug mode to get an extended error message.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: DirtRider on March 30, 2012, 07:22:58 am
Here we go

Code: [Select]
Critical error

There was an error while processing a database query.

While executing query 'INSERT INTO cpg15x_categories (pos, parent, name) VALUES ('10000', '0', '')' in plugins\mass_import\import.php on line 360 mySQL error: Field 'description' doesn't have a default value


File: C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\dasphoto\include\functions.inc.php - Line: 270



Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on March 30, 2012, 12:11:02 pm
Quote
Field 'description' doesn't have a default value
Either you adjust your database and give that field a default description or you modify the plugin code to submit the description.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: miblo69 on April 10, 2012, 10:49:57 pm
Hi,

There seems to be a problem with mass import when the directory names contain non-ASCII characters. In my case several directories contain swedish characters: åäöÅÄÖ. The import 'succeeds', but no images imported to the Albums that have Swedish characters in their names. Albums (directories) without characters are displayed correctly and contain the images as should.

I tried to locate the function in import.php, but I am no php coder.... If any one has tips on how to solve this, it would be greatly appreciated!

And, no, removing all Swedish characters in the directory names is not an option... I have some 80k pictures in 3000 directories I want to import.

Many Thanks!
~Mike
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on April 12, 2012, 10:14:32 am
removing all Swedish characters in the directory names is not an option... I have some 80k pictures in 3000 directories I want to import.
Would it be an option if the script renames your files and folders automatically before it adds them to the gallery?
Title: Re: Mass Import plugin for cpg1.5.x
Post by: miblo69 on April 20, 2012, 10:13:24 am
No, that wouldn't be an option. The whole Coppoermine Gallery suports UTF-8, so I would assume the Mass-Import would also support it.

I did some elementary troubleshooting, and I can see all the albums (including thoe with nonASCII characters) in the cpg_albums table. But the Pictures that reside in a non_ASCII directory, are not available in the cpg_pictures table. So part of the import seems to work. I.e Albums are ok, but not pictures. I would think there is a missing 'character conversion procedure' in the Picture import, but it is present in the Album import.

But my php knowledge limits me from troubleshooting this further...

Thanks,
~Mike
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on April 20, 2012, 02:13:23 pm
Please try the following. Open import.php, find
Code: [Select]
$sql = "SELECT aid " . "FROM {$CONFIG['TABLE_ALBUMS']} " . "WHERE title='" . addslashes($title) . "' AND category=" . (INT)$category . " LIMIT 1";and replace with
Code: [Select]
$sql = "SELECT aid " . "FROM {$CONFIG['TABLE_ALBUMS']} " . "WHERE title='" . addslashes(utf8_encode($title)) . "' AND category=" . (INT)$category . " LIMIT 1";
find
Code: [Select]
$sql = "SELECT pid " . "FROM {$CONFIG['TABLE_PICTURES']} " . "WHERE filepath='" . addslashes($dir_name) . "' AND filename='" . addslashes($sane_name) . "' " . "LIMIT 1";and replace with
Code: [Select]
$sql = "SELECT pid " . "FROM {$CONFIG['TABLE_PICTURES']} " . "WHERE filepath='" . addslashes(utf8_encode($dir_name)) . "' AND filename='" . addslashes($sane_name) . "' " . "LIMIT 1";

This fixes the functionality in my testbed. However, it doesn't fix the output, but that's currently not important IMHO.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: miblo69 on April 26, 2012, 11:31:17 pm
HI,

No, it stil doesnt fix the problem. I removed the database completely, and did a fresh install. Copied my albums to import, to the /albums/ directory. Started Mass_import, applied your suggested changes, and it scrolled through all files. But unfortunately nothing was imported. It creates categories for each album, but they don't contain any pictures.

So the change has effectively stopped import of *all* albums. Previously it was only albums with non-ASCII characters in the directory name. Let me know what debug info you need and I'll provide it.

Thanks,
~Mike
Title: Re: Mass Import plugin for cpg1.5.x
Post by: sindbad5 on August 06, 2014, 07:57:17 pm
I've installed the plugin. It creates categories and albums but will upload only up to 20 pictures per go. This is not helpfull  if there are some thousand images.

Tom

   
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on August 07, 2014, 12:48:15 pm
Unattended run works for me as expected. Please wait some time when you see the window you attached as screenshot. I'm sure it will refresh and the number of remaining files will decrease.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: ge-flopt on July 09, 2015, 12:22:46 am
Hopefully you can help me. Have migrated mine coppermine site to a new disk. Have all the files, but forgot to transfer the database. I can't retrieve it, because the disk is already formated. Well hopefully mass import can help me. I have only one problem. The import tool gives me the following error:
Fatal error: Maximum execution time of 30 seconds exceeded in /......./plugins/mass_import/import.php on line 55

I can change the tomeout to a higher number, but i don't know if that's the correct way to go.

Any ideas?

Specs:
Server: Raspberry pi (it does the job!)
Images: 128.148 (Full files, Normal files and thumb files included)

THX!
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Phill Luckhurst on July 09, 2015, 11:16:45 am
Yes, that is the correct way to go. Try 60 seconds and gradually increase it until the script works.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on July 09, 2015, 11:28:41 am
If I remember correctly the mass import plugin always imports a whole directory. So increasing the execution time might work for some directories, but maybe not for larger directories. I don't know if the plugin regards already existing intermediate-sized pictures and thumbnails. If so, the script just needs to add some meta data to the MySQL database. If you're hosting the database (and files) on the SD card, it's maybe a good idea to transfer them to a HDD to increase performance.

You maybe also want to run the import process on your desktop PC, which has usually much more power, and then export/import the database dump when completed. So you avoid any PHP timeout issues.


OT: I always wanted to install Coppermine on one of my Pis (early model B with 256 and 512 MB RAM) and check how it performs, but never done that. Do you use a Raspberry Pi 1 or 2? How's the performance / Coppermine page load times?
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Phill Luckhurst on July 09, 2015, 12:03:07 pm
Good idea on the PI, I may give that a whirl at the weekend.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: ge-flopt on July 09, 2015, 04:27:56 pm
I'm using a Raspberry Pi B (512 MB) it has an external harddisk, and for a "small" (not highly used, personal) coppermine installation, it works nice. :D The one thing that's slow is when you click search and it starts indexing all the words in every record.

Tip. Use Berryboot and an external harddisk. Berryboot boots the pi and then loads everything from the harddisk.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: ge-flopt on July 11, 2015, 10:16:19 pm
Well the mass update didn't work as thought. Different PC's and browsers an all stopt working eventually.

Made a php script that imports all the files, checks if the album exist and if not creates it. In about 5 minutes it loaded 5000 images.
The main difference between mass import and my script is that my script is a "hack" and it only works on a directory tree that already was once loaded into a coppermine gallery.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on July 13, 2015, 12:33:41 pm
Would you be so kind and share your solution with the community? Thanks.
Title: Re: Re: Mass Import plugin for cpg1.5.x
Post by: ge-flopt on July 13, 2015, 09:43:27 pm
Would you be so kind and share your solution with the community? Thanks.
Sure. Shall i place it in here or place it in a new topic?
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on July 13, 2015, 10:06:29 pm
I think it's good placed here as a reply.
Title: Re: Mass Import plugin for cpg1.5.x
Post by: ge-flopt on July 14, 2015, 05:07:18 pm
Here it is!
Please note the following:
- Make sure you have a backup of your pictures!
- Use it on your own risk!
- Make sure you have a backup of your pictures!
- Do not use it in a functioning Coppermine installation
- Make sure you have a backup of your pictures!
- It is tested and created to be used with PHP CLI (command line interface) so you need shell access.
- Make sure you have a backup of your pictures!
- Modify the parts of the strings in capital letters to suite your own setup.
- Make sure you have a backup of your pictures!
- Everything will be inserted into the first users account.
- Make sure you have a backup of your pictures!
- I call this a hack because it's not using any CM functionality accept from the config.inc.php and it doesn't check if you have logged in to CM.
- Make sure you have a backup of your pictures!
- I slammed this script up to meet my settings and prefferences, they may not meet yours.
- Make sure you have a backup of your pictures!

Have any questions? Please post, then i'll try to answer them.
Did i already mention: "Make sure you have a backup of your pictures!"?

Code: [Select]
<?php
include '/FULL PATH TO THE FOLDER WHERE THE NEXT FILE IS LOCATED/config.inc.php';


$CONN mysql_connect($CONFIG["dbserver"],$CONFIG["dbuser"],$CONFIG["dbpass"]) or die ("dead!!");
$searchpath '/FULL PATH OF YOUR COPPERMINE INSTALL/albums/userpics/INCLUDING USERNAME OR ID';
$basepath '/FULL PATH OF YOUR COPPERMINE INSTALL/albums/';
$basealbumpath 'userpics/USERNAME OR ID/';
$it = new RecursiveDirectoryIterator($searchpath);
foreach(new 
RecursiveIteratorIterator($it) as $file
{
/* echo $file . "\n";*/
$shortfile str_replace ($basepath""$file);
if (substr($shortfile, -2) != '/.')
{ if (substr($shortfile, -3) != '/..')
{ $basefile basename($shortfile);
if ((substr($basefile,0,7) != 'normal_') and (substr($basefile,0,6) != 'thumb_')) 
{ $shortdir str_replace("/".$basefile,"",$shortfile);
$albumname str_replace($basealbumpath,""$shortdir);
$file_size filesize($file);
list($width,$height) = getimagesize($file);
$ctime filectime($file);
$NORMAL $searchpath."/".$shortdir."/normal_".$basefile;
$THUMB $searchpath."/".$shortdir."/thumb_".$basefile;
//echo $NORMAL ."\n";
$TOTALFILESIZE $file_size;
if (file_exists($NORMAL))
{ $TOTALFILESIZE $TOTALFILESIZE filesize($NORMAL);
}
if (file_exists($THUMB))
{ $TOTALFILESIZE $TOTALFILESIZE filesize($THUMB);
}
/*
echo "FILE:      ".$file . "\n";
echo "SHORTFILE: ".$shortfile . "\n";
echo "SHORTDIR:  ".$shortdir . "\n";
echo "FILE_SIZE: ".$file_size . "\n";
echo "WIDTH:     ".$width . "\n";
echo "HEIGHT:    ".$height . "\n";
echo "CTIME:     ".$ctime . "\n";
*/
$query 'select aid from `'.$CONFIG["dbname"].'`.'.$CONFIG["TABLE_PREFIX"].'albums where title = "'.$albumname.'"';
//echo $query . "\n";
$albumid mysql_fetch_assoc(mysql_query($query));
if (empty($albumid))
{ echo "Adding folder: ".$shortdir."\n";
$albumcreatequery 'insert into `'.$CONFIG["dbname"].'`.'.$CONFIG["TABLE_PREFIX"].'albums 
(title,description,visibility,uploads,comments,votes,pos,category,owner,thumb,keyword,alb_password,alb_password_hint, moderator_group,alb_hits)
values ("'
.$albumname.'","",0,"NO","YES","YES",1000,10001,1,0,NULL,NULL,NULL,0,0)';
// echo $albumcreatequery . "\n";
mysql_query($albumcreatequery);
$query 'select aid from `'.$CONFIG["dbname"].'`.'.$CONFIG["TABLE_PREFIX"].'albums where title = "'.$albumname.'"';
//echo $query . "\n";
$albumid mysql_fetch_assoc(mysql_query($query));
}
$query 'select pid from `'.$CONFIG["dbname"].'`.'.$CONFIG["TABLE_PREFIX"].'pictures where filepath = "'.$shortdir.'/" and filename = "'.$basefile.'"';
//print $query. "\n";
$pictureid mysql_fetch_assoc(mysql_query($query));
if (empty($pictureid))
{ echo "Adding file  : ".$basefile." - ".$shortdir."\n";
$filecreatequery 'insert into `'.$CONFIG["dbname"].'`.'.$CONFIG["TABLE_PREFIX"].'pictures 
(aid,filepath,filename,filesize,total_filesize,pwidth,pheight,hits,mtime,
ctime,owner_id,pic_rating,votes,title,caption,keywords,approved,galleryicon,
user1,user2,user3,user4,url_prefix,pic_raw_ip, pic_hdr_ip,lasthit_ip,position,guest_token )
values ("'
.$albumid["aid"].'","'.$shortdir.'/","'.$basefile.'","'.$file_size.'","'.$TOTALFILESIZE.'","'.$width.'","'.$height.'",0,"0000-00-00 00:00:00",
"'
.$ctime.'",1,0,0,"","","","YES",0,
"","","","",0,"","",NULL,0,"")'
;
// echo $filecreatequery . "\n";
mysql_query($filecreatequery);
}

}
}
}
}
mysql_close($conn);
?>

   
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Surfer2010 on December 28, 2015, 04:26:20 pm
Hello,

i tried to use this Plugin, but i had the effekt, that the existing folders were imported as cataloge and album alternating.
Folder1 --> catalog
Folder2 --> album
Folder3 --> catalog
Folder4 --> album ... and so on
Why is this (it's independent of the real file/folder structure.

Is there by now a new Plugin to import an entire file-folder structure? Would be a nice thing to have.

Thanks for your help and reply.

Surfer2010

(otherwise a great tool)
Title: Re: Mass Import plugin for cpg1.5.x
Post by: ge-flopt on February 23, 2016, 10:08:48 pm
Please note that my script isn't a plugin. It's just a hack to accommodate to my own preferences.

My script does the following:
Name1/Dir1/Dir2/image1.jpg
It places image1.jpg in folder Dir1/Dir2 for user Name1.

So "real" example:
Michael/2009/vacation/swimming.jpg
Will become:
swimming.jpg in folder 2009/vacation for user Michael
Title: Re: Mass Import plugin for cpg1.5.x
Post by: kylemj on April 10, 2016, 12:24:05 pm
Hi,
Hope this is being supported. I used the plugin and it created the album name but didn't add the photos but when going to "sort my pictures" all they photos are in the root directory (select album) but there seems to be no way of deleting them since i cannot access them via the gallery
Title: Re: Mass Import plugin for cpg1.5.x
Post by: Αndré on April 13, 2016, 11:51:41 am
Not sure what happened, it usually seems to work well for other users. If it's a new gallery without any existing data I suggest to start from scratch and try again. You don't need to re-install the gallery in this case, but can just empty the "pictures" table in your MySQL database with a tool like phpMyAdmin. The same applies for the "albums" and "categories" tables.