Advanced search  

News:

CPG Release 1.6.26
Correct PHP8.2 issues with user and language managers.
Additional fixes for PHP 8.2
Correct PHP8 error with SMF 2.0 bridge.
Correct IPTC supplimental category parsing.
Download and info HERE

Pages: [1]   Go Down

Author Topic: Uploading Big Photos in Large Quantities  (Read 11820 times)

0 Members and 1 Guest are viewing this topic.

emperoradz

  • Coppermine newbie
  • Offline Offline
  • Posts: 3
Uploading Big Photos in Large Quantities
« on: October 09, 2009, 01:05:33 am »

We are running Coppermine + JUpload (latest version) on our Yearbook Server.

Here are our settings:
Max size for uploaded files (KB): 10000000000000
Max width or height for uploaded pictures/videos (pixels): 2048000
maxChunkSize: 2000
maxFileSize: 50000
maxNonTransformedPictureSize: 50000
pictureCompressionQuality: 1

We upload all kinds of photos (7MP, 8MP, 12MP, etc). The largest we usually upload is 12MP (7000-10000KB).
We want NO resizing and modification. Just plain uploads.

Now when we try to upload 100 or so 12MP files, it's fine. But when we try uploading 500 or so, after 127 it starts complaining that the files are too big, and we need to click OK repeatedly for the 400 remaining files.

Any way to turn off ALL warnings, turn of resizing of any kind, and let any picture through unblemished.

Also, when is metadata preservation coming....
Logged

Joachim Müller

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 47843
  • aka "GauGau"
    • gaugau.de
Re: Uploading Big Photos in Large Quantities
« Reply #1 on: October 09, 2009, 07:52:25 am »

We are running Coppermine + JUpload (latest version) on our Yearbook Server.
JUpload is a plugin that is not part of coppermine itself. It only get's supported within the very sub-board "cpg1.4 JUpload by etienne_sf"

Max size for uploaded files (KB): 10000000000000
Use settings that make sense: 10,000 TB is just plain stupid and might cause issues.
Max width or height for uploaded pictures/videos (pixels): 2048000
That's of course silly as well.
maxChunkSize: 2000
maxFileSize: 50000
maxNonTransformedPictureSize: 50000
pictureCompressionQuality: 1
Those are JUpload settings. I have no idea of their impact.

We want NO resizing and modification. Just plain uploads.
You don't even want thumbnails to be created? OK, that's an unusual setup.

Now when we try to upload 100 or so 12MP files, it's fine. But when we try uploading 500 or so, after 127 it starts complaining that the files are too big, and we need to click OK repeatedly for the 400 remaining files.
Happens when? Using JUpload or using coppermine's batch-add?

Any way to turn off ALL warnings
What warnings? Batch-add is an admin only feature. The admin should be able to live with some warnings.
turn of resizing of any kind
Sure: set the max pixel size for thumbnails and intermediates to the same value you have set your max resolution to.

Also, when is metadata preservation coming....
As far as Coppermine is concerned: as soon as the image libraries that Coppermine can use (GD or ImageMagick) will support meta tag preservation we'll add that feature to coppermine as well. We can't add features to coppermine that the underlying architecture don't have.

We have a strict "one question per thread" policy that you agreed to respect when signing up. Another policy says that it's absolutely mandatory to post a link to your gallery in each thread. And yet another policy says that you must have read the docs before being allowed to post on the support board. If you have issues with uploading (and as far as I am concerned batch-add is an uploading method just as well), do exactly as suggested in the docs, section "asking for support on upload issues".

If you have posted this in error on the upload sub-board and where looking for JUpload support only, say so in your reply, so a moderator can move your thread accordingly. Most of the recommendations posted above apply nevertheless though, as JUpload is using the server-side of coppermine.
Logged

emperoradz

  • Coppermine newbie
  • Offline Offline
  • Posts: 3
Re: Uploading Big Photos in Large Quantities
« Reply #2 on: October 13, 2009, 10:41:30 pm »

Ah looks like I posted it in the wrong place. Can you please move this to JUpload?

Quote
Use settings that make sense: 10,000 TB is just plain stupid and might cause issues.That's of course silly as well.Those are JUpload settings. I have no idea of their impact.
Meant to be no limit, wasn't sure how to do it so just put 10 PB

Quote
You don't even want thumbnails to be created? OK, that's an unusual setup.
We do, we just don't want the original file to be edited.

Quote
Happens when? Using JUpload or using coppermine's batch-add?
JUpload, we only use JUpload

Quote
What warnings? Batch-add is an admin only feature. The admin should be able to live with some warnings.Sure: set the max pixel size for thumbnails and intermediates to the same value you have set your max resolution to.
Says size is too big so it won't be resized (JUpload warning)

Quote
We have a strict "one question per thread" policy that you agreed to respect when signing up. Another policy says that it's absolutely mandatory to post a link to your gallery in each thread. And yet another policy says that you must have read the docs before being allowed to post on the support board. If you have issues with uploading (and as far as I am concerned batch-add is an uploading method just as well), do exactly as suggested in the docs, section "asking for support on upload issues".
It's being a private network

Quote
If you have posted this in error on the upload sub-board and where looking for JUpload support only, say so in your reply, so a moderator can move your thread accordingly. Most of the recommendations posted above apply nevertheless though, as JUpload is using the server-side of coppermine.
yup sorry. Please move it to JUpload board
Logged

Joachim Müller

  • Dev Team member
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 47843
  • aka "GauGau"
    • gaugau.de
Re: Uploading Big Photos in Large Quantities
« Reply #3 on: October 14, 2009, 08:27:16 am »

It's being a private network
Then there can only be generic advice: make sure that regular http uploads work as expected first before uploading with JUpload. Can't help you with JUpload issues, as I don't know/understand/use it, so I'm out of this thread. Bye.

It's being a private network
Then there can only be generic yup sorry. Please move it to JUpload board[/quote]Phill did as you suggested.
Logged

etienne_sf

  • Moderator
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 908
    • Wiki
Re: Uploading Big Photos in Large Quantities
« Reply #4 on: October 16, 2009, 04:37:57 pm »

Hi,

  I missed this message as I (apparently) get no alarm in my mail box, when a message is moved. Sorry for that.

Here are my answer, about JUpload :

We are running Coppermine + JUpload (latest version) on our Yearbook Server.

Here are our settings:
Max size for uploaded files (KB): 10000000000000
Max width or height for uploaded pictures/videos (pixels): 2048000
maxChunkSize: 2000
maxFileSize: 50000
maxNonTransformedPictureSize: 50000
pictureCompressionQuality: 1

We upload all kinds of photos (7MP, 8MP, 12MP, etc). The largest we usually upload is 12MP (7000-10000KB).
We want NO resizing and modification. Just plain uploads.

Now when we try to upload 100 or so 12MP files, it's fine. But when we try uploading 500 or so, after 127 it starts complaining that the files are too big, and we need to click OK repeatedly for the 400 remaining files.

Can you put here the JUpload debug output ?

 I never managed so many such big files...

Any way to turn off ALL warnings, turn of resizing of any kind, and let any picture through unblemished.

I don't know exactly about he Warnings.

To turn off resizing:
- Keep the parameter you've choosen
- Put in maxFileSize a bigger value, like you did for 'Max size for uploaded file'


Also, when is metadata preservation coming....

  You can preserve metadata. Go to the JUpload configuration page (parameter pictureTransmitMetadata). But you'll have to test if it's ok, with your configuration. With (apparently) recent camera, preserving leads to 'strange' colors. See one of the three head messages in this list (and vote for its correction ! see explanation there).


Etienne

Logged

adzempire

  • Coppermine newbie
  • Offline Offline
  • Posts: 9
Re: Uploading Big Photos in Large Quantities
« Reply #5 on: December 10, 2009, 09:15:35 pm »

USER:
------------------
Array
(
    [ID] => f9da9978139d0594aa491a6d05f594d0
    [am] => 1
    [lang] => english
    [lap] => 3
    [liv] => Array
        (
        )

    [search] => Array
        (
            [search] => speech
            [album] => search
            [plugin_search_album] => true
            [title] => on
            [newer_than] =>
            [caption] => on
            [older_than] =>
            [keywords] => on
            [type] => OR
            [album_title] => on
            [album_description] => on
            [params] => Array
                (
                    [title] => on
                    [caption] => on
                    [keywords] => on
                )

        )

)

==========================
USER DATA:
------------------
Array
(
    [user_id] => 61
    [user_name] => Thorp
    [groups] => Array
        (
           
  • => 1

        )

    [disk_max] => 0
    [disk_min] => 0
    [can_rate_pictures] => 1
    [can_send_ecards] => 1
    [ufc_max] => 3
    [ufc_min] => 3
    [custom_user_upload] => 0
    [num_file_upload] => 5
    [num_URI_upload] => 3
    [can_post_comments] => 1
    [can_upload_pictures] => 1
    [can_create_albums] => 1
    [has_admin_access] => 1
    [pub_upl_need_approval] => 0
    [priv_upl_need_approval] => 0
    [group_name] => Administrators
    [upload_form_config] => 3
    [group_quota] => 0
    [can_see_all_albums] => 1
    [group_id] => 1
)

==========================
Queries:
------------------
Array
(
   
  • => SELECT extension, mime, content, player FROM cpg14x_filetypes; (0s)
  • [1] => select * from cpg14x_plugins order by priority asc; (0s)
        [2] => delete from `pic10`.cpg14x_sessions where time<1260472192 and remember=0; (0s)
        [3] => delete from `pic10`.cpg14x_sessions where time<1259266192; (0s)
        [4] => select user_id from `pic10`.cpg14x_sessions where session_id = '5ea0482526b053e6080b8074ddfcb9bb' (0s)
        [5] => select user_id as id, user_password as password from `pic10`.cpg14x_users where user_id=61 (0s)
        [6] => SELECT u.user_id AS id, u.user_name AS username, u.user_password AS password, u.user_group+100 AS group_id FROM `pic10`.cpg14x_users AS u INNER JOIN `pic10`.cpg14x_usergroups AS g ON u.user_group=g.group_id WHERE u.user_id='61' (0s)
        [7] => SELECT user_group_list FROM `pic10`.cpg14x_users AS u WHERE user_id='61' and user_group_list <> ''; (0s)
        [8] => SELECT MAX(group_quota) as disk_max, MIN(group_quota) as disk_min, MAX(can_rate_pictures) as can_rate_pictures, MAX(can_send_ecards) as can_send_ecards, MAX(upload_form_config) as ufc_max, MIN(upload_form_config) as ufc_min, MAX(custom_user_upload) as custom_user_upload, MAX(num_file_upload) as num_file_upload, MAX(num_URI_upload) as num_URI_upload, MAX(can_post_comments) as can_post_comments, MAX(can_upload_pictures) as can_upload_pictures, MAX(can_create_albums) as can_create_albums, MAX(has_admin_access) as has_admin_access, MIN(pub_upl_need_approval) as pub_upl_need_approval, MIN( priv_upl_need_approval) as  priv_upl_need_approval FROM cpg14x_usergroups WHERE group_id in (1) (0s)
        [9] => SELECT group_name FROM  cpg14x_usergroups WHERE group_id= 1 (0s)
        [10] => update `pic10`.cpg14x_sessions set time='1260475792' where session_id = '5ea0482526b053e6080b8074ddfcb9bb' (0s)
        [11] => SELECT user_favpics FROM cpg14x_favpics WHERE user_id = 61 (0s)
        [12] => DELETE FROM cpg14x_mod_online WHERE last_action < NOW() - INTERVAL 10 MINUTE (0s)
        [13] => REPLACE INTO cpg14x_mod_online (user_id, user_name, user_ip, last_action) VALUES ('61', 'Thorp', '10.127.2.238', NOW()) (0s)
        [14] => DELETE FROM cpg14x_banned WHERE expiry < '2009-12-10 15:09:52' (0s)
        [15] => SELECT * FROM cpg14x_banned WHERE (ip_addr='10.127.2.238' OR ip_addr='10.127.2.238' OR user_id=61) AND brute_force=0 (0s)
        [16] =>  SELECT category, count(1) nb  FROM cpg14x_albums  WHERE category < 10000 OR category = 10061 GROUP BY category  (0s)
        [17] =>  SELECT aid, title, description   FROM cpg14x_albums  WHERE category < 10000 ORDER BY UPPER(title) (0s)
        [18] =>  SELECT aid, title, description  FROM cpg14x_albums  WHERE category='10061'  ORDER BY UPPER(title) (0s)
        [19] => SELECT COUNT(*) FROM cpg14x_pictures WHERE approved = 'NO' (0s)
        [20] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '0'  ORDER BY pos (0s)
        [21] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '1'  ORDER BY pos (0s)
        [22] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '2'  ORDER BY pos (0s)
        [23] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '3'  ORDER BY pos (0s)
        [24] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '4'  ORDER BY pos (0s)
        [25] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '5'  ORDER BY pos (0s)
        [26] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '6'  ORDER BY pos (0s)
        [27] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '7'  ORDER BY pos (0s)
        [28] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '8'  ORDER BY pos (0s)
        [29] =>  SELECT cid, name, description, pos  FROM cpg14x_categories  WHERE parent = '9'  ORDER BY pos (0s)
        [30] =>
                SELECT aid, title, description
                FROM cpg14x_albums
                WHERE category = 10061
                ORDER BY UPPER(title)
                 (0s)
        [31] => SELECT sum(total_filesize) FROM cpg14x_pictures WHERE owner_id = 61 (0s)
    )

    ==========================
    GET :
    ------------------
    Array
    (
       
[file] => jupload/jupload
)

==========================
POST :
------------------
Array
(
)

==========================
VERSION INFO :
------------------
PHP version: 5.2.10-2ubuntu5 - OK
------------------
mySQL version: 5.1.37-1ubuntu5
------------------
Coppermine version: 1.4.25(stable)
==========================
Module: GD
------------------
GD Version: 2.0 or higher
FreeType Support: 1
FreeType Linkage: with freetype
T1Lib Support: 1
GIF Read Support: 1
GIF Create Support: 1
JPG Support: 1
PNG Support: 1
WBMP Support: 1
XPM Support:
XBM Support:
JIS-mapped Japanese Font Support:

==========================
Module: mysql
------------------
MySQL Supportenabled
Active Persistent Links 0
Active Links 1
Client API version 5.1.37
MYSQL_MODULE_TYPE external
MYSQL_SOCKET /var/run/mysqld/mysqld.sock
MYSQL_INCLUDE -I/usr/include/mysql
MYSQL_LIBS -L/usr/lib -lmysqlclient_r 
==========================
Module: zlib
------------------
ZLib Support enabled
Stream Wrapper support compress.zlib://
Stream Filter support zlib.inflate, zlib.deflate
Compiled Version 1.2.3.3
Linked Version 1.2.3.3
==========================
Server restrictions (safe mode)?
------------------
Directive | Local Value | Master Value
safe_mode | Off | Off
safe_mode_exec_dir | no value | no value
safe_mode_gid | Off | Off
safe_mode_include_dir | no value | no value
safe_mode_exec_dir | no value | no value
sql.safe_mode | Off | Off
disable_functions | no value | no value
file_uploads | On | On
include_path | .:/usr/share/php:/usr/share/pear | .:/usr/share/php:/usr/share/pear
open_basedir | no value | no value
==========================
email
------------------
Directive | Local Value | Master Value
sendmail_from | no value | no value
sendmail_path | /usr/sbin/sendmail -t -i  | /usr/sbin/sendmail -t -i
SMTP | localhost | localhost
smtp_port | 25 | 25
==========================
Size and Time
------------------
Directive | Local Value | Master Value
max_execution_time | 100000 | 100000
max_input_time | 10000 | 10000
upload_max_filesize | 1000M | 1000M
post_max_size | 8M | 8M
==========================
Page generated in 0.061 seconds - 32 queries in 0 seconds - Album set : ; Meta set: ;
Logged

etienne_sf

  • Moderator
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 908
    • Wiki
Re: Uploading Big Photos in Large Quantities
« Reply #6 on: December 11, 2009, 09:01:00 am »

adzempire,

  What's this message about ???

Is this linked to this (old) thread ?

Etienne
Logged

emperoradz

  • Coppermine newbie
  • Offline Offline
  • Posts: 3
Re: Uploading Big Photos in Large Quantities
« Reply #7 on: December 11, 2009, 05:57:34 pm »

yup it is

The big photo error when uploading > 100ish files
Logged

etienne_sf

  • Moderator
  • Coppermine addict
  • ****
  • Offline Offline
  • Gender: Male
  • Posts: 908
    • Wiki
Re: Uploading Big Photos in Large Quantities
« Reply #8 on: December 14, 2009, 11:42:50 am »

Hi,

  Ok. So what I need is the JUpload debug. Can you read the doc here, for an howto on this point :
http://forum.coppermine-gallery.net/index.php/topic,53168.0.html

Etienne
Logged
Pages: [1]   Go Up
 

Page created in 0.058 seconds with 19 queries.