Moving the albums/ folder outside the CPG document root

[slausen]

Hi-

I would like to move the 'albums/' directory containing all of the photo and video content for a CPG installation outside of the CPG document root to allow for more flexibility in storing the ever growing amount of content managed by CPG. In the short term, this means moving the 'albums/' directory onto a separate disk partition. In the future, it may mean moving the content to a separate server, and mounting it via NFS.

As long as I change the value for 'fullpath' in the config table from 'albums/' to '/new/path/albums/' should CPG and its associated plugins still work?

Thanks.

[steveeh131047]

Does this earlier thread answer the question?
http://forum.coppermine-gallery.net/index.php/topic,41536.0.html

[slausen]

Does this earlier thread answer the question?
http://forum.coppermine-gallery.net/index.php/topic,41536.0.html

Partly, thanks a lot for that reference (I did a search but didn't find that thread).

But I am also a little confused because it seems like the starter of that thread was looking at a different configuration setting, since he asked for the location of where in the DB he should make changes. I was looking at the config table in the database. Is the 'fullpath' setting the only thing that needs to be changed (along with moving the files to the new location)? Or is there another configuration setting that needs to be modified?

[steveeh131047]

If it gives you a bit more confidence - I just moved my albums directory on a local development set-up and changed the Config setting to point to it. Everything I tried worked just fine, although I didn't check any plug-in functions.

You shouldn't need to change the "userpics" path setting - it's a "relative path" and will stay the same if the folder moves as part of your albums directory.

[steveeh131047]

But I am also a little confused because it seems like the starter of that thread was looking at a different configuration setting, since he asked for the location of where in the DB he should make changes.

All the image references I could see in my db looked "relative" to the albums url - I didn't see anything that looked "absolute"

[steveeh131047]

Just to clarify - I wasn't suggesting changing the albums path directly in the database, I was working through "Config" > "Files and thumbnails advanced settings"

[slausen]

Thanks steveeh131047!! It looks like it will work!

[slausen]

I may have spoken too soon...

I was about to implement this, but as I took a quick look at the help for that config setting, and this is what it says:

This is the base directory for your "Image Store". The path is relative to the main directory of the script.

You can use ../ in the path to move-up one level in the directory tree.

You can not use an absolute path there ("/var/my_images/" will not work) and the album directory must be visible by your web server.

This setting mustn't be changed if you already have files in your database. If you do so, all references to your existing files will break.

I would actually like to reference a file store that is outside of the CPG docroot, like this:

Code:

/content/cpg_files/albums <-- planned new location of 'albums/' directory on separate partition

Code:

/home/coppermine/cpg144/albums <-- current location of 'albums/' directory

Also, I have an alias in my httpd.conf:

Code:

Alias /imagecentral /home/coppermine/cpg144 <-- maps 'http://www.companyserver.com/imagecentral' to the CPG docroot

So I guess I am wondering how CPG is interpreting the value for "The album directory" inside the "Files and Thumbnails Advanced Settings". It seems from what's in the help that it is a relative path that will be used to create the URL string for the display of the underlying content within CPG, but in the database it is labeled 'fullpath' in the config table. Or is the database setting something different from the setting that is changeable in the Admin UI? That would make sense - I have seen in other scripts where they store two "path" values; one for the URL "path" and one for the actual "path" to the files on the filesystem.

I can create a new alias in my httpd.conf that will map http requests to the new file location, but will CPG be able to deal with that when it comes to doing things like creating thumbnails, or browsing the file system, or handling content uploads?

Example of new alias I could create:

Code:

Alias /imagecentral/newstorage /content/cpg_files <-- maps 'http://www.companyserver.com/imagecentral/newstorage' to the new location of the 'albums/' directory

But then CPG would be trying to write to a directory that doesn't actually exist in the filesystem, right? I guess I am wondering what is the best way to do this? I am thinking that probably others have run across this issue as their galleries have gotten large. What are the best practices for distributing large sets of gallery data across disks/servers?

[steveeh131047]

Everything I've done suggests the "Help" for this Config setting is correct. It's a "relative" path referenced to the cpg root. Whatever I type in this field through the Config screen appears exactly the same in the db. I don't think the label "full path" in the db should be interpreted as "absolute path".

You can certainly move the albums folder outside the cpg root - that's what I did when I experimented yesterday. I temporarily had it 2 levels above the cpg root and entered "../../albums/"  into the Config field.

Not sure what happens if you put the albums folder on a different disk/server - that's way outside my area of knowledge 

[slausen]

Hmm...

So it looks like this cannot be done the way I envision? Anyone?

[Nibbler]

Use a symlink. Search for a previous post by me containing the word 'symlink' if you need example commands.

[slausen]

Use a symlink. Search for a previous post by me containing the word 'symlink' if you need example commands.

Thanks for the suggestion, I had thought of that 

But I would prefer not to use a symlink, since (I believe) that would cause the OS to have to do extra work for every image/thumb request.

[Nibbler]

Negligible.

[slausen]

Negligible.

With respect, Nibbler, I would say that depends upon the structure of your CPG galleries, your server configuration, as well as the usage pattern (and number) of your users...

FYI: http://www.xs4all.nl/~thomas/apachecon/PerformanceTuning.html

"The symlink checking requires an extra filesystem lookup (via the lstat() systemcall) for each path element in the location being opened, resulting in several such lookups per request. None of these lookups are cached by Apache. Filesystem lookups can be relatively slow, and when using a remote file storage such as an NFS server, can be quite taxing for the NFS server as well as the webserver. A busy webserver with several deep document trees and symlink-checking enabled can easily bring down a heavier NFS server by the number of lstat() calls and their network traffic. It is possible to enable symlink checks only for specific directories (and their subdirectories), disabling it for safe paths, but with deep directory trees this can still be a burden."

[Nibbler]

That's talking about symlink checking. You don't need to enable symlink checking because you know it is safe (you created it). If your gallery is noticeably slower with the symlink I would be very surprised.

[slausen]

That's talking about symlink checking. You don't need to enable symlink checking because you know it is safe (you created it). If your gallery is noticeably slower with the symlink I would be very surprised.

Wouldn't I have to turn off symlink checking for everything in the CPG root?

And anyway, that was just the impact from the apache perspective, the key is that for every file requested via a symlink the OS has to do extra work. If you are talking about a binary that is just loaded into memory once, you can take the hit, but if you have hundreds of users browsing a gallery, and for every page request you are doing 40 or 50 lookups, it's not a path I want to go down...

http://www.virtualblueness.net/Ext2fs-overview/Ext2fs-overview-0.1-10.html

"Symbolic link, on the other hand, is analyzed at run time. A symbolic link is just a pathname which is accessible from an inode. As such, it "speaks" in the language of the abstract filesystem. When the kernel reaches a symbolic link, it will follow it in run time using its normal way of reaching directories.

As such, symbolic link can be made across different filesystems and a replacement of a file with a new version will automatically be active on all its symbolic links.

The disadvantage is that hard link doesn't consume space except to a small directory entry. Symbolic link, on the other hand, consumes at least an inode, and can also consume one block.

When the inode is identified as a symbolic link, the kernel needs to find the path to which it points."

[slausen]

I made a feature request for this: http://forum.coppermine-gallery.net/index.php/topic,51694.0.html

If anyone else is interested in having it added to CPG, please add a comment to let the dev team know! 

[Nibbler]

An alternative would be to mount the partition as /content/cpg_files/albums directly.