AppGini Community Forums

First off, thanks as always to Ahmad and everyone in this community for an excellent product. While it has suffered through some unstable versions over the years, I've yet to see any other product that delivers such value.

What follows is not an AppGini problem per se, but would expand the power of AppGini to more large scale deployments.

An application I have been running for a client features a table with each record pointing to several image files. Users are able to upload image files using the image fields, and as of now there are about 55,000 image files pertaining to records in the table. (Think insurance claim records with images of damaged auto parts.)

Recently I had to migrate the database and all files to a new server using FTP. As far as the application is concerned a Linux directory has no practical limit to the number of image or other files that can be in it. (2^32 or approximately 4.3 billion files or "inodes") Nor is there any significant performance issue accessing the files when a record is accessed for now anyway. Shared hosting FTP servers do typically limit the number of displayable files to 10000 by the FTP server LimitRecursion setting which isnt always changeable. Having a virtual private server typically allows setting LimitRecursion to a higher value, but understandably not so in shared hosting environments.

My current workaround involves deleting moved files from the origin server so the not yet moved ones become visible to the FTP server in batches of 10000 files. If it becomes a run time performance issue I will probably develop hooks to manage the directories for saving/deleting files.

What would be great is if one could specify which directories AppGini saves files to, through some configuration option enabling multiple directories. For example it could be that each file goes to a directory named after the first one or two initials in the file name, or some such. That would not only make managing the files using FTP more feasible, it would also allow for performance tuning in large scale deployments.

Again this isn't an AppGini problem per se, but a concern for larger deployments. Just putting it out there.
Thanks.

Thanks for your contribution!

A customer of mine once told me about similar experiences having > 10000 files per directory. So in that customer project we split files into subdirectories.

Possible workaround

In a project, handling product pictures, after_insert we move images into in a separate subdirectory per master-record. The subdirectory will be created automatically, if not exists. This is not too complicated:

(Code not tested)

• get id of master-record, for example 123, using SQL:
  $parent_id = sqlValue("SELECT contract_id FROM attachments WHERE id=''{$selectedID}");
• create subdirectory "/images/contract-123", if not exists
• change permissions,if required
• get filename of child-record, like so: $filename = sqlValue("SELECT file FROM attachments WHERE id=''{$selectedID}");
• move file from /images/ into /images/contract-123/, if exists
• on success, change filename in database: sql("UPDATE attachments SET file='/images/contract-123/...your...filename.jpg' WHERE id='{$selectedID}'", $eo);

Notes

• If necessary, protect the new subdirectories against direct access
• If the files are images, don't forget to move (automatically generated) thumbnails for tv and dv, too, if exist

Indeed your solution of using some unique information from master record to generate subdirectory name is elegant.
I thought about also saving the subdirectory name in the master record, but a subdirectory name generation rule like you suggest seems stronger. For now it isn't a high priority need but I think in time we may need to implement something like you suggested.
Thanks!