Nextcloud sync small files
-
A friend of mine uses NC to move data from remote users to a shared folder. Works fine. That folder syncs to a local folder on another file server via the sync client. The files are very small and lots of them. The sync client struggles due to have to make mysql database entries for each file (an assumption on my part).
The local server has a piece of custom software that looks at the folder and creates reports. Would mapping the folder via webdav be a better solution than having the data copied to the local drive? Is there a better option? Nextcloud has solved a lot of issues they were facing. This is an underlying issue. Thoughts?
-
MySQL or MariaDB entries take like a millisecond, literally. It's so blinding fast. It's the same kind of transaction that has to happen when a filesystem writes where a file is. It's tiny and effectively instantaneous. It would be pretty while if the database writing that a file was being stored could be measured in any way through performance, as it would happen at the same time that the file is being transferred.
Compare it to driving a load of corn from New York to California. And while doing so, needed to make a not in a log book that says "Shipped Corn to California". The one action, the actual moving of the corn, takes like 40 hours. The logging in the book happens in like 15 seconds. And you can make the log entry anytime during the 40 hour process and have it happen simultaneously for zero performance impact on the shipment process.
