Announcement: Our new CyberLink Feedback Forum has arrived! Please transfer to our new forum to provide your feedback or to start a new discussion. The content on this CyberLink Community forum is now read only, but will continue to be available as a user resource. Thanks!
CyberLink Community Forum
where the experts meet
| Advanced Search >
Import folder from NAS
robert@ [Avatar]
Newbie Joined: Feb 21, 2020 08:32 Messages: 8 Offline
[Post New]
The import of folders from my NAS is exteremely time consuming as soon as the number of pictures to be imported is let's say about 2000 or more. Waiting for 1 or 2 hours is normal. Is it really? Especially in the situation to avoid double import, the program hangs.
I also made a test trying to import around 11000 files via folder import is a nightmare. After running the import process for more than 12 hours, I stopped the program via the only way: killing by task manager.

Did anybody made the same experience? Would it be necessary to limit the number of files in a folder for an import from the NAS?

Thank's for any help.
optodata
Senior Contributor Location: California, USA Joined: Sep 16, 2011 16:04 Messages: 8630 Offline
[Post New]
Make sure that you have a drive letter mapped to your NAS. See this post and this long thread for reasons why.

YouTube/optodata


DS365 | Win11 Pro | Ryzen 9 3950X | RTX 4070 Ti | 32GB RAM | 10TB SSDs | 5K+4K HDR monitors

Canon Vixia GX10 (4K 60p) | HF G30 (HD 60p) | Yi Action+ 4K | 360Fly 4K 360°
robert@ [Avatar]
Newbie Joined: Feb 21, 2020 08:32 Messages: 8 Offline
[Post New]
Quote Make sure that you have a drive letter mapped to your NAS. See this post and this long thread for reasons why.


The drive was mapped, that's not my issue. Anyway, I wonder how it would be possible to import folders or pictures from my NAS without a mapped drive. In order to receive more information of what's going on during the import, I opened the task manager and followed the details of PhD11 during the process. As long as the number of pictures to import is low, around 300, everything works fine. But beyond several 1000's it becomes so slow. (in a normal case, the network speed was around 10 to 20 MBits/s)
In order to get the pictures imported, I didn't import the folder, but as "Picture" in several bunches of pictures (200 at one time) from the folder. That tooks some time, but it was quicker than importing the complete folder.
Additionally, If I import the folder, then decide to avoid double pictures, the PhD11 had even more problems, time was running, but nothing happened. Again, in the task manager, the network speed dropped to 0,1 MBit/s and the PhD11 did a "not respond" on a regular basis.
For me it seems, that the database management has some problems when dealing with larger numbers of files via a mapped drive.
On the other side, importing folders with larger number of pictures (>4000) from my computer did not create problems.
tomasc [Avatar]
Senior Contributor Joined: Aug 25, 2011 12:33 Messages: 6464 Offline
[Post New]
Quote


The drive was mapped, that's not my issue. Anyway, I wonder how it would be possible to import folders or pictures from my NAS without a mapped drive. In order to receive more information of what's going on during the import, I opened the task manager and followed the details of PhD11 during the process. As long as the number of pictures to import is low, around 300, everything works fine. But beyond several 1000's it becomes so slow. (in a normal case, the network speed was around 10 to 20 MBits/s)
In order to get the pictures imported, I didn't import the folder, but as "Picture" in several bunches of pictures (200 at one time) from the folder. That tooks some time, but it was quicker than importing the complete folder.
Additionally, If I import the folder, then decide to avoid double pictures, the PhD11 had even more problems, time was running, but nothing happened. Again, in the task manager, the network speed dropped to 0,1 MBit/s and the PhD11 did a "not respond" on a regular basis.
For me it seems, that the database management has some problems when dealing with larger numbers of files via a mapped drive.
On the other side, importing folders with larger number of pictures (>4000) from my computer did not create problems.

This is interesting. Try this experiment. Copy one folder with the 1000+ photos to a local drive and time it. Open PhotoDirector and import from that local folder. This will determine if the problem is in importing from the nas drive. Let us know if this helps.
robert@ [Avatar]
Newbie Joined: Feb 21, 2020 08:32 Messages: 8 Offline
[Post New]
Quote

This is interesting. Try this experiment. Copy one folder with the 1000+ photos to a local drive and time it. Open PhotoDirector and import from that local folder. This will determine if the problem is in importing from the nas drive. Let us know if this helps.

--------------------------------------
Results of the test run:
Number of files copied from NAS lo local picture folder: 4636 (35 folders)
time: 54 min 37 seconds
speed: varied around 3.5 to 4 MByte/s ( around 30MBits/s) controlled via task manager
during a short period of time (about 1 min) the transmission speed did rise to 20 MBytes/s
apps running: windows explorer and task manager

started PhD11
choosed import of the files copied before from NAS
Start import, detection time for all pics (4626, 10 files were not allowed) time: around 10 minutes
PhD showed very often "not responding" in the task manager,

start import with no duplicate detection
Time for the import process : 3 min 52 sec

Background apps: several, including Kaspersky Total security
CPU: never too high

Hope this helps to get the issue solved, Thank's in advance anyway
tomasc [Avatar]
Senior Contributor Joined: Aug 25, 2011 12:33 Messages: 6464 Offline
[Post New]
Thank you for running the timed tests. A straight transfer from the nas drive to local drive of 4636 images took 54.62 minutes. Importing the same images located on the local hard drive took 3.87 minutes. 54.62/3.87 = 14.11 X faster. This would mean to me that the slow interaction with the nas drive with duplicate detection is using up all available resources over the network, extending the import to more than the 12 hours.

It would be far better to copy the images to the local drive first and work with them on the local drive. Each image import took only .05 seconds. That is 20 image per second imported from the local drive with duplicate detection active.
robert@ [Avatar]
Newbie Joined: Feb 21, 2020 08:32 Messages: 8 Offline
[Post New]
Quote Thank you for running the timed tests. A straight transfer from the nas drive to local drive of 4636 images took 54.62 minutes. Importing the same images located on the local hard drive took 3.87 minutes. 54.62/3.87 = 14.11 X faster. This would mean to me that the slow interaction with the nas drive with duplicate detection is using up all available resources over the network, extending the import to more than the 12 hours.

It would be far better to copy the images to the local drive first and work with them on the local drive. Each image import took only .05 seconds. That is 20 image per second imported from the local drive with duplicate detection active.


Thank's for your advice, but I use the NAS to free space on my Laptop. I have around 65k images on my NAS. The aim is to have an overview of all pics on different supports. Copying the images from a certain folder each time from the NAS onto my Laptop to be able to work with them on PhD is not a solution.

A couple of problems appear during import from the NAS to the Laptop:
transfer speed from NAS to Laptop with windows explorer is double a s high as with PhD11 ( 20Mbit/s versus 8 MBit/s)
transfer speed of data with "avoid duplicates" becomes exteremely low when the number of treated pics is about >1000 (0,1 Mbit/s)
Time to achieve an import with 400 pics including "avoid duplicates" is done in an acceptable time.

In all cases, PhD11 is very often in "no response" (shown up in Task manager)

That's why I think tat the problem lies in the programming of the database management, which has obviously problems to treat high numbers of pix with a mapped drive, especially when choosing "no duplicates".
I could well imagine a next update which would solve the problem.
With Lightroom, this was never a problem.
Powered by JForum 2.1.8 © JForum Team