CyberLink Community Forum
where the experts meet
| Advanced Search >
PD17 out-of-memory crash with Pixelan + NBFX
Reply to this topic
optodata
Senior Contributor Private Message Location: California, USA Joined: Sep 16, 2011 16:04 Messages: 4227 Online
[Post New]
I don't know how many people have both Pixelan's DissolveMaster and NewBlue FX's Essential Effects 6 packs installed, but I've run into a serious bug that uses up all available memory when producing to 4K, which then crashes.

I made a screen recording using a hardware screen capture device since any recording being made with Screen Recorder2 gets irreparably damaged when PD crashes.



There's no annotation, but you can easily see the memory usage get out of hand as PD works through the transition. Even with the effects being applied to four 4K clips at once, I can't imagine that would take an additional 10GB+ of RAM for 1 second of timeline processing. Certainly there should be some bounds checking to prevent situation like this.

Although I used the best match from the Profile Analyzer, I believe any 4K profile will cause the crash, and it doesn't matter if you use QuickSync, nVidia or CPU producing. Even SVRT crashes when trying to process this transition

The packed project is here, and it would be great if anyone else who has these two add-ins could give this a try. I used a very short 4K clip for this test project, so it's only a 60MB download.
youtube/optodata

DS365 | Win10 Pro | 8 Core/16T i9-9900K (4.8GHz) | RTX 2070 | 32GB RAM | 6TB SSDs

Canon Vixia GX10 (4K 60p) | HF G30 (HD 60p) | HF S200 (HD 30p) | Yi Action+ 4K | 360Fly 4K 360°
Reply
tomasc [Avatar]
Senior Contributor Private Message Joined: Aug 25, 2011 12:33 Messages: 4764 Offline
[Post New]
Quote There's no annotation, but you can easily see the memory usage get out of hand as PD works through the transition. Even with the effects being applied to four 4K clips at once, I can't imagine that would take an additional 10GB+ of RAM for 1 second of timeline processing.


Some thoughts here. Many users don’t have a swap file because they believe it is not needed when using a SSD. I have never run out of memory with a permanent swap file. There may not need to add an additional 10 GB+ of ram. Could you say add a 100 GB permanent swap file to the spinning hard disk and try this 2 minute test you did shown on YT again.

It may help not to have AVG, Opera, Edge and other apps running and be disconnected from the internet for this 2 min. test.

This message was edited 2 times. Last update was at Apr 21. 2019 16:39

Reply
optodata
Senior Contributor Private Message Location: California, USA Joined: Sep 16, 2011 16:04 Messages: 4227 Online
[Post New]
Quote
Some thoughts here. Many users don’t have a swap file because they believe it is not needed when using a SSD. I have never run out of memory with a permanent swap file. There may not need to add an additional 10 GB+ of ram. Could you say add a 100 GB permanent swap file to the spinning hard disk and try this 2 minute test you did shown on YT again.

It may help not to have AVG, Opera, Edge and other apps running and be disconnected from the internet for this 2 min. test.

I have a 12GB RAM disk for temp file storage, and a 5GB swap file on my C: SSD. Running different apps obviously affects how much RAM is available, and in my tests it simply changed the point where the crash happen (as in sooner if more RAM was already in use).

I moved the swap file to a bigger drive and increased it, but only to 20-30GB because it's where Windows swaps out RAM contents, and I "only" have 20GB besides my RAMDisk. That RAMDisk also means even with nothing else running, my RAM usage is at 38%.

Producing again with the larger swap file maxxed out my RAM at just under 12GB and PD didn't crash, so increasing the swap file does solve the immediate issue. Thank you for suggesting that!

I'm still puzzled by the sheer numbers here though: considering that PD's baseline RAM usage is under 1GB with the editor open, that means to produce this single second of video it actually required the amount of RAM shown in Task Manager (12GB) plus my original swap file size (5GB) plus some additional amount of my new 20GB swap file.

The sum of the first two (17GB) wasn't enough to prevent a crash, but somewhere between 18 and 37GB was enough.

There are a total of 240 4K frames involved in that transition, and if they are all fully decompressed and sitting in RAM at the same time, they require less than 1GB (a MagicYUV conversion of the test clip has a bit rate of 1,862Mb/s which equals 233MB of data for 60 frames; 4 clips worth = 931MB), so processing this combination of transitions seems to require somewhere around 20x overhead.

Maybe that's perfectly normal, but it seems excessive to me - without any deep insight to what's actually happening.

It also shows that PD isn't managing memory usage here at all. It's great that Windows can swap out data fast enough to make a difference, but PD seems to just kept demanding more without verifying that there is actually enough space left to proceed.

Maybe if you're building a simple app that couldn't use 1GB of RAM if you ran 10 copies at once, memory management is unnecessary, but with such a demanding task as video producing, it sure seems like PD should be more aware of the system it's running on.

This message was edited 2 times. Last update was at Apr 21. 2019 19:18


youtube/optodata

DS365 | Win10 Pro | 8 Core/16T i9-9900K (4.8GHz) | RTX 2070 | 32GB RAM | 6TB SSDs

Canon Vixia GX10 (4K 60p) | HF G30 (HD 60p) | HF S200 (HD 30p) | Yi Action+ 4K | 360Fly 4K 360°
Reply
optodata
Senior Contributor Private Message Location: California, USA Joined: Sep 16, 2011 16:04 Messages: 4227 Online
[Post New]
I have reported this issue on ticket number CS002014972.

Although tomasc has identified a workaround, PD should not be causing an out of memory condition when running normally. On a system with 4GB or RAM, it has to work within that amount of memory, even if it means processing/producing the clip more slowly.

I lost 3 days on this issue because my uber machine still needed 6-7 hours to produce a 40 minute 4K video, so I had to run it overnight. The crash occurred near the halfway point so it always occurred when I was asleep. I had used nVidia HW producing the first night, then I used QuickSync and the final time was CPU only, but the crash kept happening.

It was only when I discovered that it was linked to a transition that I was able to confirm the bug, and hopefully Cyberlink will be able to update the program to prevent at least this specific out-of-memory crash in the future.
youtube/optodata

DS365 | Win10 Pro | 8 Core/16T i9-9900K (4.8GHz) | RTX 2070 | 32GB RAM | 6TB SSDs

Canon Vixia GX10 (4K 60p) | HF G30 (HD 60p) | HF S200 (HD 30p) | Yi Action+ 4K | 360Fly 4K 360°
Reply
Reply to this topic
Powered by JForum 2.1.8 © JForum Team