Announcement: Our new CyberLink Feedback Forum has arrived! Please transfer to our new forum to provide your feedback or to start a new discussion. The content on this CyberLink Community forum is now read only, but will continue to be available as a user resource. Thanks!
CyberLink Community Forum
where the experts meet
| Advanced Search >
Curious Thing - Shadow Files
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
Hi,

Lately I've taken to use H.264 AVC(MP4) to render sequences for a video I'm putting together. I figure MP4 to be a lossless file type(meaning I don't expect it to degrade after a couple of renderings). The camera I currently use is a Canon Legria HF R506, which shoots video in either AVCHD or MP4(selectable between either), so I've selected MP4. When I import the clips to my computer for editing, those clips show up in my "media library" with shadow files still generating(corner pip going from yellow to green). My problem is not with the shadow files, in fact, I find I can pull the clip onto my timeline and trim it down as needed even while the shadow file is still generating.

It's when I've created my sequences of roughly 5-to-10-minute lengths and rendered them as MP4(H.264 AVC) that I note there has been no shadow file created along with them. I chose MP4 as the rendering time is about the same as if I'd rendered at MPEG2.

I've taken to using MP4 to render sequences(segments, if you will) because sometimes they may have PiP superimposed video showing a scene from a slightly different angle. This thing with shadow files has sparked my curiosity since I started using that Canon camera. As I said earlier, the shadow file creation doesn't impede my editing work, I'm just curious as to why MP4 video I've created does not have shadow files attached.

Cheers!

Neil.

Edit: Content removed by Moderator. This is an open forum where all members are allowed to add and participate, discriminatory "riders" are unacceptable. Moderator.

This message was edited 1 time. Last update was at Jun 23. 2016 03:46

BarryTheCrab
Senior Contributor Location: USA Joined: Nov 06, 2008 22:18 Messages: 6240 Offline
[Post New]
Shadow files are created only for HD video files, so if you are rendering to SD quality then no shadow file is created. I just did a quick MP4 test to be sure. I made an HD MP4 and immediately the shadow files started generating.
If you are creating an HD file and no "proxys" start to generate, that is an oddity. Check your output quality, SD or HD?

edit: To be clear I am referring to newly rendered files having been auto-imported into the Library and whether they begin to show shadow generation has started.

This message was edited 1 time. Last update was at Jun 23. 2016 10:56

HP Envy Phoenix/4thGen i7-4770(4@3.4GHz~turbo>3.9)
Nvidia GTX 960(4GB)/16GB DDR3/
Canon Vixia HV30/HF-M40/HF-M41/HF-G20/Olympus E-PL5.
Tape capture using 6 VCR, TBC-1000, Elite BVP4+, Sony D8 camcorder with TBC.
https://www.facebook.com/BarryAFTT
[Post New]
Quote: Hi,

Lately I've taken to use H.264 AVC(MP4) to render sequences for a video I'm putting together. I figure MP4 to be a lossless file type(meaning I don't expect it to degrade after a couple of renderings). The camera I currently use is a Canon Legria HF R506, which shoots video in either AVCHD or MP4(selectable between either), so I've selected MP4. When I import the clips to my computer for editing, those clips show up in my "media library" with shadow files still generating(corner pip going from yellow to green). My problem is not with the shadow files, in fact, I find I can pull the clip onto my timeline and trim it down as needed even while the shadow file is still generating.




Shadow files are temporary, stored in hidden folder.

They are deleted automatically after 30 days or manually in PD configurations.

For the best quality Produce use Intelligent SVRT if possible. AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
To Playsound and Barry The Crab!

My comment was not so much seeking a solution to a problem, more like making a comment, an observation on the nature of these "shadow files". As to whether MP4(H.264 AVC) is a lossless file type, I just wanted to know if that was correct, it would justify my using the format more as an intermediary, when I have a lot of clips and it looks like the resultant video may total 90-or-more minutes in length, then I'll break it down to shorter sequences and join them all up later. No need for PDS files, just the sequences, numbered or alphabetised to keep them in the order I want them in their own folder.

As to my rider which was deleted, I had my reasons for that but they are personal and I won't go into them here.

Neil.
BarryTheCrab
Senior Contributor Location: USA Joined: Nov 06, 2008 22:18 Messages: 6240 Offline
[Post New]
MP4 is a lossy format. HP Envy Phoenix/4thGen i7-4770(4@3.4GHz~turbo>3.9)
Nvidia GTX 960(4GB)/16GB DDR3/
Canon Vixia HV30/HF-M40/HF-M41/HF-G20/Olympus E-PL5.
Tape capture using 6 VCR, TBC-1000, Elite BVP4+, Sony D8 camcorder with TBC.
https://www.facebook.com/BarryAFTT
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
Quote: MP4 is a lossy format.


Thanks, Barry!

Okay, I take that on board, but I'll comment that the loss in quality seems negligable(hardly noticable) compared to say, re-rendering from MPEG2 to MPEG2 again, there seems to be a noticable degree of degradation there, but MP4 to MP4? not quite as noticable. I can live with that as likely I'm only going one generation away from the original raw video clips in their(HD with shadow files) state, sucked in from the camera's SD card.

Cheers!

Neil.
AlS
Senior Member Location: South Africa Joined: Sep 23, 2014 18:07 Messages: 290 Offline
[Post New]
I've often wondered if shadow files are used for proxy editing (Proxy Video Editing is a process where you create temp lower quality Standard Definition copies of your original HD videos in the timeline for speed. Edits are then applied to original clips at render time) and if so, how does it affect the preview quality we see? Are we looking at a lower quality "shadow" version?

Neil - Assuming your editing of the MP4 clips prevents you from taking advantage SVRT (no render), the alternative to proxy is to transcode your camera MP4 to a lossless, edit friendly, intermediate codec. All professional NLEs support this and Ron (Optodata) created a front-end for PDR with the M+PD project. Your highly compressed MP4 files are decompressed and every frame is a keyframe for more accurate and better quality edits, effects, color correction etc. GoPro Studio automatically converts MP4 to the Cineform codec (movie industry standard) before edit. It gets worse in 4k and Samsung have just brought out a 4k camera that records in H.265. These highly compressed H.264/265 camera files save space but are tough to edit in the timeline. That's why many pro cameras record uncompressed (huge) files or in MOV/AVI using an intermediate codec like Cineform for pro editing. Maybe M+PD might be worth a try. I can edit 4k on my i5 CPU and no discrete GPU card using M+PD - but it chokes and dies without it. PDR tries to be a complete edit studio in one program. Adobe P-Pro edits but the fancy bits are done separately using After Effects. To avoid quality loss from taking an edited MP4 project into After Effects and outputting to MP4 again, P-Pro outputs the project as a lossless intermediate codec to After Effects where it can be rendered to MP4. It's a post production workflow.

Hope I haven't bored you!!

This message was edited 2 times. Last update was at Jun 29. 2016 06:48

Power Director 13&14 Ultimate, Photo Director 6, Audio Dir, Pwr2Go 10
Win 10 64, Intel MB DH87MC, Intel i5-4670 CPU @ 3.40GHz, 16Gb DDR3 1600, 128Gb SSD, 2x1Tb WDBlue 7200rpmSATA6, Intel 4600 GPU, Gigabyte G1 GTX960 4GB, LG BluRay Writer
The Shadowman
Senior Contributor Location: UK Joined: Dec 15, 2014 13:06 Messages: 1831 Offline
[Post New]
Neil

Give Magic+PD a try. You have absolutely nothing to lose.

You will be pleasantly surprised by the lossless quality of the conversion, but above all the speed at which it works. Our tests showed the M+PD was eight times faster than shadow files.

http://forum.cyberlink.com/forum/posts/list/46177.page

Robert Panny TM10, GH2, GH4,
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
Hello AIS & Shadowman!

Firstly AIS(Al), Sorry, but a lot of what you said went completely over my head. As for SVRT, I find myself avoiding it of late as when I've used it in the past, it seemed to cut a portion out of the overall running time of my videos(only a few frames, but enough to render the result as less than the length of the video before rendering, this I select the option to view the clip as it renders, that way I know if I've missed any transitions and can cancel the render in order to make my corrections).

Now, Shadowman(Robert), I'm not sure if Magic+PD is still available. Optodata has not been seen on this forum for quite some time either. If it is, I'll think about it but I'm wary about patches applied to Power Director after an experience I had some years back with PD7. After applying an update patch to PD7 it kept crashing on the most minor of tasks. Uninstalling PD7, then reinstaling raw from the disc and PD7 was back to full operation without the crashes. So that has made me very wary of update patches. I did apply the one-and-only patch available to PD8 when I stepped up to that version, and had no problems, but I put that down purely to good luck more than anything else. I'm especially wary about aplying any patch to PD14 lest I have to go through the whole "rigmarole" of reregister, or reactivate when I reinstall after having to uninstall due to problems caused by a patch of any kind. PD14 is nowhere near as straight-forward to install as was PD7 or PD8. With those two versions you just loaded the disc to the drive, let the disc unload its "bag of goodies" onto your computer, When it was all done you just opened the programme, filled in your details and the CDkey and that was it! Power Director was up and running! Now though(and I suspect this has applied since PD9) the internet has been required to complete the activation and the CDkey is no longer enough to activate the product, a massive retrograde step, in my view. I don't want to have to go through all that again as if I was installing PD14 for the first time. Should Magic+PD cause problems, that's what I may face in attempting a repair by reinstalling. There you have it. You might think my fears are groundless but I don't. "once bitten....!"

Neil.
MattC [Avatar]
Member Joined: Jun 03, 2016 12:12 Messages: 102 Offline
[Post New]
Does anyone know what the resolution, etc. of the shadow file is? I import 1920 x 1080 into my projects and the shadow files produced are the same file size, which could indicate either a similar frame size or a lower compression rate. My computer is bottom end of the range so, even when PDR is using the shadow file it is still slow. What would be good is if you could choose the frame size/format etc. for the shadow file. For me, this would be a low res one that PDR would find quicker to render.

Matt .
PDR14; Win10-x64; i7-6700; 3.4GHz; 16GB RAM; plenty of HDD space; AMD Radeon R5 330.
ynotfish
Senior Contributor Location: N.S.W. Australia Joined: May 08, 2009 02:06 Messages: 9977 Offline
[Post New]
Hi Matt -

If you have shadow files enabled, whatever HD videos are imported will have 720x480 MPEG-2 files generated. They'll be stored in your export folder in a "ShadowEditFiles" sub-folder.

It doesn't matter if the source videos are 720p, 1080p, 1440p, UHD, 4K or 8K shadow files will always be 720x480. Resolution and bitrate are scaled down to DVD HQ quality, but frame rate is retained. i.e. if you import a 60fps clip, the shadow file will also be 60fps.

><(((º> ~~~~~~~~~~~~~~~~~~~~~~~~~~ ><(((º> (edited after advice from my artistic director)

Neil -

You could have saved yourself a whole rambling paragraph if you'd taken the time to read some of the information provided in the Magic+PD thread.

It has NOTHING to do with patching PDR & its installation and use does not change PDR in any way.

We're fortunate that Ron (optodata) and his team did all the legwork testing and preparing all that valuable information. Ron particularly went to a great deal of trouble to make sure it was presented in a clear way, with documents and video tutorials.

Why not take the time to read/view some of the information provided?

Cheers - Tony

This message was edited 1 time. Last update was at Jun 30. 2016 17:05


Visit PDtoots. PowerDirector Tutorials, tips, free resources & more. Subscribe!
Full linked Tutorial Catalog
PDtoots happily supports fellow PowerDirector users!
AlS
Senior Member Location: South Africa Joined: Sep 23, 2014 18:07 Messages: 290 Offline
[Post New]
Quote: Hello AIS & Shadowman!

Firstly AIS(Al), Sorry, but a lot of what you said went completely over my head. As for SVRT, I find myself avoiding it of late as when I've used it in the past, it seemed to cut a portion out of the overall running time of my videos(only a few frames, but enough to render the result as less than the length of the video before rendering, this I select the option to view the clip as it renders, that way I know if I've missed any transitions and can cancel the render in order to make my corrections).
Neil.


Sorry Neil - Too much info. In spite of my long-winded attempt to explain - it's actually quite simple.


  1. Camera capture files like your HD MP4 files are highly compressd. Each generation loses some quality every time they are re-renderd. Compressed files are not ideal for editing for many reasons.

  2. Magic+PD just decompresses your camera HD MP4 files before PDR14 starts editing. How it works is not really important.

  3. You then edit as you normally would (no shadow files) and produce your H.264 MP4 - or whatever you want.


It's a normal process used in all professional editing software.

It's not complicated and really worth a try.

Al

This message was edited 1 time. Last update was at Jun 30. 2016 03:30

Power Director 13&14 Ultimate, Photo Director 6, Audio Dir, Pwr2Go 10
Win 10 64, Intel MB DH87MC, Intel i5-4670 CPU @ 3.40GHz, 16Gb DDR3 1600, 128Gb SSD, 2x1Tb WDBlue 7200rpmSATA6, Intel 4600 GPU, Gigabyte G1 GTX960 4GB, LG BluRay Writer
The Shadowman
Senior Contributor Location: UK Joined: Dec 15, 2014 13:06 Messages: 1831 Offline
[Post New]
Come on Neil

Get a grip, I even gave you the link to the site. How can you think it's "not available".

Thanks, Tony, for saying what I needed to say.

Robert

PS Mattc might find it a useful thread too

http://forum.cyberlink.com/forum/posts/list/46177.page Panny TM10, GH2, GH4,
AlS
Senior Member Location: South Africa Joined: Sep 23, 2014 18:07 Messages: 290 Offline
[Post New]
Shadow Files reply from Shadowman????
Sounds spooky...



Tony - Thanks for the info. Does that mean we are looking at low resolution shadow files in the preview window?

Al

This message was edited 2 times. Last update was at Jun 30. 2016 10:42

Power Director 13&14 Ultimate, Photo Director 6, Audio Dir, Pwr2Go 10
Win 10 64, Intel MB DH87MC, Intel i5-4670 CPU @ 3.40GHz, 16Gb DDR3 1600, 128Gb SSD, 2x1Tb WDBlue 7200rpmSATA6, Intel 4600 GPU, Gigabyte G1 GTX960 4GB, LG BluRay Writer
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
Okay, I did it! Now what? The Magic+PD has been downloaded and installed but I thought it would incorporate into the Power Director 14 folder( and behave as a plug-in). It has a separate shortcut icon on my computer. I'll toy around with it and see what it does. but if it causes problems, it gets the order of the boot!

Neil.
The Shadowman
Senior Contributor Location: UK Joined: Dec 15, 2014 13:06 Messages: 1831 Offline
[Post New]
Neil

Keep the program well away from PD.

Just let the icon sit on your desktop and open it only when you need to use it.

M+PD is a STAND ALONE program that prepares your files ready for editing in PD.

M+PD places the converted files into a folder that YOU create. It's from that folder that you take the files into PD14.

Ne'er the twain shall meet

Robert Panny TM10, GH2, GH4,
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
Quote: Come on Neil

Get a grip, I even gave you the link to the site. How can you think it's "not available".

Thanks, Tony, for saying what I needed to say.

Robert

PS Mattc might find it a useful thread too

[url=http://forum.cyberlink.com/forum/posts/list/46177.page
]http://forum.cyberlink.com/forum/posts/list/46177.page
[/url]

Okay, Shadowman(Robert), As you can see, I've used your link and downloaded and installed Magic+PD(YUV),as you see by my comments. Does this produce the compressed AVI files as used to download feature films? I've had a friend give me some features in AVI that had a total file size of around the 650-to700 megabyte mark. It would be good if I could use Magic+PD to convert my videos to such a file type and size, it would free up a huge amount of space on my external drives.(I have two external drives dedicated to video content, one is a 1TB drive for keeping feature films and TV shows, the other, a 2TB drive for home movies).

So that's what I'd now like to know.

Cheers!

Neil.
Neil.F.1955 [Avatar]
Senior Contributor Joined: Mar 07, 2012 09:15 Messages: 1303 Offline
[Post New]
Hi, again!

AIS suggested I use Magic+PD to convert(decompress) my MP4 camera shots before importing them into PD for editing, but again I want to know if I can get them into this AVI "Cineform" file type I've seen mentioned elsewhare on this forum. I figure that must be the type with the small file size. I definitely want to avoid like the plague any file type that's going to gobble up hard drive space.

Neil.
The Shadowman
Senior Contributor Location: UK Joined: Dec 15, 2014 13:06 Messages: 1831 Offline
[Post New]
Oh Dear, Neil

The reason that Magic+PD was created was for the sole purpose of creating files that acted in the same way as shadow files, but much more effeiciently. M+PD is designed to convert files such as 4K MP4 and any other high bitrate media that PD currently finds difficult to process without the lagging, jumping and freezing currently experienced by some editors.

M+PD converts the files to the .AVI format that PD can handle very easily and thus can be edited to the greater extent without problems. M+PD converts MP4 etc. at a rate of about eight time quicker than shadow files which makes the program very efficient. The downside of this is, if it is one, is that the AVIs are about 10 times the size of the original files. So, because of this, it makes sense to delete them once the project is finished.

I do not think, having said the above, that it would suit your purpose of space saving. Remember the program is only designed to make editing easier in an NLE, and that it does very well indeed.

Help me Neil - read optodata's first M+PD post thoroughly and you will know as much as I do - which aint a lot

Robert Panny TM10, GH2, GH4,
ynotfish
Senior Contributor Location: N.S.W. Australia Joined: May 08, 2009 02:06 Messages: 9977 Offline
[Post New]
Quote: Does that mean we are looking at low resolution shadow files in the preview window?


Hi Al -

After exploring this in some depth, I can say with reasonable confidence that we're looking at the original file in preview - NOT the SD shadow file.

With shadow files enabled I imported a range of clips - UHD, genuine 4K & an 8K file produced in PDR. After the shadow files had generated, I moved the clips to the timeline and took snapshots. As expected they were full resolution images.

Next I physically imported the 720x480 shadow files & repeated the procedure with snapshots, just for comparison. Under normal editing condidtions, what we're seeing in preview is the original file... illustrated in the difference between the two lots of (partial) snapshots.



Pretty much nails it, I think laughing

Cheers - Tony
Visit PDtoots. PowerDirector Tutorials, tips, free resources & more. Subscribe!
Full linked Tutorial Catalog
PDtoots happily supports fellow PowerDirector users!
Powered by JForum 2.1.8 © JForum Team