Multicam audio analysis fails if sampling rate does not match between video or audio clips

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Solved by

Jan 13, 2014 07:27

When I use the multicam mode with the attached video clips, which were taken at the same time with the 2 different cameras, the audio analysis fails to find any overlap between the clip. It just places them one after the other.

I have determined that this is related to the different ways each camera records the audio.

One clip is from my old but trusty Canon HG21 camcorder, which records audio in stereo at 48 kHz, compressed with Dolby Digital with a 256 kbps data rate.

Another clip is from my new Pentax K-30 DSLR, which records records audio in 16-bit mono at 32 kHz in uncompressed PCM - with a 512 kbps data rate.

If I re-render the Pentax clip with PowerDirector, the new file has audio at 48 kHz. I have also attached that re-rendered file.
At that point, I am able to synchronize it with the Canon clip in the multicam mode.

I feel that this bug seriously reduces the usability of the multicam mode. Essentially, I am forced to re-render every clip from my Pentax camera for no good reason. I'm never going to use the low quality 32 kHz audio anyway for production, I rely on it only for synchronizing in multicam mode.

The bug doesn't just affect cameras with different audio sampling rates, but also one camera with an audio file as well. I usually record audio separately, and use a 96 kHz sampling rate in the original recording. I didn't record separate audio for this test, though. But if I mix my separate audio to a 96 kHz WAV file, PowerDirector cannot synchronize it with any video clip in the multicam mode, as none of my cameras use a 96 kHz sampling rate for their audio.

As I have found, PowerDirector currently requires the audio sampling rate to match between all video and audio clips in the multicam mode. Surely this is not an intended limitation of multicam mode. If so, it would be documented somewhere, and the program would also warn the user that the clips can't be synchronized instead of trying to do an audio analysis that has no hope of ever completing.

IMO, the audio analysis algorithm that PowerDirector uses for synchronizing is faulty, and should be sampling-rate independent. Please fix it !

Filename	pentax-redo.m2ts	Download
Description	Clip from Pentax K-30 DSLR, re-rendered with PD for 48 kHz audio
Filesize	35003 Kbytes
Downloaded:	377 time(s)

Filename	00045.MTS	Download
Description	Clip from Canon HG21 camcorder
Filesize	31512 Kbytes
Downloaded:	346 time(s)

Filename	IMGP6185.MOV	Download
Description	Clip from Pentax K-30 DSLR
Filesize	16145 Kbytes
Downloaded:	354 time(s)

This message was edited 1 time. Last update was at Jan 13. 2014 07:31

MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 06:13

Dafydd, did you pass this on to Cyberlink, or should I report it directly through their support ? MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 24, 2014 13:36

I enjoyed your video samples, and I was first testing the multicamera mode, it worked perfectly, I used the audio 1 (highest) as reference.
Produced for AVCHD (MTS video of the same), only needed to increase the volume of audio 1.
I recommend you use the 16x9 format, all cameras to get better AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 18:45

Quote: I enjoyed your video samples,

Thanks.

and I was first testing the multicamera mode, it worked perfectly, I used the audio 1 (highest) as reference.

What do you mean by "I used the audio 1 (highest) as reference" .

The files from the camera that PD can't synchronize are 0045.MTS (from Canon HG21) and IMGP6185.MOV (from Pentax K-30).

If you used 0045.MTS with pentax-redo.mts, they will synchronize, but that means the Pentax clip has to be re-rendered before synchronization.

I recommend you use the 16x9 format, all cameras to get better

Yes, and I normally do. This was only for testing. I recorded at the lowest resolutions that my cameras could produce so that the files would be small enough to post, yet still have enough audio content in them.

This message was edited 1 time. Last update was at Jan 24. 2014 18:45

MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 24, 2014 19:41

What do you mean by "I used the audio 1 (highest) as reference" .

Google translation?
I meant the audio largest video (longer) however this is with low volume may be increased at the end directly in the timeline or opening in Wave Editor.

The files from the camera that PD can't synchronize are 0045.MTS (from Canon HG21) and IMGP6185.MOV (from Pentax K-30).

If you used 0045.MTS with pentax-redo.mts, they will synchronize, but that means the Pentax clip has to be re-rendered before synchronization.

It is normal to choose the best audio from videos, the multicam will only use 1 audio in your case I used the extra time to facilitate synchronization of videos.
After the timeline can use other audio, if you have other better.

Here I just added the 3 files, perchance the most time video 1, I selected this to be the audio guide.
see photo


Filename	multican.jpg	Download
Description
Filesize	160 Kbytes
Downloaded:	550 time(s)

This message was edited 1 time. Last update was at Jan 24. 2014 19:43

AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 20:23

Quote:

What do you mean by "I used the audio 1 (highest) as reference" .

Google translation?
I meant the audio largest video (longer) however this is with low volume may be increased at the end directly in the timeline or opening in Wave Editor.

OK, now I understand. I have no idea which of the 2 video files had the best/loudest audio. It would have been simpler to just name the files.

It is normal to choose the best audio from videos, the multicam will only use 1 audio in your case I used the extra time to facilitate synchronization of videos.

For me it doesn't matter which audio I select under "audio source". The result is always the same - PD cannot synchronize IMGP6185 (which is the original Pentax file at 32 kHz) with the other files.

I have attached pics of what happens if I select all 3 clips like you did, and if I just select the 2 clips of the original camera.

It seems your version of PD12 can synchronize IMGP6185, but mine cannot.

Which build are you using ? I am at 2420 .

Not that pentax-redo.m2ts is a manually re-rendered version of IMGP6185.MOV .

The original files out of the cameras are IMGP6185.MOV and 0045.MTS . I would like to synchronize these 2 clips only, without having to re-render IMGP6185.MOV . Can your PD12 do that ? Mine can't. That's the bug I am reporting.


Filename	2clips.png	Download
Description	Try to synchronize only the original 2 clips from the cameras
Filesize	943 Kbytes
Downloaded:	263 time(s)


Filename	3clips.png	Download
Description	Try to synchronize all 3 clips, including the re-rendered cilip
Filesize	134 Kbytes
Downloaded:	251 time(s)

This message was edited 1 time. Last update was at Jan 24. 2014 20:24

MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 24, 2014 21:15

I did exactly as shown in the 2 photos, made even choosing the right audio camera 3, this case has silence at the beginning and end of the video.
I'm using PD12, 2509 Beta version AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 21:20

Quote: I did exactly as shown in the 2 photos, made even choosing the right audio camera 3, this case has silence at the beginning and end of the video.
I'm using PD12, 2509 Beta version

So you got the same behavior I did ?

If so, how did you synchronize all 3 clips in your previous post ? Did you synchronize IMGP6185.MOV manually ?
MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 24, 2014 21:46

Here the 2 or 3 videos, always aligns vertically left after import, after clicking the Apply button, the audio synchronization happens in both cases, 2 or 3 videos.
Not sure if this makes a difference, here I have installed K-Lite_Codec_Pack_1015_Full

This message was edited 1 time. Last update was at Jan 24. 2014 22:50

AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 21:49

Quote: Here the 2 or 3 videos always aligns vertically left to import, after clicking the Apply button, the audio synchronization happens 2 or 3 channels.
Not sure if this makes a difference, here I have installed K-Lite_Codec_Pack_1015_Full

Yes, I know, the pentax-redo and 0045 videos will align. I created pentax-redo to work around the PowerDirector bug.
Maybe I shouldn't have posted pentax-redo at all, so you wouldn't have gotten confused.

What I care about is being able to take the videos right out of my cameras, and using multicam mode, without re-rendering one of them, ie. using multicam mode to synchronize 0045 and IMGP6185 . And PowerDirector 12 is unable to do that.

This message was edited 1 time. Last update was at Jan 24. 2014 21:50

MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

babindia

Senior Contributor Location: India Joined: Aug 16, 2007 06:11 Messages: 884 Offline

Jan 24, 2014 22:17

I agree with Julien Pierre. In the 1st case ie., raw clips; on clicking audio analysis the clip is shifted behind the 1st clip while with re-rendered clip it synchronizes properly. It is a bug indeed PC specs :
OS Windows 10.0 Pro
MB - AS rock Z77 extreme 11
Intel 3770K @ 4.0 Ghz OC
Gskill 32 GB RAM 1800 Mhz
6 TB HDD, SSD bootable
nVidia ASUS GTX 660 Ti
BenQ 22" LCD monitor 1920x1080

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 22:33

Quote: I agree with Julien Pierre. In the 1st case ie., raw clips; on clicking audio analysis the clip is shifted behind the 1st clip while with re-rendered clip it synchronizes properly. It is a bug indeed

Thanks, sometimes language barriers make it difficult with these highly technical issues.

Dafydd, since the problem has been confirmed, will you pass this on to Cyberlink ? Or do I need to go through their support myself ?
MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 24, 2014 23:07

Julien Pierre

This should be a specific error but Intel CPU.
Here AMD see my signature.
I have seen in recent updates, coreção error for Intel CPU ... AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 24, 2014 23:30

Quote:

Julien Pierre

This should be a specific error but Intel CPU.
Here AMD see my signature.
I have seen in recent updates, coreção error for Intel CPU ...

Uh ???

This issue has nothing to do with the CPU. And I have tried it on different computers with both AMD and Intel CPUs. It happens on both.
MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 25, 2014 06:18

Quote:
This issue has nothing to do with the CPU. And I have tried it on different computers with both AMD and Intel CPUs. It happens on both.

I mensionei K-lite Codec Pack, it was no accident, files here MOV a Genius camcorder showed without audio in PD10/11, K-Lite esolved.
In PD12 do not know moved. AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 25, 2014 06:22

Quote:

Quote:
This issue has nothing to do with the CPU. And I have tried it on different computers with both AMD and Intel CPUs. It happens on both.

I mensionei K-lite Codec Pack, it was no accident, files here MOV a Genius camcorder showed without audio in PD10/11, K-Lite esolved.
In PD12 do not know moved.

I'm sorry, but once again I don't understand you. It seems you are trying to describe some problem completely different from the issue in this thread.

MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 25, 2014 06:51

Quote:
I'm sorry, but once again I don't understand you. It seems you are trying to describe some problem completely different from the issue in this thread.

I do not understand, you seem to want to make me believe what does not, what is working here.
I also have errors here in PD12, which supports dis Cyberlink can not replicate this and no solution so far. AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 25, 2014 07:07

Quote:
I also have errors here in PD12, which supports dis Cyberlink can not replicate this and no solution so far.

I don't doubt it, but your issues are different from this multicam audio synchronization bug, and you should discuss them in a separate forum thread.
MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

Julien Pierre

Contributor Joined: Apr 14, 2011 01:34 Messages: 476 Offline

Jan 25, 2014 07:34

playsound,

I re-read all your messages. I was intrigued by your mention of K-Lite since you weren't sure it made a difference.

I downloaded it (10.2 mega) - and to my surprise, it did.
When k-lite is installed, PD12 is able to synchronize the original 2 clips.

I then uninstalled k-lite, and the problem came right back. So K-lite definitely fixes something. I just wonder how many other problems it might create, since it's not supported by Cyberlink. I would expect PD12 multicam to work without additional CODECs.

Edit:

I also tried to synchronize a video at 48 kHz with a WAV file from separate audio at 96 kHz, and PD12 still can't do it.
I have to remix the WAV to 48 kHz to match the camera. K-Lite didn't fix that part.

This message was edited 1 time. Last update was at Jan 25. 2014 07:51

MSI X99A Raider
Intel i7-5820k @ 4.4 GHz
32GB DDR4 RAM
Gigabyte nVidia GTX 960 4GB
480 GB Patriot Ignite SSD (boot)
2 x 480 GB Sandisk Ultra II SSD (striped)
6 x 1 TB Samsung 860 SSD (striped)

2 x LG 32UD59-B 32" 4K
Asus PB238 23" HD (portrait)

PlaySound

Senior Contributor Joined: Dec 02, 2010 10:26 Messages: 2031 Offline

Jan 25, 2014 09:07

Quote: playsound,

I re-read all your messages. I was intrigued by your mention of K-Lite since you weren't sure it made a difference.

I downloaded it (10.2 mega) - and to my surprise, it did.
When k-lite is installed, PD12 is able to synchronize the original 2 clips.

I do not like installing K-Lite_Codec_Pack.
In the first PD12 installation, new machine I was reluctant to install K-Lite.
I do not remember why, I think something to do with Media Player, just installing.
It is common to attribute any error in PD. but we know PD, using the tools Windowns and others. AMD-FX 8350 / 8GB DDR3
SSD SUV400S37240G / 2-HD WD 1TB
AMD Radeon R9 270 / AOC M2470SWD
Windows 7-64 / PD16 Ultimate