Hi
I did have same problem black screen or freze when I try play mkv files in 3d fullscreen or in fullscreen. It was not powerdvd 12 fualt. I manger track it down to amd catalyst driver 12.4 issuse. When u try install it, check see if really say 12.4 in the controll panel if not. you need do follow step.
1. unstall all amd/ati drivers for the graphice card include the mangers.
2. restart the systsem to normal windows mode.
3. reboot now to safe mode (do not try take any short cut if u do, u end up and start from 1. )
4. run driver sweeper in windows safe mode, and select all ati and amd and realteck drivers. (other wise it refuse install the driver proper)
5. and press on analasying and clean the setting.
6. after it run one time, restart windows to normal mode.
7. u will now see windows at res 800x600 screen (vista/win7) 640x480 (xp/2000).
8. install u catalyst 12.4 now
9. after it install reboot the system.
the solv was partly taken from sapphire forum for their amd 7770 card. from people that have audio issue with catalyst drv and they disover the 12.4 was not propper install.