Hello everyone!

I've recently built an HTPC to use with my  Benq W5000 video projector.
It has an ATI3870 video card (Catalyst 8.512.0.0) and runs on Windows XP Pro SP3.
The projector is hooked at one of the card's DVI connectors, using a DVI>HDMI cable and works at Video (16-235) levels. A 19" LG Flatron monitor is connected at the other DVI plug.
Currently, I'm only projecting HD material, using Media Player Classic, KMPlayer and PowerdDVD.

My problem is that the video card seems to output PC levels (0-255) instead of Video (16-235), so I have to manually make software adjustments to fix this. I tried to use the special ATI DVI>HDMI dongle, as I've read that this would automaticaly adjust output to Video levels, however this didn't seem to happen. My 35ft video cable has a DVI conector at the HTPC's end, so I had to use an extra HDMI>DVI converter. Do you think probably this has messed things up? Unfortunately  I don't have a 35ft HDMI>HDMI cable to check...

Anyway, I'm currently experimenting with player-software and "Avivo Video">"Basic color", adjusting Brightness and Contrast settings. However, I'm not sure this gives constant results and I have a feeling that it messes with grayscale...

It would be much appreciated if you could help me find the best way to fix this issue. Is there a best way to deal with it, using software adjustments?
Or do you think I should buy a new 35ft HDMI>HDMI cable so that ATI's dongle may work?

Cheers!

Alombar:

Hello everyone!

I've recently built an HTPC to use with my  Benq W5000 video projector.
It has an ATI3870 video card (Catalyst 8.512.0.0) and runs on Windows XP Pro SP3.
The projector is hooked at one of the card's DVI connectors, using a DVI>HDMI cable and works at Video (16-235) levels. A 19" LG Flatron monitor is connected at the other DVI plug.
Currently, I'm only projecting HD material, using Media Player Classic, KMPlayer and PowerdDVD.

My problem is that the video card seems to output PC levels (0-255) instead of Video (16-235), so I have to manually make software adjustments to fix this. I tried to use the special ATI DVI>HDMI dongle, as I've read that this would automaticaly adjust output to Video levels, however this didn't seem to happen. My 35ft video cable has a DVI conector at the HTPC's end, so I had to use an extra HDMI>DVI converter. Do you think probably this has messed things up? Unfortunately  I don't have a 35ft HDMI>HDMI cable to check...

Anyway, I'm currently experimenting with player-software and "Avivo Video">"Basic color", adjusting Brightness and Contrast settings. However, I'm not sure this gives constant results and I have a feeling that it messes with grayscale...

It would be much appreciated if you could help me find the best way to fix this issue. Is there a best way to deal with it, using software adjustments?
Or do you think I should buy a new 35ft HDMI>HDMI cable so that ATI's dongle may work?

Cheers!

You are correct that your ATI card is outputting sRGB (0-255) and should be driving your projector with BT.709 levels (16-235 using either RGB or YCbCr).

With ATI cards, they default to sRGB, UNLESS you are specifically using the ATI HDMI dongle AND are driving a display that reports itself as an HDMI device per the EDID.  On top of this, the ATI HDMI dongle must be of the correct type for the particular video card.  When you have all of the above, ATI cards actually output using YCbCr with a 16-235 range (not RGB because the HDMI spec prefers YCbCr).

Many displays and projectors do not have HDMI ports, nor HDMI compliant EDIDs and with this subset of displays there is no way to get an ATI card to output using BT.709 levels without adjusting the brightness and contrast in CCC.

In your situation, if I'm not mistaken, you do actually have a projector with an HDMI port.  If you install the HDMI dongle that came with your card and then use an HDMI cable (or even the cable you have combined with another, but standard off the shelf DVI-HDMI adapter) your video card "should" start to use BT.709 (16-235).

FYI, Nvidia cards do not have any requirements to use an HDMI dongle, and the latest drivers allow the user to manually select sRGB vs BT.709 (so I've been told).  On the other hand Nvidia cards do not correctly identify many HDTVs with DVI ports as TVs (they often "think" they are monitors and in this case you can't see anything until a driver is installed and configured using a PC monitor first).  In your case, switching to an Nvidia card should also solve the problem (personally I still prefer ATI cards for Media Center PCs though).

One additional change you need to do is to add the "UseBT601CSC=1" registry setting.  By default ATI video cards assume SD video is in sRGB and HD video is in BT.709.  The HD is correct, but the SD is incorrect.  This registry setting tells the video card to expect SD video to be using BT.601 levels (BT.601 is used by all DVDs, SD TV tuner cards, and all SD hardware video devices).  I recommend using DXVAChecker to add this registry entry.

---

STB w/R5000HD USB I/O, Gigabyte GA-P35-DS4, Quad Q6600, 4.0 GB RAM, ATI HD 3870 512MB, Ultra XVS 600W PSU, 3x SATA 500GB, 2x SATA 300GB, LG GGC-H20L, PVR-250, Toshiba 51H83 (51" HDTV), Yamaha RX-V2400 Amp, 5x Energy Speakers, SVS Sub, Harmony 880 Remote

Thank you very much for your reply.
I tried again the trick with the ATI DVI>HDMI dongle, off the shelf HDMI>DVI adapter, DVI>HDMI cable. Unfortunately there was no way to see Below-black or Peak-whites bars, without making software adjustments (W5000 is supposed to be able to pass a BB test).

One strange thing is that, when I' m using VMR9 as renderer, Avivo video adjustments (or HW/SW from within KMPlayer) don't seem to have any effect.
Using Overlay as renderer, makes them work properly. Any ideas how to fix this?

Alombar:

Thank you very much for your reply.
I tried again the trick with the ATI DVI>HDMI dongle, off the shelf HDMI>DVI adapter, DVI>HDMI cable. Unfortunately there was no way to see Below-black or Peak-whites bars, without making software adjustments (W5000 is supposed to be able to pass a BB test).

One strange thing is that, when I' m using VMR9 as renderer, Avivo video adjustments (or HW/SW from within KMPlayer) don't seem to have any effect.
Using Overlay as renderer, makes them work properly. Any ideas how to fix this?

Ahh.... you are trying to calibrate using BTB/WTW...

Here are the issues:

1. Current "standard" PC implementations for colorspace conversions discard BTB and WTW data. 
2. By making AVIVO adjustments you are only able to see BTB and WTW by making your Media Center "out of calibration". 
3. AVIVO adjustments only apply to ATI's video processing engine.  Software codecs (such as FFDShow) take most of AVIVO's video processing out of the pipeline and depending on the renderer selected, AVIVO is used in different ways.

I'll try to explain.  In general, the way PCs handle colorspace (using HD video decoded using AVIVO as an example) is:

1. The source HD video uses BT.709.  This means that while the entire 0-255 range is digitally available, only the range 16-235 is intended to be visible as separate grey levels.  Everything below 16 is intended to be black and everything above 235 is intended to be full brightness/white.  As you know, outside this range is BTB and WTW, and this is sometimes used for calibration; but not for "viewing".  Source SD video uses BT.601 and this is similar (but slightly different) to BT.709.
2. AVIVO's video processing engine transforms the BT.709 (for HD, or BT.601 for SD) into the standard PC colorspace sRGB (0-255).  This means that the range 16-235 is expanded to fill the entire 0-255 range.  Everything outside 16-235 is lost in this process.  ATI does use 10-bit math here so rounding errors shouldn't be an issue.  In effect this means a number such as 16 is mapped to 0, 235 is mapped to 255 and 50 is mapped to something like 39.59.  All numbers below 16 are mapped to 0 and all numbers above 235 are mapped to 255.  Note when adjusting brightness/contrast in the AVIVO settings within CCC you are tweaking this step.  Default brightness/contrast performs the 16-235 to 0-255 transform and changing AVIVO brightness/contrast changes this expansion.
3. When an HDMI dongle is used along with a display that "asks" for YCbCr, the video card performs the reverse transform at final output.  Since 10-bit math is used, there shouldn't be any round off errors and the output will be accurate within the 16-255 range.  Unfortunately since during step #2 BTB and WTW is lost and there is no way to regain this.  Note when adjusting the overall brightness/contrast in the "color" settings within CCC you are tweaking this step.  Default brightness/contrast performs the 0-255 to 16-235 transform when the HDMI dongle is used and a display requesting YCbCr is used (HDMI port).  Adjustments to the overall "color" brightness/contrast changes this "compression".

So, if you think about the whole transform, while the source video does actually have a 0-255 range (including the "intended to be invisible BTB and WTW"), the default HDMI output from the video card only has a 16-235 range.  Everything below 16 in the source ends up as 16 and everything above 235 in the source ends up as 235.

For a properly calibrated system this solution works fine since BTB and WTW isn't intended to be visible anyway.  Unfortunately, if you are using a calibration disc that uses BTB and WTW to aid in calibration, you will not be able to properly calibrate your display.

If you REALLY want to keep BTB and WTW data, you can do this by removing the ATI HDMI dongle (to remove the step #3 transform) and adjusting the AVIVO brightness/contrast so there is no expansion performed at step #2 above (the AVIVO brightness/contrast settings you need are +16/86%).  The downside to this is only video processed by AVIVO will be displayed correctly on your projector.  All non-AVIVO video processing, photographs, desktop, video games, PC applications, etc. will output using the full (intended to be visible) 0-255 range and with your projector properly calibrated you will only see what is in the 16-235 range (in other words darks will be output in the BTB range and entirely black and brights will be output in the WTW range and clipped beyond full brightness on your projector).  Many software video codecs (such as FFDShow) can also be "tweaked" to turn off video expansion, but you will need to do this for each one you intend to use and you still won't get photographs, video games, etc. to display correctly.

 

My advice is to keep things simple and use ATI's intended solution.  Install the HDMI dongle, use an HDMI connection to your projector, use default brightness/contrast settings everywhere in CCC and then get yourself some calibration sources that don't rely on BTB and WTW to calibrate your projector.  You will get a quality image everywhere, including photographs, video games and all video (regardless of codec used).  Also you will find all video sources will be calibrated without any tweaking on the PC end (SD TV in Media Center, DVDs in Media Center, DVDs in PowerDVD, Bluray/HD-DVD in PowerDVD, etc.). 

Some people argue that software codecs and Nvidia video cards only use 8-bit processing so these expansion transforms will have round off errors.  I doubt these potential errors are really visible to the viewer so even when AVIVO is not used, I still consider the most practical solution to stick with the default where everything is transformed initially to sRGB and then transformed at final output to whatever the display requires.

Do make sure you also add the "UseBT601CSC=1" registry entry to ensure SD video is processed correctly.  Without this, SD video will look washed out.

---

STB w/R5000HD USB I/O, Gigabyte GA-P35-DS4, Quad Q6600, 4.0 GB RAM, ATI HD 3870 512MB, Ultra XVS 600W PSU, 3x SATA 500GB, 2x SATA 300GB, LG GGC-H20L, PVR-250, Toshiba 51H83 (51" HDTV), Yamaha RX-V2400 Amp, 5x Energy Speakers, SVS Sub, Harmony 880 Remote

Thanks again for your detailed answer. The truth is that I'd really like to keep BTB and WTW information, for two reasons:
1) I find them to be the most consistent and reliable way to adjust brightness and contrast, without guessing.
2) There are several reliable sources, that say WTW area does include useful visible information. Discarding it completely, would erase small nuances of near-white grades, converting them all to white (BTB should never be visible on digital displays, indeed).

So, I think I'll experiment a little bit more the hard way.

With so many people using HTPCs as sources for their projectors/HDTVs nowadays, it's really dim-witted on behalf of the video cards manufacturers to have this approach: they should had kept it simple and pass the whole video range over HDMI connections, instead of discarding everything out of 16-235, expanding and then reducing back to 16-235. Not only useful information is lost, but (despite 10-bit math) some unnecessary rounding errors are definitely introduced.

May I ask something more about CCC settings? What is the difference between "Avivo color" settings (under "DTV" group) for brightness/contrast and "Basic color" (under "Avivo video" group)? Could they both be used for the purposes described? Would the first one provide more universal results and/or work with VMR9?

Alombar:

Thanks again for your detailed answer. The truth is that I'd really like to keep BTB and WTW information, for two reasons:
1) I find them to be the most consistent and reliable way to adjust brightness and contrast, without guessing.
2) There are several reliable sources, that say WTW area does include useful visible information. Discarding it completely, would erase small nuances of near-white grades, converting them all to white (BTB should never be visible on digital displays, indeed).

So, I think I'll experiment a little bit more the hard way.

You are not alone in wanting to keep BTB and WTW data.  I'm not convinced I would notice any degradation if comparing one to the other side by side, but the engineer in me wants perfection and for nothing to be "thrown away".  I have looked at the unaltered RGB raw data with a few DVDs and Bluray movies and the ones I looked at did have a 16-235 range.  I suspect you are correct though and that "some" DVDs are mastered to use some of the WTW range (noting that SD video actually uses a 0-240 range already for Y and to upconvert it to HD and display it correctly on your HDTV the colorspace needs to be transformed from BT.601 to BT.709 during the upconverstion).  The current HTPC video solution does indeed perform this transformation (BT.601 ->  BT.709) during upconversion of SD to HD resolutions.

It isn't that difficult to keep BTB and WTW (per my last post), but your system will then ONLY be calibrated for video.  When you look at photographs, video games, etc., the full 0-255 will be output to your HDTV, without compression into the correct 16-235 range.  On top of this SD sources will likely be messed up somewhat (not badly, but I don't think the transforms are quite right when using Media Center and AVIVO/Purevideo and tweaking things to maintain BTB/WTW).  Personally when I weigh loosing BTB and WTW vs having very poor looking photographs and video games, I'd rather stick with loosing BTB/WTW.

Alombar:

With so many people using HTPCs as sources for their projectors/HDTVs nowadays, it's really dim-witted on behalf of the video cards manufacturers to have this approach: they should had kept it simple and pass the whole video range over HDMI connections, instead of discarding everything out of 16-235, expanding and then reducing back to 16-235. Not only useful information is lost, but (despite 10-bit math) some unnecessary rounding errors are definitely introduced.

I don't know if it is so much the video card manufacturers that are at fault.  The reality is PCs were PCs first and everything was designed around the full 0-255 RGB range that PC monitors use.  Only very recently have digital HDTVs started to be used as displays and the simplest way to make them work is to compress the 0-255 into 16-235 as a final output step.  Consider too that likely 99.9% of PCs are hooked up to displays with a 0-255 range and there is no such thing as BTB or WTW with these (even when video is being played).

The reality is it would be much more complex to switch the basic visible RGB range to 16-235 for the 0.1% of the systems that are hooked up to HDTVs.  Keeping it simple is what they did by adding the ability, at the very end of the pipeline, to convert the final output to HDTV friendly levels.

Also, if you run through the transforms, converting 16-235 into 0-255 and then back, with 10-bit processing there is an exact 1:1 mapping from the original 16-235 to the final 16-235.  There isn't any rounding error at all.  A bigger question to me is do the video card manufacturers actually use 10-bit processing.  ATI claims that they do, but it would take some work for us to actually confirm this is true.  There is also some question as to if the OS might somehow throw away the 2 least significant bits during the handoff to the renderer (I don't know enough to confirm either way).

Quite honestly, one of the questions I often ask myself is why didn't the entire TV industry switch to the 0-255 range when TVs started to accept digital signals.  The only reason 16-235 is the visible range is historically to handle all of the tolerances involved with older analog systems and to ensure nothing is lost at the low and high end of the voltage.  Since with digital there is no need for this, it would be much simpler if the entire 0-255 was used as the visible range.  On top of this it would actually produce a better image since there would be a big improvement to the number of different colors available (finer resolution in the colorspace - 256^3=16777216  vs (235-16)^3=10503459 - this is almost 60% greater fidelity in the colorspace).  I suspect part of the reason 16-235 was chosen was to actually reduce the required bandwidth for compressed HD content.

Alombar:

May I ask something more about CCC settings? What is the difference between "Avivo color" settings (under "DTV" group) for brightness/contrast and "Basic color" (under "Avivo video" group)? Could they both be used for the purposes described? Would the first one provide more universal results and/or work with VMR9?

I explained this in my last post.

AVIVO -> video processing when AVIVO is being used

Basic color -> final output, affecting everything (unless overridden by an application)

---

STB w/R5000HD USB I/O, Gigabyte GA-P35-DS4, Quad Q6600, 4.0 GB RAM, ATI HD 3870 512MB, Ultra XVS 600W PSU, 3x SATA 500GB, 2x SATA 300GB, LG GGC-H20L, PVR-250, Toshiba 51H83 (51" HDTV), Yamaha RX-V2400 Amp, 5x Energy Speakers, SVS Sub, Harmony 880 Remote

Once more, thank you very much! Knowledgeable members willing to help, are priceless  for rookies like myself!

BTW Here's an interesting chat about WTW. Look for posts from sspears and dmunsil.

Alombar:

Once more, thank you very much! Knowledgeable members willing to help, are priceless  for rookies like myself!

BTW Here's an interesting chat about WTW. Look for posts from sspears and dmunsil.

I decided to delete those other rambling posts I made while digging into the video pipeline in greater detail.

I have finally figured out exactly where BTB and WTW is lost and my original posts were correct - however, here are additional details:

In the TV, video and regular engineering world, anytime someone says YCbCr, it means a visible level range of 16-235 (238 or 240 also, depending on the exactly color space BT.601 for SD and  BT.709 for HD) is used.  However, it turns out that in Microsoft's world YCbCr just means the 3 variables include Y, Cb and Cr and to simplify, Microsoft calls digital YCbCr YUV (or YUV2, UYVY, UYVP, NV12, etc. depending on sub-sampling levels).  This does not have anything to do with the colorspace, nor calibrated visible range.  Note that outside of Microsoft, YUV would actually mean analog video with a voltage equivalent 16-235 range.

Since PCs have always used the full 0-255 range for visible color/grey levels, by default all video decoding filters expand the original 16-235 range into 0-255.  The occurs regardless if the source YCbCr format remains YCbCr or if it is converted to RGB.

In particular, if you were to examine the YCbCr levels being passed from Microsoft's MPEG-2 decoder to Microsoft's EVR renderer when Media Center is playing Recorded TV, you would find that they have actually been expanded so the visible range is now 0-255.  This MPEG-2 decoding step throws away the BTB and WTW by expanding the range, even though the video is still "labeled as" YCbCr.  BTW, the MPEG-2 decoder can also convert and output in RGB, but always the visible range defaults to an expanded 0-255.  Media Center in particular configures the MPEG-2 decoder to output YCbCr.  Note as mentioned before the AVIVO settings control the MPEG-2 decoder and enable us to turn off this expansion using brightness/contrast if we prefer.

Since PCs always operated using the full 0-255 range and something like 99.9% of PC displays use the full 0-255 range, it sortof does make sense that everything is converted to 0-255.  In most cases this should slightly improve the image through the various denoise, scaling, etc. steps when displayed on a PC monitor.  However, those of us using HDTVs that require BT.709 (16-235 range), would be better off if Microsoft stuck with industry convention and required YCbCr to use a 16-235 visible range.  If this was the case, the renderer would be able to expand it to a 0-255 range for a PC monitor, or leave it, unaltered (including BTB/WTW) and output it to an HDTV.  Note, as mentioned before, the overall "color" settings control the renderer's output, but unfortunately the settings apply to everything being displayed (there is no way to adjust the renderer specifically for YCbCr video for example - this means if we make adjustments to maintain BTB/WTW we screw up the display of photographs, video games, desktop, etc.).

For now we will likely have to live with "hacks" if we want to maintain WTW/BTB data (and sacrifice the ability to display photographs, video games, desktop, etc. properly). 

Microsoft's next generation video pipeline, Media Foundation, is designed to maintain the exact colorspace throughout the pipeline.  If I'm reading this correctly, in the future (Windows 7 possibly?) we should see BTB/WTW maintained and output from our Media Center PCs.

---

STB w/R5000HD USB I/O, Gigabyte GA-P35-DS4, Quad Q6600, 4.0 GB RAM, ATI HD 3870 512MB, Ultra XVS 600W PSU, 3x SATA 500GB, 2x SATA 300GB, LG GGC-H20L, PVR-250, Toshiba 51H83 (51" HDTV), Yamaha RX-V2400 Amp, 5x Energy Speakers, SVS Sub, Harmony 880 Remote

Looks like you're learning along with me :-)
Well, I guess we'll have to wait and see what comes with Windows 7...