Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

I have a TV capture card that has a feed coming in as a YUV format. I've seen other posts here similar to this question and attempted to try every possible method stated, but neither of them provided a clear image. At the moment the best results were with the OpenCV cvCvtColor(scr, dst, CV_YUV2BGR) function call.

I am currently unaware of the YUV format and to be honest confuses me a little bit as it looks like it stores 4 channels, but is only 3? I have included an image from the capture card to hope that someone can understand what is possibly going on that I could use to fill in the blanks.

The feed is coming in through a DeckLink Intensity Pro card and being accessed in a C++ application in using OpenCV in a Windows 7 environment.

Update

I have looked at a wikipedia article regarding this information and attempted to use the formula in my application. Below is the code block with the output received from it. Any advice is greatly appreciated.

BYTE* pData;
    videoFrame->GetBytes((void**)&pData);
    m_nFrames++;
    printf("Num Frames executed: %d\n", m_nFrames);
    for(int i = 0; i < 1280 * 720 * 3; i=i+3)
        m_RGB->imageData[i] = pData[i] + pData[i+2]*((1 - 0.299)/0.615);
        m_RGB->imageData[i+1] = pData[i] - pData[i+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[i+2]*((0.299*(1 - 0.299))/(0.615*0.587));
        m_RGB->imageData[i+2] = pData[i] + pData[i+1]*((1 - 0.114)/0.436);
                "I am currently unaware of the YUV format" --> Wikipedia. Read up about it: en.wikipedia.org/wiki/YUV#Conversion_to.2Ffrom_RGB
– NickLH
                Oct 31, 2011 at 13:36
                Thanks for linking that article, but am I to assume that I will be working with a y, u, v, and a y1 and using those variables to produce an RGB equivalent? Also would I have to create 4 different arrays to take in each individual set of values to merge them all together?
– Seb
                Oct 31, 2011 at 13:55
                You will only need 3 arrays since there are 3 color components in RGB. But yeah, you basically need to calculate the color components individually and store them somewhere.
– NickLH
                Oct 31, 2011 at 13:59

In newer version of OPENCV there is a built in function can be used to do YUV to RGB conversion

cvtColor(src,dst,CV_YUV2BGR_YUY2);

specify the YUV format after the underscore, like this CV_YUYV2BGR_xxxx

This will only work properly if the YUV source is SD. It might look right with a basic visual inspection but the colors aren't actually correct. This is because the latest OpenCV source right now (2.4.10) for YUV conversion only supports the 601 colorspace of PAL/NTSC/SECAM and not 709 of HD or 2020 of UHD. – G Huxley Mar 24, 2015 at 22:34

It looks to me like you're decoding a YUV422 stream as YUV444. Try this modification to the code you provided:

for(int i = 0, j=0; i < 1280 * 720 * 3; i+=6, j+=4)
    m_RGB->imageData[i] = pData[j] + pData[j+3]*((1 - 0.299)/0.615);
    m_RGB->imageData[i+1] = pData[j] - pData[j+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[j+3]*((0.299*(1 - 0.299))/(0.615*0.587));
    m_RGB->imageData[i+2] = pData[j] + pData[j+1]*((1 - 0.114)/0.436);
    m_RGB->imageData[i+3] = pData[j+2] + pData[j+3]*((1 - 0.299)/0.615);
    m_RGB->imageData[i+4] = pData[j+2] - pData[j+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[j+3]*((0.299*(1 - 0.299))/(0.615*0.587));
    m_RGB->imageData[i+5] = pData[j+2] + pData[j+1]*((1 - 0.114)/0.436);

I'm not sure you've got your constants correct, but at worst your colors will be off - the image should be recognizable.

You were right about the constants not being correct. I see a nice and clear image. I'll just have to research YUV422 constants to correct the issue. Thanks for this advice. – Seb Nov 1, 2011 at 13:05

I use the following C++ code using OpenCV to convert yuv data (YUV_NV21) to rgb image (BGR in OpenCV)

int main()
  const int width  = 1280;
  const int height = 800;
  std::ifstream file_in;
  file_in.open("../image_yuv_nv21_1280_800_01.raw", std::ios::binary);
  std::filebuf *p_filebuf = file_in.rdbuf();
  size_t size = p_filebuf->pubseekoff(0, std::ios::end, std::ios::in);
  p_filebuf->pubseekpos(0, std::ios::in);
  char *buf_src = new char[size];
  p_filebuf->sgetn(buf_src, size);
  cv::Mat mat_src = cv::Mat(height*1.5, width, CV_8UC1, buf_src);
  cv::Mat mat_dst = cv::Mat(height, width, CV_8UC3);
  cv::cvtColor(mat_src, mat_dst, cv::COLOR_YUV2BGR_NV21);
  cv::imwrite("yuv.png", mat_dst);
  file_in.close();
  delete []buf_src;
  return 0;

and the converted result is like the image yuv.png.

you can find the testing raw image from here and the whole project from my Github Project

It may be the wrong path, but many people (I mean, engineers) do mix YUV with YCbCr.

Try to

cvCvtColor(src, dsc, CV_YCbCr2RGB) 

or CV_YCrCb2RGB or maybe a more exotic type.

This will only work properly if the YUV source is SD. It might look right with a basic visual inspection but the colors aren't actually correct. This is because the latest OpenCV source right now (2.4.10) for YUV conversion only supports the 601 colorspace of PAL/NTSC/SECAM and not 709 of HD or 2020 of UHD. – G Huxley Mar 24, 2015 at 22:35 Opencv 2.4.* does have YUV enumerator and there is no CV_YCbCr2RGB. There is CV_YCrCb2RGB instead. – Temak Apr 27, 2015 at 8:46

The BlackMagic Intensity software return YUVY' format in bmdFormat8BitYUV, so 2 sources pixels are compressed into 4bytes - I don't think openCV's cvtColor can handle this.

You can either do it yourself, or just call the Intensity software ConvertFrame() function

edit: Y U V is normally stored as

There is a Y (brightness) for each pixel but only a U and V (colour) for every alternate pixel in the row.

So if data is an unsigned char pointing to the start of the memory as shown above.

pixel 1, Y = data[0] U = data[+1] V = data[+3]
pixel 2, Y = data[+2] U = data[+1] V = data[+3]

Then use the YUV->RGB coefficients you used in your sample code.

I looked into that function, but it doesn't work on a IDeckLinkInputFrame class. The best method would seem to do this myself. Thank you for mentioning what a YUVY' is, but would you happen to know the formula for conversion based on what you stumbled on? If you have no idea please do not go out of your way to help me. Thanks – Seb Oct 31, 2011 at 20:34

Maybe someone is confused by color models YCbCr and YUV. Opencv does not handle YCbCr. Instead it has YCrCb, and it implemented the same way as YUV in opencv.

From the opencv sources https://github.com/Itseez/opencv/blob/2.4/modules/imgproc/src/color.cpp#L3830:

case CV_BGR2YCrCb: case CV_RGB2YCrCb:
case CV_BGR2YUV: case CV_RGB2YUV:
    // ...
    // 1 if it is BGR, 0 if it is RGB
    bidx = code == CV_BGR2YCrCb || code == CV_BGR2YUV ? 0 : 2; 
    //... converting to YUV with the only difference that brings 
    //    order of Blue and Red channels (variable bidx)

But there is one more thing to say.
There is currently a bug in conversion CV_BGR2YUV and CV_RGB2YUV in OpenCV branch 2.4.* .

At present, this formula is used in implementation:

Y = 0.299B + 0.587G + 0.114R
U = 0.492(R-Y)
V = 0.877(B-Y)

What it should be (according to wikipedia):

Y = 0.299R + 0.587G + 0.114B
U = 0.492(B-Y)
V = 0.877(R-Y)

The channels Red and Blue are misplaced in the implemented formula.

Possible workaround to convert BGR->YUV while the bug is not fixed :

  cv::Mat source = cv::imread(filename, CV_LOAD_IMAGE_COLOR);
  cv::Mat yuvSource;    
  cvtColor(source, yuvSource, cv::COLOR_BGR2RGB); // rearranges B and R in the appropriate order
  cvtColor(yuvSource, yuvSource, cv::COLOR_BGR2YUV);
  // yuvSource will contain here correct image in YUV color space
        

Thanks for contributing an answer to Stack Overflow!

  • Please be sure to answer the question. Provide details and share your research!

But avoid

  • Asking for help, clarification, or responding to other answers.
  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.