Infognition forum
February 08, 2012, 09:32:58 PM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Last GraphEditPlus version: 1.4.0   Last Video Enhancer version: 1.9.7
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: USB Video Capture and graphic overlay - newbie warning  (Read 1092 times)
matterama
Newbie
*

Karma: +0/-0
Posts: 5


View Profile
« on: August 10, 2010, 10:52:33 PM »

Hi all,
We have an application that uses a USB video capture device (Belkin) to acquire a stream from an analog camera output.  We want to draw some simple graphic over the display, namely a crosshair that can be moved around.  The idea is that I could just create a bitmap and mix it into the stream.  Well, that's my idea...

I can (using Graph Edit Plus) create a filter that has the following:
USB Capture->AVI Decompressor->OverlayMixer(input 0)->Video Renderer

But I cannot get anything to connect up to the second input of the Overlay Mixer.  I see in the Windows help a line saying the Overlay Mixer isn't used for USB capture devices, but I don't know if that is just a recommendation or what.

The filter described above works to display the video correctly.  But I am at a loss of how to connect up a secondary filter that lets me draw something simple on the display.

Does anyone have any suggestions / ideas / samples out there for this kind of situation?

Thanks!
Logged
Dee Mon
Administrator
Hero Member
*****

Karma: +8/-0
Posts: 512



View Profile WWW
« Reply #1 on: August 11, 2010, 09:53:57 AM »

I'm not sure Overlay Mixer is the right filter for this task. If you need to employ video card's ability to blend images it's better to use Video Mixing Renderer 9, but the simplest solution will be to just insert a Sample Grabber between AVI Decompressor and Video Renderer, setup a callback (see ISampleGrabberCB interface) and in SampleCB method just draw the reticle in provided video data.
Logged
matterama
Newbie
*

Karma: +0/-0
Posts: 5


View Profile
« Reply #2 on: August 11, 2010, 10:09:02 AM »

Hi Dee Mon,

This is used on older platforms with unknown video hardware, so I am staying away from VMR9. 

I like the sample grabber concept, and its what I thought I should be heading towards.  Couple questions:

1) Can you point at a sample / demo code of doing using the ISampleGrabberCB interface callback?
2) Does the sample grabber live in the filter graph and get attached the video renderer?  So is it only a situation where I overlay on that video data or do I have to worry about clocking / timing also?

Thanks!
Logged
Dee Mon
Administrator
Hero Member
*****

Karma: +8/-0
Posts: 512



View Profile WWW
« Reply #3 on: August 11, 2010, 11:24:21 AM »

First, declare a class inherited from ISampleGrabberCB:
Code:
class CMyCB : public ISampleGrabberCB {
public:
    STDMETHODIMP QueryInterface(REFIID riid, void **ppv) {     
if (NULL == ppv) return E_POINTER;
        if (riid == __uuidof(IUnknown))
        {
            *ppv = static_cast<IUnknown*>(this);
             return S_OK;
        }
        if (riid == __uuidof(ISampleGrabberCB))
        {
            *ppv = static_cast<ISampleGrabberCB*>(this);
             return S_OK;
        }
        return E_NOINTERFACE;
    };                                                         
    STDMETHODIMP_(ULONG) AddRef() {                             
        return S_OK;
    };                                                         
    STDMETHODIMP_(ULONG) Release() {                           
        return S_OK;
    };

//ISampleGrabberCB
    STDMETHODIMP SampleCB(double SampleTime, IMediaSample *pSample);
    STDMETHODIMP BufferCB(double SampleTime, BYTE *pBuffer, long BufferLen) { return S_OK; };
};

Then add drawing logic to SampleCB method:
Code:
STDMETHODIMP CMyCB::SampleCB(double SampleTime, IMediaSample *pSample)
{
if (!pSample) return E_POINTER;
long sz = pSample->GetActualDataLength();
BYTE *pBuf = NULL;
pSample->GetPointer(&pBuf);
        //draw to pBuf here...
return NOERROR;
}

Create an object of that class, let's name it pMyCB;

When you build your graph add a Sample Grabber:
Code:
        CComPtr<IBaseFilter> pGrabber;
        pGrabber.CoCreateInstance(CLSID_SampleGrabber);
pGraph->AddFilter(pGrabber, L"Grabber");
CComQIPtr<ISampleGrabber> pSG( pGrabber );
AM_MEDIA_TYPE mt;
ZeroMemory(&mt, sizeof(AM_MEDIA_TYPE));
mt.majortype = MEDIATYPE_Video;
mt.subtype = MEDIASUBTYPE_RGB32; //or what colorspace you work in
pSG->SetMediaType(&mt);
pSG->SetCallback(pMyCB, 0);
Then connect decoder to sample grabber and sample grabber to video renderer.

After you run the graph for each video frame SampleCB method will be called and pBuf will point to array of width * height * 4 bytes: blue, green, red and alpha (unused) for each pixel. If your device provides video in another colorspace (like YUY2) and you don't want to waste time on conversion then change mt.subtype accordingly and learn how pixels are stored in YUY2 (a bit more complicated than RGB32).

No worries about timing. Your callback receives sample time (elapsed since starting the graph) in seconds as a double. You can also query pSample for additional info like its mediatype, including width and height.
Just one note: only the first sample will contain mediatype, so if you need this info get it from the first one and remember.
Logged
matterama
Newbie
*

Karma: +0/-0
Posts: 5


View Profile
« Reply #4 on: August 11, 2010, 06:35:00 PM »

Wow!  Thanks very much!  I'lll give this a try and let you know how it works out!   Thank you again!
Logged
matterama
Newbie
*

Karma: +0/-0
Posts: 5


View Profile
« Reply #5 on: August 19, 2010, 12:43:50 AM »

Hi,

Everything seems like it should be working.  I implemented your code and everything looke good.  However, when I tried to connect the sample grabber and avi decompessor, it failed.  The sequence I use (with Graph Edit's help) is:

1) create graph (success)
2) add video capture (success) (Note: used both USB video capture and QuickCam pro USB webcam with identical results)
3) add sample grabber to graph (success)
4) connect video capture to sample grabber (success)
5) add avi decompressor (success)
6) connect sample grabber to avi decompressor (FAIL - DX error code 8004003 - "Unknown")
7) add video renderer (success if I skip the above step)
Cool connect avi decompressor to video renderer (haven't gotten here)

So then, I created a filter in Graph Edit plus that doesn't do the callbacks of course, but shows that samples are being grabbed and video is rendered end to end, and it works great when run out of GE plus.  Taking the code that was generated from that and moving it into our project directly resulted in the above fail at the same step (grabber to avi decompressor).  So we may be doing something in our pojrect that kilss this ability, or we don't have something that the avi decompressor or grabber needs maybe?

Do you have any hints?  The 8004003 code wasn't very helpful...

Thanks!

BTW, the exact line that fails is:

    //connect QuickCam for Notebooks Pro and SampleGrabber
    hr = pGraph->ConnectDirect(GetPin(pQuickCamforNotebooksPro, L"Capture"), GetPin(pSampleGrabber, L"Input"), NULL);
Logged
Dee Mon
Administrator
Hero Member
*****

Karma: +8/-0
Posts: 512



View Profile WWW
« Reply #6 on: August 19, 2010, 09:13:59 AM »

Hi!

First, there is inconsistency in your words: does the fail occur in point 4 or point 6? In your list point 6 is indicated as failing but the source code line quoted below is point 4.

If you can't connect camera and sample grabber in your project but can do it in GEP, it probably means too restrictive media type requirements you set in ISampleGrabber::SetMediaType(). Just set majortype and subtype, as quoted above, not the whole mediatype as in generated code from GEP.

Next, you don't really need to manually insert AVI Decompressor. I understand the video goes through sample grabber in already uncompressed format (like RGB32, YUY2 etc.). If you want to render it in a window you can just call RenderStream with source filter being sample grabber and destination being NULL. It will create, add to graph and connect video renderer and any necessary filters automatically. It will most probably insert AVI Decompressor for converting from your current video format to the one acceptable by video renderer.

Another way, if you want a different video renderer filter, you can add it to graph manually and then connect via RenderStream sample grabber and video renderer. Video format converter (if necessary) like AVI Decompressor will be inserted automatically.

To see how to use RenderStream go to GEP Preferences (from Options menu) and select "use RenderStream" in code generation part, then re-generate code for your graph.
« Last Edit: August 19, 2010, 09:16:51 AM by Dee Mon » Logged
matterama
Newbie
*

Karma: +0/-0
Posts: 5


View Profile
« Reply #7 on: August 19, 2010, 09:22:21 AM »

Ah,you are right, I mistakenly posted the wrong code snip.  The fail is on step 6.

I'll work on it with your comments, thank you again for your support.  I also appreciate the comments on the Render Stream, as the application we are modifying uses that method.  I wasn't aware GEP had that option.

Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.13 | SMF © 2006-2011, Simple Machines LLC Valid XHTML 1.0! Valid CSS!