Techniques coupled in with super resolution?

[Dee Mon]

Thanks for the link!

This article is in line with many research works: a lot of good math, very nice pictures and very far from practical use.

On Page 6 of their article PDF:

Computational performance. Our C++ implementation
takes about two hours on an Intel Core i7 Q820 workstation
with 16 GB RAMs when super resolving a 720×480
frame using 30 adjacent frames at an up-sampling factor of
4. The computational bottleneck is solving the optical flow
equation in Eqn. 13, which takes about one minute for a
pair of high-res and low-res frames.

2 hours for 1 frame! How do you like it?

[Henery]

Ok,i admit,i didn´t read the whole PDF and i knew it would be slow.But not that it was that slow! After reading that,even the Neural Spline sounds faster.I have to compare them to find out.

MotionDSP´s Ikena is only for law enforcement use and it costs 7000!! And the quality is maybe a little bit better than in VReveal.

I have one idea which would increase VE´s quality.Even the most advanced image scalers take maximum of 16 sampling pixels to interpolate the new pixel.Teranex broadcasting hardwares have HQV´s http://www.hqv.com/index.cfm?page=tech.scaler content adaptive scaler which uses 1024 sampling points (taps).Thats why it can maintain high frequency detail and edges much better.I know this makes processing slower,but i believe this would increase VE´s spatial upscaling quality.But i dont know if it works properly,or at all with block matching motion compensation.

[Henery]

I have found again a new image and video upscaling method and this time the algorithm is fast.It´s called "Image and Video Upscaling from Local Self-Examples" http://www.cs.huji.ac.il/~raananf/projects/lss_upscale/index.html Make sure to watch those examples and PDF.

At the end of the article:
"We implemented our method in C++ and run it on an Intel Core 2 Quad CPU 2.8GHz machine."
"It takes us 4 seconds to upscale an image of 200-by-200 pixels by a factor of 4 along each axis when we run on a single core."
And about the GPU implementation they say "This implementation allows us to upscale videos form 640x360 to 1920x1080 (a factor of 3) at 23.9 FPS (including display time)."

Results look good and it´s fast,almost realtime.Sounds very exciting.I really would like to see how this looks combined with motion estimation and multiframe SR.

[Dee Mon]

Thank you for the link, looks very interesting!

Update: I've read the paper and really like the idea. But I think it should  only be used on edges. There's no sense in using it in flat smooth areas, and on sharp textures it produces unwanted artifacts. I think we could detect edges and apply this method there while using our current upscaling technique in other areas. (all this on the step of spatial upscaling, before fusing with motion-compensated previous frame)

[Henery]

I have lately dug around the net finding new methods and ideas for super-resolution and motion estimation.

1) Phase Correlation Motion Estimation:
http://scien.stanford.edu/pages/labsite/2000/ee392j/projects/liang_report.pdf

Phase correlation methods seem to be computationally much efficient than block matching motion estimation.In the article it is said that phase correlation works better when there is large scale translational motion.Block matching algorithms work better on regular and small scale motion.I was thinking if it would be possible to combine this method to work with block matching? Block mathing would only handle small and regular motion while phase correlation processes only for large scale motion.


2) A Modular Approach to Image Super-Resolution Algorithms:
http://www.kbs.twi.tudelft.nl/docs/MSc/2006/vanOuwerkerk/thesis.pdf



Idea here is to break several super-resolution and image enlargement methods to algorithmic parts and then use them together,resulting in much higher quality scaling.I think this method´s speed depends on what algorithms are used so it may be quite slow.

3) Traditional super resolution methods use either single or multiple images to interpolate the higher resolution result.I was thinking if we could use large bank of high quality images or patterns to get better upscale results? SONY 4K VPL-VW1000ES projector uses this kind of technology. Take a look at this video at 3:30 to better understand what i mean:
http://www.youtube.com/watch?v=u6F-GHT7YyEfeature=related

At the moment VE is slow,but maybe in future when it is faster and the GPU support is (hopefully) added this would be possible.

I hope these suggestions will be inspiring.

Edit: That sonys projector upscaling method uses something i believe is called "statistical learning based pattern recognition",and it is super-resolution too

[Manoman455]

Was looking around for an anti-aliasing filter for avisynth that's better then Sangnom and stumbled upon this:
http://forum.doom9.org/showpost.php?p=808013&postcount=19
There isn't much information on it but there is some info at the bottom.
Apparently it is a combination of eedi3 and temporal super resolution.