Ah, Vista!
In Vista MS moved ROT-related functionality from quartz.dll to proppage.dll, so you need to locate and register (using regsvr32) proppage.dll to be able to work with ROT.
A quote from the internets:
In order ... to work on Windows Vista, you actually need to have the version of GraphEdit from the Windows SDK for Windows Vista and register the proppage.dll that comes with it. If you have the older version of GraphEdit from the previous version of the Windows SDK, it still won't work. It has to be the one from the Windows SDK for Vista. The reason for this is that Microsoft (in their infinite wisdom) removed almost all the proxy/stub pairs from Quartz.dll in Vista and moved them all into the proppage.dll file. Without the proxy/stub pairs registered, Windows can't marshal proxies from one thread apartment to another.