I have a producer/consumer scenario where the producer is implemented as a tbb::pipeline. However due to memory constraints the pipeline isn't allowed to produce all items at once and has to wait for the consumer to consumer items.
I previously solved this by blocking the pipeline output on the master thread (seehttp://software.intel.com/en-us/forums/showthread.php?t=73642&o=a&s=lr). However this solution has some flaws,over-subscriptionand blocking. The output blocking seems to be indicated as a bottleneck by Intel Parallel Amplifier 2011 BETA.
I have now revised the general design of my system and would like to have some advice how to adapt the tbb::pipeline to this.
Instead of having a buffer (implemented with tbb::concurrent_bounded_queue) and do blocking pushes. I now use a reactive approach where I can register a callback which is called as soon as an item is removed/consumed from the buffer, and thus avoid the need of the producers being implemented as active objects and instead invoke task::enqueue through callbacks.
What I am looking for is something like:
pipeline starts n tokens with task::enqueue (where n is the size of the buffer)item is popped from buffercallback is invoked which starts a token in pipeline with task::enqueue
This way I won't need an additional thread to call pipeline::run from and which is used in processing and thus causing over-subscription. I also avoid most blocking.Is there anyway to do this with the current implementation of tbb::pipeline or will it require a new implementation?EDIT:On a side note.I remember that there was a list of projects using tbb. I would like to add casparcg (www.casparcg,com) to this list as we are soon in the process of using tbb in our stable versions and have been generally very satisfied with this library so far. Where can I find this list?