I am building a CD ripper application in C++ and Qt. I would like to parallelize the application such that multiple tracks can be encoded concurrently. Therefore, I have structured the application in such a way that encoding a track is a "Task", and I'm working on a mechanism to run some number of these Tasks concurrently. I could, of course, accomplish this using threads and write my own Task queue or work manager, but I thought TBB might be a better tool for the job. I have a couple of questions, however.
- Is encoding a WAV file into a FLAC, Ogg Vorbis, or Mp3 file something that would work well as a tbb::task? The tutorial document states that "if threads block frequently, there is a performance loss when using the task scheduler because". I don't think my encoding tasks would block for mutexes frequently, but the will need to access disk relatively frequently, since they must read the WAV data from disk in order to encode. Is this level of disk activity problematic in the sense described by the tutorial?
- Does TBB work well with Qt? When using Qt threads, you can use Qt's signals/slots mechanism transparently across threads. Would the same be true if I were using tbb::tasks instead of Qt threads? Would there be any other "gotchas"?
Thanks for any insights you can provide.
Hi,
I'm happy to see that someone has started to integrate Qt applications with TBB. Hopefully your application is open source too, at which point I'll be very very happy. I'll try to help you as much as I can.
Let's start with your first question regarding encoding as a tbb::task. First, you are right that blocking a TBB task is a no-no. tbb::tasks are there for computational work, which in your case will be computation for the encoding. I'm not sure how hard encoding is, computationally-speaking. The tasks should represent fairly small units of computation for load balancing, the smaller the units are the better the load balancing will work.
I'll try to illustrate this with an example. You described encoding an entire track as a tbb::task. Let's say you have 4 tracks, and 4 processors. Suppose that 3 of these tracks are fairly short, and finish quickly. The scheduling algorithm cannot break up the work any further, so you will have 3 idle cores while a single core bears all the computational work. Sometimes you will get a speed-up, sometimes not. Now, suppose that instead you break up work to represent encoding say two seconds of track (I'm not sure how encoding works, so forgive me if this isn't right). Each tbb::task would include an id for the 2 second segment, say the track number it belongs to and the offset of the 2 seconds into the track. Now you break up encoding of tracks into many many 2s segments... this eliminates the former problem that occurs when one track is longer than others.
Now we have the I/O problem. You will have to determine if your problem is I/O bound, or computationally bound. The tbb::pipeline seems like it could help you with this problem. You could setup a pipeline where you read a segment of memory, encode it, then write it back to disk.
Finally, Qt threads. What are you using the Qt threads for? I am assuming this is for the graphical interface only? It would likely be best to just link your encoding module separately, so that it can manage its own threading via TBB. That is to say, write your encoder as a separate module in C++ with TBB, and define an external inteface to the module so that threading is entirely hidden. Then you can call this module from your Qt application so that it does not know it is even calling a threaded program. Technically this could lead to oversubscription, which happens when there are more threads than processors. Whether or not this is a problem depends entirely upon how and why you are using threads.
I hope that this is a satisfactory response. Currently I am traveling in Europe. I should have access to the Internet for the next few days, and after that I will disappear until I get back to Canada on the 27th.
AJ
