CPU Utilization

[KrishnaPG]

I am not able to build the code, but from the initial scans, it looks like you are using Lock-free constructs for the queuing.

Lock-free constructs may help in heavy throughput scenarios, where continuous streams are available for hand-off. But most of the times, in production, load comes sporadically. In such cases (where load is less 80% of the time), the threads will be ticking unnecessarily (due to their lock free nature).

A more reasonable approach in such cases would be, to gracefully degrade from lock-free waiting to locked-waiting (if no new work-item or message appears for few iterations). Please give it a thought.

Thanks.

[rohitjoshi]

@KrishnaPG thanks for your suggestion. Defiantly a great idea. I will try to add support during these holidays but would love to have your PR :)