This is a fantastic post by Erik where he explains the nuance between IO-bound and CPU-bound operations in programming.

… libraries have dedicated APIs for I/O scheduling work, separate from other types of operations

…. but why is this the case? Why don’t we use a single thread pool for all background operations? The operating system will handle the scheduling of these threads the same

I love how this specific question is framed (a good interview question for advanced mobile developers):

Why can’t we just split work into:

  1. main (or UI) thread and
  2. “all other background work” thread pool?

This way the UI shouldn’t stutter and if you have a pool of threads running in the background, all that background work will conveniently be executed, no problem. Right?

Even though both I/O-bound and CPU-bound work need to be executed in the background for mobile development, they have different demands and stress the operating system (OS) differently. Hence the need for different thread pool types to handle this kind of work differently.

  • CPU operations (like number crunching, computation etc.) run incredibly fast
  • I/O operations on the other hand (reading from a database, network etc.) take longer as the OS has to wait for underlying hardware to retrieve and deliver the data
  • Typically the underlying hardware sends out a signal IOWait to the OS while the underlying hardware is out and about doing the IO work
  • Instead of sitting and twiddling its thumbs, the OS should be finishing other tasks in the meantime (especially ones that are not waiting on underlying hardware) through other threads

When the operating system encounters an IOwait, it pauses the execution of that thread and is free to pick up another thread that is ready to run. Theoretically, an operating system can have an unlimited number of threads that are sleeping due to IOwait, without the user noticing any difference

That last point is subtle: this is why you don’t want to mix CPU and I/O threads in the same pool. You could potentially have a whole bunch of them IOWaiting and precious CPU work that could otherwise be done in the meantime, will also hang in the process waiting for that I/O work. Armed with this knowledge, if you’re into Rx you might ask:

How many maximum threads can the IO Scheduler have?

Unbounded! This should also makes sense now since we don’t want one I/O thread ever waiting on another since IOWaiting can take quite some time. Note that this is still a “pool” so threads are recycled but if all them are currently doing some work, Rx will spawn a new one everytime.

How many maximum threads can the Computation Scheduler spawn?

Basically, the number of processors your CPU has. My Pixel for example has a quad core processor, so that’s 4. This should also make sense now because CPU work should run blazingly fast, you probably don’t want an unbounded pool but some kind of optimum upper limit. That upper limit is the number of cores you have handy.

Erik’s a really smart guy. You should follow him on Twitter.