How CPU Cores, Threads, and Coroutines Work Together – A Real-Life Case Study

How CPU, Threads, and Coroutines Work Together
Photo by Luan Gjokaj on Unsplash and added text over it

Introduction

Knowing the relationship of CPU cores, threads, and coroutines is fundamental for maximizing performance in concurrent programming. Using a practical case study, this post will help us to simplify these ideas.

At the close of this post, you will grasping:

  • How CPU cores and threads handle concurrent tasks
  • How coroutines leverage threads efficiently
  • Why coroutines don’t necessarily block execution, even if some tasks are long-running

Let’s dive in! 🚀

Understanding the System Setup

Imagine we have a system with the following configuration:

  • Dual-core CPU with Hyper-Threading:
    • 2 Physical Cores
    • 4 Logical Threads (since each core can handle 2 threads simultaneously)
  • Task to execute: 12 coroutines
    • 8 CPU-bound (compute-intensive, running for 5 minutes each, non-suspending)
    • 4 I/O-bound (may suspend due to network or file operations)

To better understand how CPU cores and threads work, check out our detailed article: CPU Cores vs Threads: Key Differences Explained

Step-by-Step Execution Process

Let’s explore how CPU, threads, and coroutines work together in handling these 12 coroutines.

Launching the Coroutines

  • We launch all 12 tasks (coroutines) at once.
  • The Kotlin coroutine dispatcher (Dispatchers.Default) assigns tasks to available threads.
  • Since our system has 4 logical threads, it can only run 4 tasks at the same time.

Dispatcher Assigns Threads

  • The first four tasks are distributed to the four threads (T1, T2, T3, T4).
  • The leftover eight tasks stand in a queue.

Execution Begins

Each thread starts executing a coroutine. Here’s the mix:

  • CPU-bound tasks keep running for 5 minutes, using up CPU resources.
  • I/O-bound tasks may suspend, freeing up their threads.

Handling CPU-bound Coroutines

  • These do not suspend, meaning they continuously occupy their assigned threads.
  • This creates a bottleneck: only 4 CPU-bound tasks run simultaneously.
  • The remaining 4 CPU-bound tasks must wait until a thread is available.

Handling I/O-bound Coroutines

  • These may suspend (e.g., waiting for a network response).
  • When a task suspends, its thread is released.
  • The dispatcher then assigns this freed-up thread to another task from the queue.

How Coroutines Ensure Fair Execution

One major question arises:

If the first 4 coroutines are non-suspending CPU-bound tasks, won’t they block the remaining 8 coroutines from running? OR Will I/O tasks execute only after CPU-bound tasks finish?

No, They will still get CPU time, but they can experience delays because of the aggressiveness of CPU bound tasks. The operating system (OS) scheduler is in charge of CPU time and distributes it to tasks in small time slices. While CPU bound tasks are greedy and will run for as long as possible, the OS still allows I/O bound tasks to execute for some amount of time.

How the OS Scheduler Works:

  • The OS uses time-sharing (task switching) to fairly distribute CPU time.
  • Each task gets a small time slice (usually a few milliseconds) which the CPU switches between without pause.
  • If a task pauses (like I/O bound tasks do while waiting for data), the CPU immediately switches to another task.

Understanding Coroutine Scheduling

  1. Coroutines Are Not Threads
    • Unlike traditional threads, multiple coroutines can run on a single thread.
    • Kotlin’s coroutine system is designed to share CPU time fairly.
  2. Time-Slicing & Fair Scheduling
    • The dispatcher ensures fair execution by preempting long running coroutines.
    • It pauses the first 4 coroutines temporarily even if they are CPU bound and non suspending to let other coroutines run.
  3. Thread Reallocation
    • Whenever an I/O bound coroutine suspends, the dispatcher takes the thread from it and gives it to a waiting coroutine.
    • That way, though some of the coroutines are still running in the first batch, the CPU bound coroutines can keep executing.
    • This enables the remaining CPU bound coroutines to run even when the first set of coroutines are still executing.

What Happens in Our Case?

You have 4 CPU threads and 12 tasks competing:

  • 8 CPU-bound tasks → Never pause, so they always want CPU.
  • 4 I/O-bound tasks → Suspend occasionally (waiting for network or file I/O).

Since the CPU cannot run more than 4 tasks at a time, the OS rapidly switches between tasks.

  1. At any given moment, 4 tasks are running.
  2. I/O-bound tasks will run whenever they get a chance.
    • If an I/O-bound task needs CPU, the OS will temporarily pause a CPU-bound task.
    • If an I/O-bound task is waiting (suspended), it does not take CPU time.
  3. CPU-bound tasks still dominate execution.
    • They take up most of the CPU time.
    • I/O-bound tasks might get delayed but will still execute.

How the 12 Coroutines Complete Execution

Total Execution Time Breakdown

  1. First 4 CPU-bound tasks finish in 5 minutes.
  2. The next batch of 4 CPU-bound tasks starts, finishing at 10 minutes.
  3. I/O-bound tasks suspend and resume, but they complete within ~10 minutes too.
  4. By 10 minutes, all tasks (coroutines) have completed execution.

What Problems Can Happen?

  • I/O-bound tasks experience delays → Because CPU-bound tasks keep running.
  • If all 4 CPU threads are busy, an I/O-bound task must wait for a time slice.
  • If the OS gives priority to CPU-bound tasks, I/O-bound tasks may lag behind more than usual.

I hope this article will help you to understand the coordination between CPU Cores, Threads and Coroutines, to execute tasks.

Share this guide or post a remark with your thoughts and questions if you found it beneficial. The technology discussion should be ongoing.

Till next time, Rajat Sarangal.

February 23, 2025
Share