threadpool.h File Reference

Work-stealing thread pool for concurrent task execution. More...

#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>

Include dependency graph for threadpool.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
struct	Task
	A unit of work submitted to the pool. More...

Typedefs
typedef struct Threadpool	Threadpool
	Opaque handle to a thread pool instance.
typedef struct Task	Task
	A unit of work submitted to the pool.

Functions
Threadpool *	threadpool_create (size_t num_threads)
	Create a new thread pool.
bool	threadpool_submit (Threadpool *pool, void(*function)(void *), void *arg)
	Submit a single task to the pool.
size_t	threadpool_submit_batch (Threadpool *pool, void(**functions)(void *), void **args, size_t count)
	Submit multiple tasks to the pool in a single call.
void	threadpool_wait (Threadpool *pool)
	Block until all currently submitted tasks have completed.
void	threadpool_destroy (Threadpool *pool, int timeout_ms)
	Drain all pending tasks, stop all workers, and free the pool.

Detailed Description

Work-stealing thread pool for concurrent task execution.

This thread pool uses the Chase-Lev (2005) work-stealing deque algorithm with the Le et al. (2013) correction for the last-item ABA hazard.

Architecture

Each worker thread owns a private double-ended queue (deque):

• The owner pushes and pops tasks at the bottom end using a single release store — no CAS, no lock, no contention with other threads.
• Thieves (idle workers) steal one task at a time from the top end using a single seq_cst CAS.

External submitters (threads that are not pool workers) push tasks into a shared GlobalQueue protected by a mutex. Workers drain this queue in batches of BATCH_SIZE tasks per mutex acquisition, then distribute the extras into their own private deques for subsequent lock-free consumption.

External submitter
     │
     ▼
[ GlobalQueue ]  ──batch drain──▶  worker[0] deque  ◀── steal ── worker[1]
                                   worker[1] deque  ◀── steal ── worker[2]
                                   ...

Threading model

Operation	Cost
Worker push/pop (own deque)	One release store — zero contention
Thief steal	One seq_cst CAS — contended only between thieves
External submit (single)	One mutex round-trip per task
External submit (batch)	One mutex round-trip per GLOBAL_Q_SIZE tasks
Worker drain from global	One mutex round-trip per BATCH_SIZE tasks

Compile-time knobs

Internal constants (defined in threadpool.c, not overridable from the header):

Constant	Value	Meaning
DEQUE_SIZE	4096	Slots per private Chase-Lev deque
GLOBAL_Q_SIZE	16384	Slots in the shared external submission queue
BATCH_SIZE	64	Tasks pulled from the global queue per mutex acquisition
YIELD_THRESHOLD	8	Spin rounds before a worker parks on a condvar

Lifecycle

Threadpool* pool = threadpool_create(8);
 
// Single-task submission
threadpool_submit(pool, my_fn, my_arg);
 
// Batch submission (preferred for large workloads)
void (*fns[N])(void*) = { ... };
void*  args[N]        = { ... };
threadpool_submit_batch(pool, fns, args, N);
 
// Wait for all current tasks to complete (pool remains usable)
threadpool_wait(pool);
 
// Tear down
threadpool_destroy(pool, -1);

Thread safety

All public functions are safe to call from any thread concurrently, including from within a running task. A task may call threadpool_submit or threadpool_submit_batch to enqueue follow-on work; in that case the task is treated as a worker thread and pushes directly into its own private deque at zero contention cost.

Definition in file threadpool.h.

Typedef Documentation

Task

typedef struct Task Task

A unit of work submitted to the pool.

Tasks are value types — they are copied into the pool's internal queues on submission. The caller does not need to keep the Task object alive after threadpool_submit or threadpool_submit_batch returns.

Threadpool

typedef struct Threadpool Threadpool

Opaque handle to a thread pool instance.

Obtain via threadpool_create(). All fields are private; interact with the pool exclusively through the functions declared in this header.

Definition at line 99 of file threadpool.h.

Function Documentation

threadpool_create()

Threadpool * threadpool_create

(

size_t

num_threads

)

Create a new thread pool.

Spawns num_threads worker threads and returns a handle to the pool. Workers enter a startup barrier and do not begin stealing or executing tasks until every worker slot in the internal array has been initialised — this prevents a race where an early-starting worker dereferences a NULL pointer while the creation loop is still running.

Parameters

num_threads

Number of worker threads to create. Clamped to 1 if 0 is passed.

Returns

Pointer to the new pool, or NULL if memory allocation or thread creation fails. On partial failure all successfully created threads are joined and all memory is freed before returning NULL.

Note

The returned pointer must be freed with threadpool_destroy().

Complexity

O(num_threads) — one pthread_create per worker.

Definition at line 597 of file threadpool.c.

References cond_init(), lock_init(), threadpool_create(), and threadpool_destroy().

Referenced by threadpool_create().

Here is the call graph for this function:

Here is the caller graph for this function:

threadpool_destroy()

void threadpool_destroy	(	Threadpool *	pool,
		int	timeout_ms
	)

Drain all pending tasks, stop all workers, and free the pool.

Shutdown sequence:

1. Acquires idle_lock and waits until all queues are empty and num_active reaches zero (subject to timeout_ms).
2. Sets the shutdown flag atomically while still holding idle_lock, preventing workers from re-entering their park condvar between the flag store and the subsequent broadcast.
3. Broadcasts on work_available to wake every parked worker.
4. Spin-waits for num_threads_alive to reach zero.
5. Joins every worker thread, frees per-worker memory, destroys all synchronisation objects, and frees the pool struct.

The pool pointer is invalid after this function returns.

Parameters

pool	Pool to destroy. If NULL the function returns immediately without error.
timeout_ms	Maximum milliseconds to wait for in-flight tasks to complete before proceeding with shutdown. Pass -1 to wait indefinitely (recommended for correctness; use a positive value only when a hard deadline is required and task cancellation is acceptable).

Warning

Calling threadpool_submit or threadpool_submit_batch after threadpool_destroy is undefined behaviour — the pool memory has been freed.

Do not call threadpool_destroy from within a running task. The num_active counter will never reach zero while the calling task is still executing, causing an infinite wait (with timeout_ms == -1) or a forceful shutdown that frees memory still on the call stack.

Definition at line 774 of file threadpool.c.

References cond_free(), cond_wait_timeout(), lock_acquire(), lock_free(), lock_release(), thread_join(), and threadpool_destroy().

Referenced by threadpool_create(), and threadpool_destroy().

Here is the call graph for this function:

Here is the caller graph for this function:

threadpool_submit()

bool threadpool_submit	(	Threadpool *	pool,
		void(*)(void *)	function,
		void *	arg
	)

Submit a single task to the pool.

Determines the submission path based on the calling thread's identity:

• Worker thread (a thread created by this pool): pushes the task directly into the caller's own Chase-Lev deque bottom with a single release store — zero lock contention, zero shared cache-line traffic. Falls back to the global queue if the private deque is full (capacity 4096 slots; this path is not reached under normal load).
• External thread (any other thread): pushes the task into the shared GlobalQueue under a mutex. Costs one pthread_mutex_lock + pthread_mutex_unlock round-trip. Blocks if the global queue is full (capacity 16384 slots) until a worker drains a slot.

After pushing, wakes one parked worker if any are sleeping.

Parameters

pool	Pool to submit to. Must not be NULL.
function	Task function. Must not be NULL.
arg	Argument passed verbatim to function. May be NULL.

Returns

true if the task was enqueued, false if pool or function is NULL or if the pool is shutting down.

Note

For bulk external submission prefer threadpool_submit_batch, which amortises the mutex cost across many tasks.

Complexity

O(1) amortised.

Definition at line 656 of file threadpool.c.

References threadpool_submit().

Referenced by threadpool_submit().

Here is the call graph for this function:

Here is the caller graph for this function:

threadpool_submit_batch()

size_t threadpool_submit_batch	(	Threadpool *	pool,
		void(**)(void *)	functions,
		void **	args,
		size_t	count
	)

Submit multiple tasks to the pool in a single call.

Reduces mutex acquisition overhead from O(count) to O(count / GLOBAL_Q_SIZE) compared to calling threadpool_submit in a loop. For a batch of 10 000 tasks with GLOBAL_Q_SIZE = 16384, this is a single mutex acquisition instead of 10 000.

Submission paths

Worker thread: iterates functions and pushes each task into the caller's own deque with deque_push_bottom (lock-free). Any tasks that do not fit in the private deque (overflow past 4096 slots) are forwarded to the global queue via gq_push_batch.

External thread: builds a flat Task array on the stack (if count ≤ BATCH_SIZE) or heap, then hands it to gq_push_batch which holds gq_mutex for the duration of each chunk write. If the global queue fills up mid-batch, gq_push_batch waits on not_full and resumes from where it left off — the caller blocks until all count tasks are enqueued or the pool shuts down.

Parameters

pool	Pool to submit to. Must not be NULL.
functions	Array of count function pointers. NULL entries are silently skipped and do not count toward the return value.
args	Array of count argument pointers, or NULL to pass NULL as the argument to every task.
count	Number of entries in functions (and args if non-NULL).

Returns

Number of tasks successfully enqueued. Equal to the number of non-NULL entries in functions under normal operation. May be less than count if the pool begins shutting down mid-batch; the caller should treat a short return as a fatal error. Returns 0 if pool is NULL, functions is NULL, or count is 0.

Example

#define N 10000
void (*fns[N])(void*);
void*  args_arr[N];
for (int i = 0; i < N; i++) { fns[i] = process; args_arr[i] = &items[i]; }
 
size_t submitted = threadpool_submit_batch(pool, fns, args_arr, N);
assert(submitted == N);

Complexity

O(count / GLOBAL_Q_SIZE) mutex acquisitions for external threads; O(count) deque stores (lock-free) for worker threads.

Definition at line 690 of file threadpool.c.

References threadpool_submit_batch().

Referenced by threadpool_submit_batch().

Here is the call graph for this function:

Here is the caller graph for this function:

threadpool_wait()

void threadpool_wait

(

Threadpool *

pool

)

Block until all currently submitted tasks have completed.

Returns only when both conditions hold simultaneously:

1. Every worker's private deque is empty.
2. The global submission queue is empty.
3. No worker is currently executing a task (num_active == 0).

Condition 3 ensures correctness when a task itself calls threadpool_submit or threadpool_submit_batch — the pool is not considered idle until those follow-on tasks have also completed.

The pool remains fully operational after threadpool_wait returns. New tasks may be submitted immediately.

Parameters

pool	Pool to wait on. If NULL the function returns immediately.

Note

Do not call threadpool_wait from within a running task. The task holds a slot in num_active, so the wait condition can never be satisfied while that task is still on the call stack — this will deadlock.

Typical usage

// Phase 1
threadpool_submit_batch(pool, fns1, args1, N1);
threadpool_wait(pool);
process_phase1_results();
 
// Phase 2 — pool is still alive and ready
threadpool_submit_batch(pool, fns2, args2, N2);
threadpool_wait(pool);
 
threadpool_destroy(pool, -1);

Definition at line 750 of file threadpool.c.

References cond_wait(), lock_acquire(), lock_release(), and threadpool_wait().

Referenced by threadpool_wait().

Here is the call graph for this function:

Here is the caller graph for this function: