A line of jobs

Think of it when work arrives faster than it can be done, or a slow job should not hold up the person who asked for it.

In everyday life

The ticket dispenser at a deli counter: customers take a number, and staff serve one number at a time. A rush makes the wait longer, but nobody is turned away.

One case

A bank emails 6,000 monthly statements at 08:00, and its mail provider accepts 1,000 a minute. With no line to wait in, the bank sends them all at once; the provider takes 1,000 and rejects the other 5,000. With a queue, a worker sends 1,000 a minute and the last statement goes out at 08:06.

The line turned a burst into six steady minutes
The case, cell by cell1 cell = 1 minute
Statements sent, with a queue
100020003000400050006000
Statements sent, without one
100010001000100010001000
Statements rejected
5,000
Minutes to clear the line
6
6,000 statements at 08:00. The provider accepts 1,000 a minute.

Every case

Work rarely arrives at an even pace. A queue absorbs the bursts: the sender adds a job and moves on, and each workerA program, or one part of a program, that takes one job at a time and does it. takes the next job when it is free. A crashed worker loses nothing when three things hold: the queue keeps its jobs on disk, a worker confirms a job only after it finishes, and the queue hands out unconfirmed jobs again. The price is delay, and a job can be handed out twice, so every job must be safe to run twice.

The word

Engineers call this

queueA line of jobs waiting to be done. One program adds jobs to the back. Workers take jobs from the front, at the pace they can handle.

Put slow or bursty work on a queue. Watch how long the line gets, and make every job safe to run twice.

Where this idea leads

Skills that encode it

  • principle-make-operations-idempotent pstackA “claimed job” left behind by a crash must be recognizable as stale, so the next run converges safely.