nrml
Simulation and online learningWayfair

Multi-Armed Bandit: Explore, Observe, Repeat

Repair UCB1 selection and build a repeatable Bernoulli reward simulation.

The scenario

Repair UCB1 selection and build a repeatable Bernoulli reward simulation.

A failure to reason about

Use a small simulation and online learning scenario to compare current behavior with the requirement.

Illustrative context for the topic. The exercise’s full requirements arrive in the workspace.

Questions to bring to your review

  • Which caller makes the reported behavior visible?
  • What focused regression would distinguish the current behavior from the requirement?
  • What existing behavior must remain intact after the change?

Skills you will practice

  • Exploration and exploitation
  • State invariants
  • Probability
  • Reproducible testing

Read the existing code, reproduce the reported behavior, review proposed changes, and verify your final implementation. You can use AI assistance while staying responsible for the result.

Prepare with a worked example

See how nrml evaluates an attempt, including the limits of automated evidence.