Five minutes. One change. A fair comparison. Something you can learn.
Trying something is only the start.
You change a routine. The next day goes better. Did the change help? Perhaps. But the task, your energy, or the people around you may also have changed.
An experiment gives a question a fair chance of being answered. You choose a change, compare conditions, and record what happens. The result may support your idea, weaken it, or leave the question open.
Choose the question. Make a comparison.Keep the result.
What would "better" look like?
Suppose you wonder whether silencing your phone would help you stay with a task. Start with a narrow question: "During a 20-minute work session, do I make fewer off-task phone checks with notifications silenced?"
- A: My usual notification settings.
- B: Notifications silenced.
- Measure: Number of off-task phone checks.
- Prediction: Fewer checks in B.
Define a check before starting: each time you unlock the phone for something unrelated to the task. Keep its location and the counting method the same.
This is an illustrative experiment, not a claim about what your result will be. Choose a reversible change you can test without disrupting essential responsibilities.
Change one thing you can name.
For this first experiment, keep the contrast simple. If B also includes a new room, a different task, and a longer session, you are testing that whole package. You cannot tell which part helped.
Use similar tasks, the same session length, and roughly the same time of day. Record exceptions: poor sleep, an interruption, or an unusually difficult task.
Give both conditions a fair chance.
Plan four pairs of comparable sessions. Within each pair, use A once and B once. Flip a coin beforehand to decide which comes first. Keep your schedule rather than choosing B only on promising days.
Pairing helps you compare similar occasions. Randomizing the order helps avoid always giving one condition the earlier or later slot. Neither makes the sessions identical.
Write the planbefore you know the outcome.
Decide what improvement would be useful to you, when the trial ends, and what would make you stop early. For this example: an average of at least one fewer check in B is worth investigating further.
Eight sessions are a manageable pilot, not a universal sample-size rule. This short comparison suits an immediately reversible setting; lasting changes need a different design.
Keep every session.
Record results as you go, including the ones that disappoint you. Here is an invented set of phone-check counts; columns show conditions, not their order.
| Pair | A / usual | B / silenced |
|---|---|---|
| 1 | 4 | 2 |
| 2 | 3 | 3 |
| 3 | 5 | 2 |
| 4 | 3 | 1 |
| Average | 3.5 | 2.0 |
B averaged 1.5 fewer checks and had fewer checks in three of the four pairs. That meets the practical threshold chosen beforehand. It is a reason to try B again, not proof of a general law.
Say what you learned. Leave room to revise.
A fair conclusion: "In these sessions, I checked my phone less with notifications silenced." That does not yet establish better work, explain the mechanism, or predict the result for everyone else.
You knew which setting you were using. Expectations, task differences, and chance remain possible influences. Review the notes and repeat if the decision matters.
If results are mixed, keep the uncertainty. If little changes, ask whether the intervention was too small or the measure missed what matters. Revise the next test without rewriting this one.
Plan one small test.
Use a notebook. Ten minutes of planning can make the next few days more informative. Keep the original plan beside the results.
- Question + prediction.What do I expect to change, for whom, and under which conditions?
- A + B.What am I comparing? What will stay as similar as possible?
- Measure + threshold.What will I count or observe? What difference would matter to me?
- Schedule + limits.How many trials? In what order? When will I review, and what would make me stop early?
- Log + context.For each trial: date, condition, result, and anything unusual. Keep failures and interruptions visible.
- Result + next step.What changed? What else could explain it? Keep, revise, repeat, or abandon the change?
If a pattern persists, the next question is what keeps producing it. Guide 04 will turn from individual tests to the systems around them.
- Notice
- Question
- Test (you are here)
- Understand
- Build
- Teach
I didn't write this because I've escaped anything. I wrote it because I keep finding walls I didn't know were there, and I'd rather compare notes than keep them to myself.
If this works, you'll need me less over time, not more. If I've got something wrong, I want to know.
Found a wall?
Tell me what you found. I read these.
Worth sending: the assumption you questioned · what you discovered · something you disagree with · something you can't explain · an experiment you ran · a system you escaped — or one you built instead.
— Andrew
Method reading is listed at andrewvalentine.net/sources. Done reading? Hand it to someone.