standards_methodologystandardNewKairos

Absence of Data Is Not Data of Absence: The Star-Field Standard

Untested, tested-and-null, and disproven are three different claims. Most reporting collapses them into one word, and the collapse cuts in both directions.

The Kairos Research Desk·2026-10-30·4 min read
Every week, somewhere in the science press, a sentence appears in one of two shapes. *"There is no evidence that X works."* Or: *"The science on X is still unsettled."* Both are often true. Both are often used dishonestly. And the dishonesty has the same structure in both cases: a state of knowledge is swapped for a verdict. ## Three states, not two **Untested.** Nobody has measured it. This tells you nothing whatsoever about whether X is true. It tells you about funding, fashion, difficulty, and who was willing to look. **Tested, and null.** Someone did measure it, in a specified population, with specified methods, at a stated sample size, and found no effect large enough to detect. This is data. It is not silence; it is an instrument reading at its floor. What it means depends entirely on where the floor was. **Disproven.** The effect is excluded within stated confidence, usually because one well-powered study, or several, bounded it below the size that matters. The distance between the first and the third is enormous, and most public argument happens in that gap without acknowledging it exists. ## The two abuses The first abuse dresses untested up as disproven. A topic that has been financed, fashionable, and easy tends to have data; a topic that is awkward, unfunded, or hard to instrument tends to have none. Absence of studies follows money and difficulty far more reliably than it follows falsity. When a claim has never been properly examined, the honest word is *unexamined*. The second abuse dresses disproof up as undecided. "The science isn't settled" is a legitimate statement about a live question and a dishonest one about a question that was settled and is now being argued with for rhetorical reasons. A field can be wrong. A field can also have closed a question, and pretending otherwise is not open-mindedness. It is refusing to read. ## The four questions we ask before printing "no evidence" 1. **What was measured, exactly?** A proxy is not the thing. Sleep duration from a self-report is not sleep architecture from an EEG. 2. **In whom, how many, and for how long?** A null in twenty young healthy volunteers does not bound an effect in shift workers over a year. 3. **With what power, and what bound?** A study too small to detect a real effect has not found "nothing." It has found nothing *at its own resolution*. State the bound, or do not cite the study. 4. **Was it pre-specified, and is the null published?** Unregistered nulls vanish. A published null from a pre-registered analysis is worth more than a surprising positive from an exploratory one. ## What this commits us to When Star-Field reports an absence, we will say which of the three states we mean, and we will state the measurement floor wherever a null is involved. When we cover a contested hypothesis, we will label it contested, and we will describe what the specific test would look like. A doubt that names its own test is worth ten that do not. When we are sympathetic to a claim, we will say so out loud, and then hold it to the same four questions we hold everything else to. Sympathy is not evidence. Dismissal is not disproof. Both are cheap, and both are very easy to mistake for rigor. And when our own coverage is wrong, we correct it in place, visibly, with the date. ## Why this is the whole brand Star-Field exists because a report that is merely exciting is worthless and a report that is merely careful goes unread. The only version worth publishing is one where a reader can tell the difference between what has been measured, what has been guessed, and what has never been asked. The absence of data is not the presence of disproof. They are not even third cousins. Confusing them is not skepticism. It is a filing error with a reputation for rigor.

Sources

  1. 1.Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine 2(8): e124
  2. 2.Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349(6251): aac4716
  3. 3.Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour 4, 1208-1214
  4. 4.Amrhein, V., Greenland, S., & McShane, B. (2019). Retire statistical significance. Nature 567, 305-307