IID vs Non-IID Entropy

Why this matters

Entropy estimation is only meaningful relative to assumptions about the source. Two datasets can contain the same symbols and have similar histograms while differing dramatically in predictability if one contains temporal structure.

For example, a stream can be perfectly balanced between zeros and ones but alternate deterministically. A frequency count looks ideal, while the next bit is trivial to predict after observing the previous one.

IID is therefore not shorthand for “the histogram looks uniform.” It is a model about both distribution and independence.

Independent

Independence means that knowing earlier samples does not change the modeled probability distribution of the next sample.

Real physical sources may contain memory, bandwidth limits, oversampling, filtering effects, environmental coupling or acquisition-chain correlation. Those effects can create dependence even when the marginal distribution looks healthy.

Identically distributed

Identically distributed means the distribution is stable across samples under the model.

A source that drifts with temperature, startup state, supply conditions or aging may not fit that assumption across all relevant conditions.

Why an IID claim is attractive

When a source is genuinely IID, the mathematical model can be simpler and the resulting entropy assessment may be less conservative.

That creates a dangerous incentive to “choose IID” because the result looks better. The assumption must come from source evidence, not the desired entropy number.

NIST's EntropyAssessment repository explicitly notes that most commonly used entropy sources are not IID and that additional justification is necessary to support an IID claim.

What the non-IID path does differently

The non-IID path applies estimators intended to capture different forms of predictability in data without assuming full independence. The conservative result is driven by the most limiting applicable estimate rather than by a single pleasant-looking statistic.

This does not mean non-IID data is automatically bad. Many useful physical entropy sources are not IID. The point is to account for structure rather than ignoring it.

IID is not a property proven by one chart

A distribution plot, autocorrelation chart or generic test suite can provide evidence. None of those alone converts a physical source into an IID process.

A defensible IID claim considers:

• the physics of the source;
• measurement bandwidth and sampling strategy;
• analog and digital filtering;
• dependence introduced by post-processing;
• behavior across operating conditions;
• statistical evidence;
• restart behavior.

Relationship to restart testing

IID/non-IID analysis of sequential data and restart analysis answer different questions.

Sequential data characterizes the stream during operation. Restart data examines repeated initialization behavior. A source can look relatively well behaved during one long acquisition but show repeated startup patterns across many restarts.

That is why the complete assessment should not treat a long continuous file as a substitute for properly collected restart data.

How this applies to QRNG

A quantum process may be fundamentally unpredictable while the measured digital sequence still contains classical dependence.

Examples include detector dead time, analog bandwidth, sampling electronics, digital filtering, environmental coupling or post-processing state. A QRNG evaluation therefore needs to distinguish the underlying quantum claim from the statistical model of the digitized samples.

“Quantum” does not automatically imply IID.

Practical default

For a preliminary engineering assessment, the conservative default is:

1. retain raw samples before conditioning where possible;
2. do not assume IID without justification;
3. run the non-IID assessment first;
4. treat IID as an advanced claim requiring evidence;
5. collect restart data separately when evaluating restart behavior.

This is why QRNG.io's local assessment kit should present Non-IID as the recommended first workflow and put IID behind an advanced explanation.

What a result does not prove

Even a strong entropy estimate under a chosen track does not establish that:

• the physical model is correct;
• the source is quantum;
• the device has been evaluated across all operating conditions;
• health tests are sufficient;
• conditioning is implemented correctly;
• a formal validation certificate exists.

The statistical track is one part of a larger assurance case.

Sources and further reading

1. NIST SP 800-90B
2. NIST EntropyAssessment repository
3. NIST Entropy Validation Server