Probability and evidence

Prerequisites: Fractions, multiplication, and a table of counts.

• Distinguish P(flag | defect) from P(defect | flag).
• Compute a posterior using natural frequencies.
• Derive a decision threshold from error costs.

Count the population you are conditioning on

Probability describes uncertainty within a stated model. P(A) is the probability of event A. P(A and B) is the probability that both occur. Conditional probability P(A | B) restricts attention to outcomes where B occurs: P(A and B)/P(B), provided P(B)>0. A detector’s chance of firing on a defect is therefore different from the chance that a flagged item is defective.

Bayes’ rule follows by writing the same joint probability in two orders:

P(A and B) = P(A | B) P(B) = P(B | A) P(A)
P(A | B) = P(B | A) P(A) / P(B)

The prior P(A) describes uncertainty before observing B. The likelihood P(B | A) describes how often that evidence occurs under A. The posterior P(A | B) describes uncertainty after observing B. These are roles in a calculation, not guarantees that the input probabilities are well estimated. For the underlying rules, read Berkeley CS 188’s probability notes.

An alert is evidence, not a conclusion

Imagine exactly 1,000 manufactured parts. Of these, 20 are defective. An inspection device flags 18 defective parts and 98 good parts. Treat these counts as the complete toy population.

Actual status	Flag	No flag	Total
Defective	18	2	20
Good	98	882	980
Total	116	884	1,000

The chance of a flag given a defect is 18/20=90%. The chance of a defect given a flag is only 18/116≈15.52%. The detector finds most defects, but defects are rare and false alarms accumulate across many good parts. A flag still changes the defect probability substantially, from 2% to about 15.52%.

Calculate the same result with rates. The prior is 0.02, the true-positive rate is 0.9, and the false-positive rate is 0.1. The probability of a flag is 0.9×0.02 + 0.1×0.98 = 0.116. The defective-and-flagged probability is 0.018. Divide 0.018 by 0.116 to recover the posterior.

Now change only the prior to 0.20. Holding the detector’s conditional rates fixed gives posterior 0.18/(0.18+0.08)≈69.23%. This hypothetical calculation isolates the effect of prevalence. In a real setting, the detector’s rates might also change across populations; that assumption needs measurement.

Decisions need costs as well as probabilities

Suppose accepting a defective part costs 10 units and discarding a good part costs 1 unit. Correct decisions cost zero in this simplified model. For estimated defect probability p:

expected cost of accept  = 10p
expected cost of discard = 1(1-p)
discard when 1-p < 10p, meaning p > 1/11

The threshold is about 9.09%. A flagged part at estimated p≈15.52% should be discarded under this cost model. The calculation does not require a 50% threshold. If an additional inspection costs less than the expected improvement in the eventual decision, inspection could be a third action. That would require its own conditional outcomes and costs.

Expected cost is an average under a distribution. It does not promise a particular outcome on an individual part. A useful report states both the estimated probabilities and the cost assumptions, so someone can challenge either one.

Independence is a substantive assumption

For independent events A and B, P(A and B)=P(A)P(B). Conditional independence is different: two observations might be independent after fixing a hidden cause while remaining correlated overall. Two sensors made with the same flawed component may fail together. Multiplying their error rates as though they were independent would overstate the benefit of redundancy.

To make this concrete, suppose two flags are exact copies of one sensor’s output. Observing both is no stronger than observing one. A calculation that treats them as independent counts the same evidence twice. Before combining signals, ask what common causes or duplicated measurements connect them.

In your exercise notes, show the count table before applying a formula. A correct denominator is often the difference between a useful probability and a confident mistake.

Practice exercise

A batch has 2,000 parts. Exactly 100 are defective. A detector flags 80 of those and 190 of the 1,900 good parts. Build the full two-by-two count table. Calculate precision, recall, and the defect probability after no flag. If discarding a good part costs 2 units and accepting a defective part costs 30 units, derive the threshold for discarding a part from an estimated defect probability p.

Compare with the worked solution

Flagged: 80 defective and 190 good. Unflagged: 20 defective and 1,710 good. Precision is 80/270≈29.63%; recall is 80/100=80%. P(defect | no flag)=20/1730≈1.16%. Discarding has expected cost 2(1−p); accepting costs 30p. Discard when 2(1−p)<30p, or p>1/16=6.25%, with indifference at equality. This threshold assumes those are the only costs and the probability estimates apply to these parts.

Check your understanding

What fraction of flagged parts in the exercise is defective?

• 80%
• 5%
• 190/2000
• 80/270

Check the reasoning

Conditioning on a flag restricts the denominator to the 270 flagged parts.

What happens to the discard threshold if the cost of accepting a defect increases?

• It falls.
• It rises.
• It must stay at 50%.
• The prior probability becomes irrelevant.

Check the reasoning

The threshold is C_false_positive / (C_false_positive + C_false_negative). Increasing the latter cost lowers the threshold.

Learn AI curriculum · Next: Linear models and gradients