Can a machine learn the wrong pattern?

Start with one exact question

A fictional print shop asks whether a dot’s center is inside a circular guide. Its decorative A/B batch mark is not part of that geometric rule. The dot’s visible size is not what is classified.

Choose the information available

The model receives numeric coordinates and the selected fixed features. It does not read pixels. A straight model sees x and y; a curved model also receives x²+y². The designer supplies this useful representation, while training learns its weights.

Learn by changing actual numbers

Every training update computes prediction errors from supplied annotations and adjusts all weights together. The score maps, weight table and loss curves use the same recorded weight snapshot. Playback reveals real optimization steps.

Find the first misleading success

A flexible model can fit the deliberately wrong outside-card annotation. It may then call a nearby outside dot inside with a high score. Compare the supplied label with the separately defined guide, then correct and refit.

Swap only the wrapper

The correlated-batch preset puts inside examples in A and outside examples in B. A batch-only model can look perfect on familiar marks. Reverse only the marks and inspect its predictions without changing positions or weights.

Repair a specific problem

Remove the batch feature or supply both marked versions at each training location. These paired cards share lineages; they are not independent measurements. Verify the repair on appropriate probes rather than assuming more examples always help.

Choose before testing

Use validation for choices, freeze the currently inspected weights and threshold, then open the reserved envelope. Once answers have influenced your choices, placing those same answers back in an envelope cannot make them unseen again.

Exact synthetic geometry

Each dot has integer coordinates u,v from −1000 to 1000; x=u/1000 and y=v/1000. The answer is inside when u²+v²≤490000, including the boundary. The guide radius is 0.7 and both map axes have equal scale. This is invented teaching data, not a production benchmark.

The predictor does not receive the answer

The feature vector contains only selected position calculations and possibly a stored visible batch mark. Guide answers are evaluator metadata. Training uses the annotations supplied on training cards, including intentionally wrong ones.

One logistic-regression family

The score is σ(w·f), where σ(z)=1/(1+exp(−z)). The main feature choices have 3, 4 or 29 weights. Flexible features add 25 fixed Gaussian bumps centered on a declared grid with width 0.2. These centers and widths are not learned hidden neurons or an infinite kernel.

A declared objective

Training minimizes mean binary log loss plus λ/2 times the sum of squared non-intercept weights. The intercept is unpenalized. The workbench reports data loss, penalty and total objective separately. Natural logarithms give nats per example.

A genuine simultaneous update

The gradient averages (score−annotation) times each feature, then adds the non-intercept penalty gradient. All coordinates update from the old weight vector. The conservative step η=1/B uses B=Σ||f||²/(4n)+λ, an upper bound derived from σ(1−σ)≤1/4. Validation and reserved labels are absent.

A budget is not convergence

Weights start at zero and at most 5000 updates are recorded. Editing examples, features or λ starts a fresh fit. In unregularized separable logistic regression, weights may continue growing as loss approaches zero. A finite update budget does not promise exact convergence or zero loss.

Capacity and the target shape

On the symmetric clean grid, the straight model settles at a constant inside score 9/25=0.36. A straight boundary cannot enclose the center while excluding the surrounding outside examples. Running more identical updates cannot add a missing representation.

Accuracy and loss can disagree

On the fixed noisy flexible example, λ=0 gives 35/36 validation classifications and mean loss about 0.106845; λ=0.02 gives 36/36 but loss about 0.345015. Regularization improves one criterion here while worsening another. Neither is a universal ranking of model quality.

A score is not verified certainty

A fitted score near 0.96 can be wrong on the deliberately noisy fixture. Calibration is about the relationship between predicted probabilities and observed outcomes across suitable data; a confident single prediction does not establish it.

Thresholds change decisions

Classify inside when the unrounded score is at least the threshold. Raising the threshold cannot increase true-positive or false-positive counts on fixed data. It leaves weights, scores and log loss unchanged. Exact ties count as inside.

Four counts, different questions

Inside is the positive class. TP finds an inside center, FP calls an outside center inside, FN misses an inside center, and TN rejects an outside center. Accuracy divides all correct predictions by all cards; precision divides TP by inside predictions; recall divides TP by actual inside cards. A zero denominator is undefined.

Independent fitting is only part of evaluation

Training fits weights; validation guides choices. A reserved set evaluates a frozen choice. Using reserved scores to tune settings compromises their untouched role even when none of those rows enters a gradient. The exposure marker persists locally.

A fixed reproducible envelope

The 64 reserved integer-grid positions use a documented 32-bit LCG with coordinate seed 48 and independent mark seed 49, rejecting original training/validation coordinates and duplicates. Their balanced class count happens for this seed; the generator does not enforce it. The code and answers remain inspectable, so this is not a secure exam.

Data edits can contaminate a split

Validation and reserved records stay fixed when training is edited. A shared coordinate or lineage is flagged. Changing an ID does not make a copy independent. The original validation grid is deliberately designed, rather than a representative random sample of a factory population.

Shortcut, leakage and shift

A pre-existing decoration can support a misleading shortcut without being a future answer leak. Reversing marks changes the input distribution while the geometric rule stays fixed. For a batch-only predictor, the target relationship conditional on its restricted input changes. An inspection-answer field created afterward is excluded for a different reason: it is unavailable at prediction time.

A changed specification is another problem

If a shop changes its guide radius from 0.7 to 0.5, the same position (0.6,0) changes answer. That is a new target rule, not just a different mix of positions. Our interactive workbench keeps radius 0.7 fixed; the old weights would not automatically adapt to a new specification.

Checking a model before trusting a score

Write the prediction question, when each input is available, what the labels mean and which data influenced selection. A percentage without those details can conceal a different task.

Investigating an apparent failure

Inspect individual errors and change one factor at a time. A missing representation, wrong annotation, changed input distribution and leaked answer require different explanations.

Evaluating tools you encounter

Ask whether a reported result used a frozen choice on appropriate new examples. This toy exercise does not certify any real product, but it makes the evaluation questions concrete.

What is the exact target question?

• Is the dot center inside the guide?
• Is the card valuable?
• Does the decoration look attractive?

Why can the straight model miss the enclosed inside region?

• Its chosen features cannot express that enclosure
• The test must be leaking
• More identical updates always fix any representation

A noisy fit calls an outside dot inside with score 0.96. What follows?

• The guide is wrong
• The model can extend a wrong annotation into a confident error
• Every flexible model always fails

Swapping only A/B flips the prediction of frozen weights. What did you establish?

• The circular guide moved
• This model depends on the mark at this location
• Every correlation is target leakage

You choose λ after reading final-test scores, but never fit on their rows. Is the score untouched?

• Yes
• No, those answers influenced selection
• Only if it is high

TP=1, TN=1, FP=1, FN=1. Accuracy and recall are…

• 1/2 and 1/2
• 1/4 and 1
• 1 and 1/4

Raise the threshold with the same frozen weights. What stays fixed?

• Every predicted class
• Scores and log loss
• Every false-positive count

Change guide radius from 0.7 to 0.5; the same dot at (0.6,0) changes answer. What changed?

• The target specification
• Only playback speed
• The old model automatically learned

Logistic model, likelihood, gradient and feature representations

Stanford CS229 notes, Chapters 2 and 5. Feature bank, data, penalty convention and step bound in this lesson are explicitly authored choices.

Log loss and regularization

Mean binary cross-entropy and regularization concepts. Our scalar values and experiments come from the declared synthetic model.

Training, validation and test roles

Separate fitting, selection and final evaluation. A fixed designed grid does not become representative of production merely by being held out.

Leakage and preprocessing boundaries

Data unavailable at inference or evaluation information must not leak into fitting. Our feature coordinates and centers are fixed in advance.

Selection can itself overfit finite evaluation data

Cawley and Talbot, JMLR 2010. Supports the exposed-test state even when test rows never enter a gradient.

Shortcuts can fail under changed conditions

Geirhos and colleagues, 2020. This lesson’s simple logistic batch-mark analogy is an original example, not a reproduction of their neural-network results.

Calibration differs from accuracy

Guo and colleagues, ICML 2017. A fitted probability estimate is not automatically an observed correctness frequency.

Threshold and confusion counts

Inside is our positive class; exact ties count as inside and undefined denominators are retained explicitly.

Changed input distributions and evaluation

Koh and colleagues, ICML 2021. No WILDS datasets, images or performance numbers are imported into this synthetic workbench.