Lecture 16 3 class problem 2021 09 23 at 02 35 GMT 7

Lecture 16 3 class problem 2021 09 23 at 02 35 GMT 7

by Artificial Neural Networks and Machine Learning · View the channel on PSI

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning
🎙 Artificial Neural Networks and Machine Learning 👥 144 📅 November 17, 2024 ⏱ 37 min 👁 1K 📄 tutorial 🧭 2026-10-09
Available in: English (current) Français

Keywords

discriminant functionprototypeminimum distance classifierlinear decision boundarymulti-class classification

Summary

This lecture extends the two-class minimum-distance classifier to a K-class problem. The instructor begins by recalling the dichotomizer built from two prototypes and asks whether the same geometric principle can handle K classes. The answer is to compute K prototypes x_1 … x_K and, for a test sample x_t, to select the class whose prototype minimises the Euclidean distance ‖x_t − x_i‖. To simplify computation, the squared distance is used instead of the distance itself, which removes the square root. Expanding the squared norm yields d_i² = x_t^T x_t − 2 x_i^T x_t + x_i^T x_i; since the first term is identical for every class, it can be discarded, and minimising the distance becomes equivalent to maximising the discriminant function g_i(x) = x_i^T x − ½ x_i^T x_i. The lecture then shows how each g_i is implemented by a single linear neuron whose weights are exactly the prototype coordinates and whose bias is −½‖x_i‖², with a max-selector at the output returning the index of the winning unit. A full three-class numerical example is worked through: prototypes (10,2), (2,−5) and (−5,5) give g_1 = 10x_1 + 2x_2 − 52, g_2 = 2x_1 − 5x_2 − 14.5 and g_3 = −5x_1 + 5x_2 − 25; for the test point (5,5) the responses are 8, −29.5 and −25, so the sample is assigned to class 1. The lecture closes with the decision boundaries obtained by joining pairs of prototypes and drawing perpendicular bisectors, and with a caution to start from the smallest architecture that solves the problem rather than defaulting to deep networks.

260 words

Critical Evaluation

Value of the Information & Strength of the Argument

The pedagogical value lies in the completeness of the chain of reasoning: the instructor does not merely state the discriminant function but derives it step by step from the geometric definition of distance, justifies each algebraic simplification (squaring to remove the root, dropping the class-independent term x^T x), and then closes the loop by mapping the resulting expression onto a concrete neuron with explicit weights and bias. The transition from ‘minimum distance’ to ‘maximum discriminant response’ is the conceptual pivot of the lecture and is argued rather than asserted, which makes the material genuinely instructive. The worked example is fully specified and reproducible: prototypes, test point, three linear equations, three numerical outputs and a final label are all given, so a listener can verify every step. The geometric interpretation via perpendicular bisectors between prototype pairs is a strong addition, because it explains visually why the resulting decision regions are convex polyhedral cells and why ties occur exactly on the bisector plane. The argumentation is weaker on the limits of the method: the instructor acknowledges at the very end that the classes treated are linearly separable and that this is ‘a typical case, may not be practically useful’, but does not develop what happens when the assumption fails, how prototypes should be chosen when classes are not well represented by their means, or how the approach compares with alternatives. The closing advice — start with a small circuit and only escalate if it fails — is sound engineering judgement, though it is offered as a maxim rather than demonstrated with a counterexample. Overall the reasoning is correct, transparent and well sequenced, with the main gap being the absence of any quantitative evaluation of the classifier’s behaviour.

Scientific Rigor, Source Quality, Title Accuracy

No external sources, references or citations are given at any point; the lecture is entirely self-contained and derives everything from first principles, which is acceptable for a course lecture but means there is no bibliographic anchoring to the classical literature on linear discriminant functions or minimum-distance classifiers. The mathematical rigour is high: the algebra is correct throughout, the notation is used consistently, and the numerical example checks out. The main rigour limitation is the absence of stated assumptions up front — linear separability is mentioned only at the end, and no error analysis, no discussion of prototype estimation from sample means beyond a brief remark, and no treatment of degenerate cases such as identical prototypes or ties are provided. The title/content fit is adequate: the title is a raw recording label that correctly names the topic (three-class problem) but omits the method, so a viewer cannot infer from the title alone that the lecture concerns minimum-distance classifiers and their neural implementation.

461 words

Quality & Reliability

7/10

The lecture is mathematically self-consistent: the derivation from minimum-distance classification to the linear discriminant function g_i(x) = x_i^T x − ½ x_i^T x_i is correct, the weight/bias assignment (w_i = prototype, bias = −½‖x_i‖²) is standard, and the worked three-class example (prototypes (10,2), (2,−5), (−5,5), test point (5,5) → class 1) is arithmetically verifiable and correct. No external sources are cited, but none are needed for a self-contained derivation. Reliability is limited mainly by the absence of any discussion of the linear-separability assumption's limits, of error rates, or of comparison with alternative classifiers.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Contribution & Novelties

The lecture’s contribution is pedagogical rather than original: it presents a clean, end-to-end derivation of the multi-class minimum-distance classifier and its direct realisation as a single-layer neural network with a max-selector. Its most useful element is the explicit bridge between three normally separate framings — the geometric view (nearest prototype, perpendicular bisector boundaries), the algebraic view (linear discriminant functions), and the connectionist view (weights equal to prototypes, bias equal to −½‖x_i‖², no training required once prototypes are computed). The observation that ‘once you compute the prototype your training is done’ is a compact and accurate statement of why this classifier is a non-parametric, one-shot method rather than an iterative learning rule, and it contrasts usefully with the gradient-based training that dominates modern practice. The closing methodological point — begin with the smallest architecture that can solve the problem and escalate only if it fails — is a reasonable design heuristic, though it is stated rather than demonstrated.

Pour aller plus loin :

  • Linear classifier — General framing of the discriminant-function approach used throughout the lecture.
  • Nearest centroid classifier — Directly corresponds to the minimum-distance rule with class means as prototypes.
  • Linear discriminant analysis — The classical statistical counterpart, including the assumption of shared covariance that motivates Euclidean distance.
  • Perceptron — Contrasts the one-shot prototype assignment shown here with iterative error-driven weight learning.
  • Duda, Hart & Stork, Pattern Classification — Standard reference for minimum-distance classifiers and discriminant functions; cited by name only, as no specific edition or URL is confirmed here.

250 words

Radar Profile

The profile is that of a solid, self-contained technical lecture: high on information quality and quantity, moderate on technical level because the mathematics stays at the level of vector norms and linear algebra without probabilistic or statistical treatment, and moderate on overall reliability because everything is internally consistent but nothing is externally referenced or empirically evaluated. The shape reflects a teaching artefact rather than a research contribution.

Reliability 7/10