Algorithm
An algorithm is a specified procedure for carrying out a computation or solving a problem. In machine learning, algorithms can train a model, search for an answer or apply a trained model to an input.
Binary search finds a value in a sorted list by checking the middle item and repeatedly discarding the half that cannot contain it. The sorted-order requirement is part of why the procedure works. Applying it to an unsorted list can miss an item that is present.
An algorithm is the procedure; code is an implementation of it. Different implementations can follow the same procedure while using different languages or memory layouts. Some algorithms also use random choices, so specifying the steps does not require identical results on every run.
A training algorithm, such as gradient descent, adjusts a model’s parameters. The resulting model is the learned system used for predictions. Separating the two helps diagnose a failure: the procedure, the examples and the model’s objective can each need a different fix.
Sources
- National Institute of Standards and Technology: Dictionary of Algorithms and Data Structures — algorithm — Defines an algorithm as computable steps and lists deterministic and randomized variants.
Go deeper
- Massachusetts Institute of Technology: Introduction to Algorithms course
Lectures and exercises on correctness, search, data structures and computational cost.