Skip to content

AI Glossary · Concepts

What is Autotuning?

Autotuning

Autotuning automatically tests candidate settings or implementations for a workload and keeps a valid option based on measured performance.

Also known as: auto-tuning

· Updated · Chain of Thought

AI HardwareAI Infrastructure

  • 1 episode
  • 1 guest clip
  • 2 sources

Autotuning defines a search space, runs candidates and chooses from measured results. Triton’s autotune API, for example, evaluates kernel configurations when specified key arguments change. Its documentation warns that testing configurations runs the kernel multiple times, and provides ways to reset or restore values that would otherwise be updated repeatedly (Triton, autotune). The measurement procedure is part of the optimization.

For example, test a few block sizes for a matrix operation at the input shapes your service actually receives. Check the output against a reference before keeping a faster candidate. Include warmup and repeated measurements, and distinguish time spent searching from time saved during later execution. Avoid evaluating one configuration with clean state and another with state left over from an earlier run.

Separating the older optimization loop from an AI-guided search, Anush Elangovan says, “Search space exploration has been around for a while. Now, what’s new is using AI and LLMs for search space exploration, right?” (episode 72, 24:05).

AMD’s Hyperloom description connects profiling and optimization across serving, frameworks and GPU kernels. Treat its reported results as results of that implementation, not a guarantee for yours. For builders, retain the baseline, tested configuration and correctness result. The AI hardware hub covers profiling; AI infrastructure covers the full serving workload.

Autotuning: search, remeasure, keep only what holds up An autotuning loop as AMD describes its Hyperloom optimizer. Profile the workload to find hotspots. Search for a candidate change at the serving, framework or GPU kernel layer. Remeasure the candidate end to end. Keep it only if throughput improves and accuracy holds against the baseline; otherwise discard it. Either way, the loop searches again. Below: the older approach defines a search space and samples it with methods such as genetic algorithms or particle filters; the newer one uses an AI model to choose what to try next. Autotuning: measure first, keep only what holds up The loop AMD describes for its Hyperloom inference optimizer. 1 · PROFILE Run the workload and find the hotspots 2 · SEARCH Pick a change to try serving · framework · kernel 3 · REMEASURE Benchmark it end to end 4 · THE GATE Throughput improves and accuracy holds against the baseline? yes Keep the change the new baseline no Discard it either way, search again THE OLD WAY Define a search space, then sample it with genetic algorithms or particle filters. WHAT’S NEW An AI model makes more informed picks about what to try next in that space.
The loop and the keep rule follow AMD’s description of Hyperloom. The old-versus-new contrast is Anush Elangovan’s, from episode 72. Download the image

Hear it from the guest

“Search space exploration has been around for a while. Now, what's new is using AI and LLMs for search space exploration, right?”
Anush Elangovan, AMD EP 72 · around 24:37 · read the transcript

Quotes lightly edited to remove filler words.

Sources

From the conversation

Edge AICompound AI SystemsContinuous BatchingInference