Definition
A post-hoc explanation is produced after a model has made a prediction or taken an action. The explanation may analyze how input features affected the result, perturb the input and observe changes, fit a simpler local approximation, or ask a language model to describe a rationale.
Methods such as SHAP can estimate feature contributions under stated assumptions. Those contributions can help people inspect patterns, compare cases, and find possible errors. They do not automatically reveal the original model's internal reasoning or establish that a feature caused the outcome in the real world.
Evaluate two qualities separately. Fidelity asks whether the explanation accurately reflects the model. Usefulness asks whether it helps the intended audience make a better decision. An explanation can score well on one and poorly on the other.
Distinguish it from nearby terms
An intrinsic explanation comes from a model whose relevant decision process is understandable by design. A post-hoc explanation is layered onto an already-produced result. A causal explanation addresses what produced an outcome in the world, which is a stronger claim than describing a model's associations.
Check your understanding
A classifier's explanation says age was the most important feature. When age is varied across reasonable values, the prediction does not change. The readable story has failed a basic fidelity check.