Our research explores how machines, like humans, can learn from surprise. We developed two related methods—Latent Personal Analysis (LPA) and Learning via Surprisability (LvS)—that build on the principle that the most informative insights often come from what deviates from expectations. Both approaches use surprisal to extract meaningful, interpretable representations: LPA focuses on uncovering personal signatures within a domain, while LvS extends this idea to dynamic timeline data. Rather than relying on frequency or volume alone, these methods highlight the most surprising features—those that differ sharply from what is expected—allowing us to detect anomalies, shifts, and distinctive patterns. Applied across domains from cybersecurity to historical text analysis, this approach advances interpretable machine learning by aligning it more closely with cognitive processes of attention and learning.