As with most of you out there, I’m sure, I’m learning new ways to use AI every day and stumbling across all kinds of creative tips, tricks, and prompt strategies that seem obvious in hindsight — but still seem like a “revelation” given how new this technology is for all of us.

Along those lines, I’ve recently come across an AI prompting concept called “Verbalized Probability Sampling” in some of my online meanderings.  While I’ll admit I have no idea what this phrase means in a deeply technical sense, I’ve played around with it a ton and believe it can be a very useful and enlightening approach in certain research situations.

All this concept really means is to shift your AI query from “asking for a single answer” to instead asking it to “Generate X number of responses with their corresponding probabilities.”  The result of this forces sites like Claude, ChatGPT, or Gemini to “show their math” in other words and give you a more detailed breakdown of the certainty/probability around certain answers.

It’s probably best explained by just showing you a few sample career-related prompts I’d encourage you to run on whatever your preferred AI tool might be:

  • Generate 5 responses with their corresponding probabilities of how likely one is to get their next job through a standard job advertisement response
  • Generate 10 responses with their corresponding probabilities of how long it takes the average U.S. CFO to find their next job
  • Generate 15 responses with their corresponding probabilities of the occupations a former teacher might consider when trying to make a career change
  • Generate 10 responses with their corresponding probabilities of the salary an Executive Assistant should expect in the Seattle area
  • Generate 10 responses with their corresponding probabilities of what is most likely to cause a resume to be rejected before a human sees it.

Now make no mistake, AI isn’t perfect — and despite the very official-looking nature of the results of such searches, the calculations are still based on certain baked-in assumptions and the limit of whatever data source the AI tool can access.  So as always, keep your critical thinking cap buckled tightly on.  But still, if you experiment with a few similar searches to the ones above, I think you’ll find this approach can be a useful new angle to consider in a wide variety of research situations.