The Inherent Variability of AI Recommendations: A New Frontier for Brand Visibility
In the rapidly evolving landscape of generative AI and search, a fundamental truth has emerged: artificial intelligence responses are inherently probabilistic. When you pose the same question to ChatGPT or similar large language models multiple times, you receive different responses. This variability isn’t a bug—it’s a core feature of how these systems operate. Recent research by Rand Fishkin and subsequent studies have revealed profound implications for how brands should approach AI visibility tracking and search engine optimization in the age of generative AI.
The research methodology involved running 12 distinct prompts through ChatGPT 100 times each, totaling 1,200 interactions. These prompts were carefully designed to test two critical variables: competitive versus niche B2B software categories, and simple versus nuanced prompts with specific personas and use cases. The findings challenge conventional wisdom about AI consistency and provide actionable insights for B2B marketers navigating this new terrain.
The Research Methodology: Testing AI Consistency at Scale
Prompt Design and Category Selection
The study employed a systematic approach to prompt creation, dividing them into two primary categories:
- Competitive vs. Niche Categories: Six prompts addressed highly competitive B2B software categories (accounting software, CRM platforms, etc.), while six focused on less crowded niches (user entity behavior analytics, specialized compliance software). Categories were identified using Contender’s database, which tracks ChatGPT’s brand associations across 1,775 software categories.
- Simple vs. Nuanced Prompts: Within each category set, half used straightforward queries (“What’s the best accounting software?”) while half incorporated specific personas and use cases (“For a Head of Finance focused on financial reporting accuracy and compliance, what’s the best accounting software?”).
Execution and Data Collection
Researchers executed each prompt 100 times through the free, logged-out version of ChatGPT, using different IP addresses for each interaction to simulate 1,200 unique users. This approach eliminated potential biases from user history or session data, providing a clean dataset of how ChatGPT responds to first-time queries.
Key Findings: The Reality of AI-Generated Recommendations
The “Open Slots” Phenomenon
Across 100 responses to a single prompt, ChatGPT mentions an average of 44 different brands. However, this number varies significantly by category—one response set included as many as 95 distinct brands. This creates what researchers term “open slots”—opportunities for brands to appear in AI-generated responses.
Competitive Categories: ChatGPT mentions approximately twice as many brands per 100 responses in competitive categories compared to niche ones. This aligns with the underlying data used to select categories, confirming that AI models reflect market reality in their probabilistic outputs.
Simple vs. Nuanced Prompts: Surprisingly, nuanced prompts mentioning specific personas and use cases didn’t consistently narrow brand recommendations. While they sometimes yielded fewer brands, the pattern wasn’t consistent across categories. This suggests ChatGPT may lack the deep brand understanding needed to tailor recommendations precisely to specific use cases.
The Return of “10 Blue Links” in AI Form
Each individual ChatGPT response mentions an average of 10 brands, with a range from 6 to 15. This consistency across response types creates a familiar pattern for SEO professionals—it’s essentially the AI equivalent of Google’s traditional “10 blue links.” The critical difference lies in which brands appear and how frequently they rotate across responses.
While each response contains roughly 10 brands, competitive categories draw from a much deeper pool of potential mentions. This creates a dynamic where brands must compete not just for inclusion, but for consistent inclusion across multiple AI responses.
The Dominance Hierarchy: Who Wins in AI Recommendations?
The 80% Rule
Only about 5 brands, on average, achieve “dominant” status—being mentioned 80% or more of the time across 100 responses. This represents just 11% of all brands mentioned. These dominant brands typically share common characteristics:
- Established market presence and brand recognition
- Strong online visibility across multiple channels
- Comprehensive digital footprint including reviews, case studies, and industry coverage
- Frequent mention in training data and online discussions
For example, in accounting software, dominant brands include QuickBooks, Xero, Wave, FreshBooks, Zoho, and Sage—all well-established players with significant market presence.
The Niche Advantage
Niche categories offer significantly better opportunities for brand dominance. In these less crowded spaces, 21% of mentioned brands achieve dominant status (mentioned 80%+ of the time), compared to just 7% in competitive categories. This creates a strategic imperative for smaller or emerging brands: differentiation through specialization.
In competitive categories, 72% of brands languish in the “long tail,” appearing less than 20% of the time. This stark distribution highlights the winner-take-most dynamics emerging in AI-generated recommendations.
Industry Statistics and Market Context
Recent industry data provides important context for these findings:
- According to Gartner, 80% of enterprises will have used or deployed AI APIs or models by 2026
- McKinsey research indicates that generative AI could add $2.6 trillion to $4.4 trillion annually to the global economy
- A BrightEdge study found that 57% of marketers believe generative AI will have a significant impact on SEO within the next year
- Search Engine Journal reports that 45% of professionals now use AI tools for content creation and research
These statistics underscore the urgency for brands to develop effective AI visibility strategies. As generative AI becomes increasingly integrated into search and discovery processes, traditional SEO approaches must evolve to address probabilistic recommendation systems.
Actionable Strategies for B2B Marketers
Strategic Category Selection
For brands not currently in the dominant tier, strategic category positioning is crucial:
- Niche Down Strategically: Identify underserved segments within broader categories where you can establish dominance. For example, rather than competing in “accounting software,” position as “the best accounting software for commercial real estate companies in North America.”
- Leverage Unique Differentiators: Emphasize specialized features, industry-specific capabilities, or unique value propositions that distinguish your brand in AI training data.
- Build Category Authority: Create comprehensive content that establishes your brand as an authority in specific niches, increasing likelihood of AI recognition.
Improving AI Visibility Tracking
Most current AI visibility tracking tools provide misleading data by checking prompts only once per time period. Given ChatGPT’s probabilistic nature, this approach is fundamentally flawed. Effective tracking requires:
- Multiple Iterations: Run key prompts at least 5-10 times when collecting data to establish patterns rather than single-point observations
- Statistical Significance: For precise measurement, consider running prompts 50-100 times, though this may be impractical for ongoing monitoring
- Tier-Based Analysis: Categorize visibility into tiers: long tail (<20% mention rate), visible middle (20-80%), and dominant tier (80%+)
Content and Brand Signal Optimization
To improve AI recognition and recommendation frequency:
- Comprehensive Digital Footprint: Ensure your brand appears across diverse, high-quality sources including industry publications, review sites, case studies, and authoritative directories
- Structured Data Implementation: Use schema markup to help AI systems understand your products, services, and industry positioning
- Thought Leadership: Publish authoritative content that addresses specific use cases and personas, increasing relevance for nuanced prompts
- Community Engagement: Participate in industry discussions, forums, and professional networks where AI training data is sourced
The Limitations of Current AI Understanding
The research reveals important limitations in how AI systems understand and recommend brands:
- Lack of Deep Brand Knowledge: ChatGPT often cannot distinguish between brands with similar surface-level characteristics, leading to inconsistent recommendations for nuanced queries
- Training Data Biases: Recommendations reflect patterns in training data, which may emphasize larger, more frequently mentioned brands
- Contextual Understanding Gaps: AI systems struggle to fully comprehend specific use cases and match them with appropriately specialized solutions
These limitations create both challenges and opportunities. Brands that can effectively communicate their unique value propositions across multiple channels and formats are more likely to achieve consistent AI recognition.
Future Research Directions
The current study opens several important avenues for further investigation:
- Brand Narrative Analysis: How does ChatGPT describe the brands it consistently recommends? What language patterns indicate deeper brand understanding?
- Search Intent Consistency: Do different prompts with the same underlying search intent produce similar recommendation sets?
- Ranking Dynamics: How consistent is brand order within responses? Do dominant brands tend to appear first?
- Cross-Platform Comparison: How do recommendations vary across different AI platforms (Claude, Gemini, etc.)?
Conclusion: Navigating the Probabilistic Future of Search
The research clearly demonstrates that AI-generated recommendations operate on probabilistic principles, creating a dynamic and variable landscape for brand visibility. For B2B marketers, this requires fundamental shifts in strategy and measurement:
- Embrace Variability: Accept that AI recommendations will never be perfectly consistent and develop strategies that account for this inherent variability
- Focus on Patterns, Not Points: Measure AI visibility across multiple iterations to identify patterns rather than relying on single data points
- Prioritize Differentiation: In a world of probabilistic recommendations, clear differentiation becomes even more critical for consistent inclusion
- Invest in Comprehensive Visibility: Build brand presence across diverse channels and formats to increase AI recognition across multiple query types
As generative AI continues to transform how professionals discover and evaluate solutions, brands that understand and adapt to these probabilistic dynamics will gain significant competitive advantages. The future of search isn’t about securing a single top position—it’s about achieving consistent visibility across an ever-changing landscape of AI-generated recommendations.
The transition from deterministic to probabilistic search represents one of the most significant shifts in digital marketing since the advent of search engines themselves. Brands that successfully navigate this transition will not only survive but thrive in the new era of AI-driven discovery.
