Ngaire Crawford, Director of Insights, ANZ
In leadership meetings across the industry, a single question has become unavoidable: “What is our AI strategy?” Behind this question is often the unspoken hope for an “AI Easy Button”: a mythical, one-click solution to our most complex measurement challenges. As someone who spends a large portion of my time designing these new frameworks, I’m infinitely more excited about the blueprints and the foundations than what colour the house is painted.
For the first time in my career, we have the tools to stop using proxies and start building what we’ve always wanted: true, at-scale, sophisticated measurement. The real opportunity isn’t in automation, which lets the AI decide; it’s in the architecture and design of systems for the AI to follow. For decades, I’ve been frustrated by proxies. I’ve watched organisations use metrics like Impressions and Share of Voice as proxies for impact and influence. Too many people have been measuring the loudness of their voice, not whether anyone was actually listening.
Much of the history of communications measurement has been a story of ‘good enough’ data. And in some cases, data that wasn’t even good at all (*cough* AVEs).
Why a blueprint still needs an architect
But before we can harness the potential of AI, we have to be honest about the technology and tools we’re working with. As anyone who’s ever used a “smart” tool knows, they can be… well, confidently wrong.
The new challenge isn’t just “Garbage In, Garbage Out.” The new challenge is that the AI has become a high-speed, frighteningly convincing echo chamber. When a machine delivers a flawed insight, it does so with the resolute certainty of a supercomputer, laundering that flaw into a “fact.”As architects, our job is to audit the blueprints and stress-test the materials before we build the house. When my team and I test these models, we’re not just looking for what they do right. We’re methodically hunting for where they go wrong.
Where we continue to see a critical need for human intervention and expertise:
- Context Blindness: AI is a brilliant pattern-matcher, but it has limited real-world context and struggles to identify the intent of what’s being analysed. It can miss the nuance of language, the authority of a source, or whether something is fact or speculation.
- Language Bias: This is my personal favourite and takes a few forms. AI is trained on text, but it isn’t (yet) trained on human subtext. This can look like missed nuance for slang used by younger audiences or emerging shifts in the meaning of language. Models are ultimately impacted and biased by their training data, so this can also mean larger systemic biases are amplified and not appropriately interrogated.
- Viewpoint Collapse: While AI can sometimes get locked into a perspective based on its training, it can also collapse multiple, distinct viewpoints (like a speaker’s sarcastic intent vs. the literal text) into a single, flat monolith. This drastically changes the outcomes of your analysis and ultimately the understanding of your audience.
This is the methodical, behind-the-scenes work that often goes unseen, and it is the crucial due diligence needed. It’s not as flashy as writing a press release faster, but it’s the only way to build a tool you can actually trust to make a strategic decision.
New tools, same bedrock principles
This testing isn’t just about finding technical bugs or funny hallucinations. We’re testing these new AI models against the foundational, hard-won principles of communications measurement that our industry has spent years formalising.
AI is an incredibly powerful new tool, but it doesn’t get a free pass. It still has to follow the rules of good measurement.
- Measure outcomes, not just outputs: This has always been our goal. An AI-driven approach that only counts outputs (like mentions or sentiment) 1,000 times faster is still just a faster measure of noise. It doesn’t tell you if a single mind was changed or a single action was taken.
- Demand transparency: A metric is useless if you can’t explain how it’s calculated. This is my biggest critique of the current “plug-and-play” approach to AI. If a vendor provides a proprietary ‘Reputation Score’ of 7.2, and they can’t (or won’t) tell you the formula, it’s not a metric. It’s marketing.
- Link activity to business objectives: This is the most important rule of all. The only reason to measure is to inform a strategic decision that ladders up to a business goal. A tool that just produces data, but no clear insight linked to your specific objectives, has failed.
When we stop seeing AI as a magic box and start seeing it as a powerful, scalable engine, one that we must build and steer based on these principles, then it becomes truly transformative.
The payoff: the tools are finally catching up to our ambition
A new frontier of opportunity is here. Such as the capability to move from being reactive to being predictive, and it takes careful design to get this right. Our traditional analysis has been brilliant at explaining what has just happened. Now, as architects of these new systems, we are building and testing AI models that can scan the horizon for the faint signals that precede a major narrative shift.
We can empower movement from broadcasting and the old spray and pray approach; to precision, deliberate engagement of stakeholders and audiences. This is another area where the craft of measurement design is essential. AI gives us the power to see the micro-communities and specific, high-authority voices that actually shape opinion. The work is in designing the models that can identify them accurately.
Finally, we can (at last!) move from quantifying to qualifying at scale. For me, this is the most exciting and complex challenge. For 20 years, I’ve had to choose: a large-scale quantitative study (which missed nuance) or a small-scale qualitative review (which couldn’t be scaled). As architects, we can now design frameworks that don’t just give a “positive” score but confirm that a specific strategic message landed, with the right audiences, and in the intended context.
That is the opportunity. It’s not magic. It’s the methodical, patient engineering we’ve been waiting for. It’s the difference between a “plug-and-play” gimmick and a truly strategic asset. The real payoff isn’t just faster reporting, it’s about fundamentally upgrading behaviours and expectations of measurement. This isn’t an overnight shift. As any research leader will tell you, a new methodology takes time, testing and refinement to get right.
The future we’ve been waiting for
For my entire career, we’ve been strategic thinkers working with tools that could only show us the past. We were forced to be historians, meticulously analysing what had already happened to predict future behaviour. The key to using this new, complex technology effectively is; strong communication, articulation and critical human thinking. The power of any AI is unlocked by the quality of the question you ask it. It’s a system that rewards clear, precise, and strategic language.
This is a massive homefield advantage for communicators, who have spent their entire careers honing the exact skills required to be the architects of this new era. The AI we are using today is the worst it will ever be. It will only get better, faster, and more capable from here. This is what’s so thrilling, and it’s just the beginning. This new generation of AI driven approaches doesn’t replace our intuition, it amplifies it. As communicators (and researchers!) this is the moment to level up. We get to be the explorers and the strategists who connect communications directly to business, policy and societal outcomes.
We’re not just building better measurement and deeper insights; we’re leading a more intelligent, more responsive and more impactful profession. What an incredibly exciting time to be in this industry.
Ready to be the architect of your own measurement strategy?
To learn how to build the right KPIs and tell a compelling story with your data, register for our live webinar:
- Topic: Making Communications Count: Build your KPI confidence and storytelling”
- Date & time: 12 November, 11am AEDT/ 2pm NZT
- Hosted by: Ngaire Crawford, Director of Insights for ANZ, Isentia.
The post Beyond the “Easy Button”: architecting a new, smarter era of comms measurement appeared first on Isentia.
