So, here’s a sentence I never expected to type: I’ve been working on building an AI version of my husband.
Yes. You heard that right. While Joe (aka JoeV1) is recovering from a hip replacement, I’ve been busy prototyping JoeV2 — a digital clone designed to answer clinical and operational questions at our allied health clinic while the original naps, ices his hip, or dreams of tunes in an Irish pub somewhere near Tulla.
Let’s be clear. JoeV2 may never misplace his keys or wallet, and he’s unlikely to leave a trail of coffee mugs in strange places. But he’s also not quite as good a kisser. Trade-offs.
But this is more than a cheeky sci-fi side quest. JoeV2 is part of a bigger story — one about data, ethics, and whether we’re feeding our precious insights into something that might never feed us back.
What Pre-Trained AI Models Cost Care Organisations
I tasked our brilliant new physio Sijie (who has a brain wired suspiciously like mine) with researching whether we should build JoeV2 from scratch or adopt a pre-trained model that we could fine-tune to reflect Joe’s particular brand of clinical wisdom.
Amid the excitement of virtual husbands, one thing stopped us in our AI-powered tracks…
We pay them… to take our data?
Some of the pre-made models we found would let us plug in our clinic’s data to power their clinical reasoning engines while simultaneously using our data to improve their own model. So essentially, we would be paying them for the privilege of handing over our data to make their product better.
Extractive Technology and the Hidden Cost of Care Data
This all reminded me of a brilliant conversation I had with the legendary Barbara G in Aotearoa last year. We were working on a whole-of-community health workforce plan (as you do), and she introduced me to the concept of extractive technology.
Extractive technology is like mining: valuable stuff gets pulled out, profits are generated… but the earth doesn’t get replenished. The system is one-way. Something is taken, used, profited from — and rarely restored.
Are some AI models doing the same?
Care organisations generate some of the richest, most complex data that exists — not because it’s lucrative, but because it’s grounded in real lives, professional judgement, and long-term relationships. When that data improves an AI system, where does the benefit go? Who gains the insight? And how does value return to the people whose stories made it possible?
From Data Providers to Data Custodians in Health and Care
As clinicians, we’re not just data providers. We’re data custodians. The insights we generate from and with our clients have power. That means we have a duty to make sure the AI tools we use are trained ethically, transparently, and ultimately benefit the people whose lives inform the data in the first place.
AI can be a powerful, and highly functional tool in care settings. Used well, it can surface patterns that are easy to miss, reduce duplication, bring together fragmented health information, and support practitioners to make clearer, more consistent decisions. But its value lies in support, not substitution. AI works best when it augments human judgement and expertise rather than attempting to replace it.
An AI model can’t conjure Joe’s years of clinical experience, relationship-building, and nuanced understanding of patient needs from thin air. What it can do, however, is be grounded in that expertise. By anchoring JoeV2 in real clinical knowledge, structured data, and lived experience, we can create a human-centred AI model that understands the complexity of patient journeys and care pathways. An AI model that supports human interaction and decision-making, rather than competing with it.
How UNPLEXi Designs Responsible AI in Real-World Care Settings
This thinking sits at the core of how we approach AI at UNPLEXi.
The intelligence in our system doesn’t come from the model alone, but from the careful capture of meaningful information, rigorous oversight, and the use of internationally recognised, evidence-based frameworks.
Our work is grounded in real, lived data and deliberately designed to work against bias rather than amplify it. Just as importantly, our systems are shaped by people with clinical expertise, lived experience, and decades of practical knowledge. We’re intentional about representation, and we’re working closely with our amazing cultural sensitivity advisor, Georgia, to ensure cultural context is a design input from the beginning.
From Theory to Practice: Adapting AI to Real Care Workflows
We also understand that every organisation is different. Rather than taking a one size fits all approach, we adapt our platform to each organisation’s unique needs and workflows.
When we partner with an organisation, we begin by carefully mapping how people move through intake, admission and transition, how changes in need are identified and responded to over time, and where information fragments or disappears along the way.
From there, we adapt UNPLEXi to reflect the organisation’s real-world context, testing workflows alongside frontline teams and refining them until they make sense in practice, not just on paper. The goal is always the same: to understand what’s actually happening, support better decisions, and measure whether the changes we make are genuinely reducing risk and improving care.
At a practical level, this means our system is designed so that outputs can always be traced back to evidence. Needs statements are linked to source documents and changes are tracked over time. Uncertainty, contradictions, and missing information are flagged, rather than smoothed over.
Better evidence leads to better decisions, and better decisions reduce crises before they happen. AI doesn’t have to be extractive to be powerful. But it does require us to be intentional about where value flows, how evidence is handled, and who ultimately benefits.
What We Should Be Demanding From AI in Care
So, if we’re going to use AI in our clinics — and let’s be honest, the AI train has left the station — then let’s also demand:
Clarity on how our data is used.
Models that don’t just extract but genuinely serve.
You can (and should) demand this from us.
Maybe JoeV2 will never be as charmingly chaotic as the original. But if we build him with the right intentions, evidence, and ethics—maybe he’ll be a useful assistant for our clinic and our clients… and not just a weirdly competent banjo-playing robot.
A.