We use AI at UNPLEXi. We’re open about that. In many ways, we’re proud of how it helps us navigate complexity, surface insight, and support better decision-making. But after finishing The New Age of Sexism: How the AI Revolution is Reinventing Misogyny by Laura Bates, pride gave way to something sharper.
FIRE. IN. MY. BELLY.
In health and social care, systems (and staff) are often stretched to cover complex work. These pressures can make the promise of AI automated efficiency, scale and speed can look like an easy answer to everything. But while AI can support better care, it can also reproduce and intensify the very biases we’re trying to undo. When that happens in systems caring for vulnerable people, the consequences are serious. That means it’s increasingly important to question and critique the way your AI operates, and what might be influencing it.
From “Coded Misogyny” to Coded Bias in Health and Social Care
The New Age of Sexism is a must read for everyone, but especially those who work in health and social services, care for vulnerable and under-represented communities, and are using AI to assist their work.
In The New Age of Sexism, Laura Bates unpacks the term “coded misogyny.” Coded misogyny refers to how misogynistic bias becomes embedded in AI through its training data, assumptions and design choices. But as she and others rightly point out, women are not the only ones affected.
Bias exists wherever power is uneven. And as our information and worldview are increasingly shaped by what we see online, it’s accurate to say we’ve entered an era of coded bias. The problem with coded bias is that the algorithms that shape our information reflect the world as it has been, not as it should be.
We were never neutral to begin with.
Bias didn’t start with AI. Every one of us brings bias into how we perceive and understand an individual’s needs. Whether it be professional training, lived experience, cultural frames, gender, or even the funding packaging a person arrives with – we are all, always, operating with some form of coded bias.
Sometimes these lenses can be useful. Training, professional knowledge, and expertise matter. But these lenses don’t just shape how care is delivered, but can also narrow what we see. Practitioners, support workers, and clinicians are almost always acting in good faith, but like all of us, are working with coded bias. When those biases go unexamined, they can inadvertently compound. Over time, this makes it harder for people to get the care and support they need.
When AI Amplifies Bias in Health and Care Systems
This is where AI can become dangerous if used uncritically. AI systems are trained on existing data, and existing data is not neutral. It reflects:
- Historical inequities
- Dominant perspectives and power structures
- Whose voices have been privileged – and whose have been ignored
If we don’t actively question the data that AI (and algorithms) are trained on, AI doesn’t neutralise or objectify bias. It amplifies it. AI outputs often appear to be polished and confident, meaning that they carry authority even when they’re wrong. That should always give us pause – but even more so in health and social care, when an incorrect output can have significant impact on indivuals lives and wellbeing.
So What Does Responsible AI in Care Look Like?
I don’t pretend to have all the answers. But there are things we can (and must) do if we’re going to use AI responsibly in this space.
1. Do your homework on the AI you use
Not all AI is created equal. Take the time to understand:
- What data it was trained on
- Whose perspectives are centred
- Whose are missing
- How outputs are generated
And critically examine what comes back — language, assumptions and framing. Bias rarely announces itself loudly.
2. Get involved in shaping ethical AI
The AI revolution isn’t something happening to us. It’s something we can influence. There is growing demand for:
- Clinicians and care professionals training AI tools
- People auditing systems for bias
- Professionals helping to re-train models where problems are found.
This work matters. To dismantle bias, vulnerable groups need their voices and experiences to be front and centre – with support from advocates who are willing to challenge systems, not just navigate them.
3. Slow, Ethical Practice Still Matters
In a world where everything is expected to happen in five seconds, it’s worth saying this out loud: Good care and good advocacy can take time. Continue to do the slow, purposeful work that put that fire in our bellies that drove us to be health and social care professionals in the first place.
How UNPLEXi Uses AI Responsibly
At UNPLEXi, we use AI, but we don’t treat it as neutral, magical, or beyond question. Our approach is grounded in a few non-negotiables:
1. Evidence comes first
- Our systems are built from international frameworks (WHO’s ICF), over 30 peer-reviewed research papers, and decades of lived experience as carers, clinicians, practitioners and neurodiverse individuals. We capture rich, nuanced data across impairment, activity, participation and wellbeing. Crucially, we track how this changes over time, not just at a single point.
2. Bias is actively interrogated
- UNPLEXi’s system flags what’s missing. It identifies gaps and inconsistencies, so users can question assumptions rather than reinforce them. We continuously refine our models using real-world feedback, with systems co-designed and tested alongside organisations in rural Australia to reflect lived, changing realities.
3. Integrity, and responsibility are essential, not optional.
- We’re acutely aware of the risks that come with extractive, opaque technology – especially in care systems where power is uneven. If something doesn’t look right, we expect you to question it and we design the system so you can. We love to hear your feedback, questions, and ideas so we can continuously improve.
Staying on the Right Side of AI in Care
AI will continue to evolve and integrate into systems. The question isn’t whether we use it — but how, why, and for whom.
For us, it’s with transparency and explicit safeguards against bias. It’s to make complexity more navigable for people and organisations alike, not to chase automation for automation’s sake. It’s for the individuals and communities whose experiences are overlooked or misunderstood by existing systems.
At UNPLEXi we’ll be keeping that fire in our bellies. It’s what drives us to question our assumptions, challenge bias where we find it, and design technology that serves people – not the other way around.
If you want to use your expertise, experience, or understanding to help design an AI system that combats existing biases, reach out. We’d love to hear what you’ve got to say.
A, and the UNPLEXi team