When AI makes decisions about people

Most of us have probably used AI by now, even if only to ask a question or get help writing something.

It can be useful. Sometimes surprisingly useful.

But there is a difference between asking AI to help you with something and having AI make a decision about you.

And I think that difference is becoming more important.

We are hearing more about companies using AI to help with recruitment, monitor work, assess performance and make decisions that once depended mainly on people.

Some of this has been happening for years through ordinary computer systems. AI is bringing new possibilities, and new questions, into the picture.

The question that interests me is quite simple.

What happens when a decision affects your life, but you don't really know how it was reached?

A decision you cannot explain

Imagine applying for a job.

You spend an evening updating your CV, checking the application and making sure everything is right.

A few days later, you receive an email saying you haven't been successful.

Nothing unusual there. Most of us have experienced rejection at some point.

But suppose an AI system helped decide which applications would be considered.

You might reasonably wonder whether anyone actually read yours.

Perhaps the system was looking for particular experience. Perhaps it identified something missing. Perhaps it simply helped organise applications for a human recruiter.

You don't know.

And that's the difficulty.

Being unsuccessful is one thing. Not knowing whether you were fairly considered is another.

Of course, people make poor recruitment decisions too. Human judgement has never been perfect.

AI might even help make some decisions more consistent.

But consistency and fairness are not necessarily the same thing.

A system can apply the same approach to everyone and still overlook something important about an individual.

It's not just about jobs

Think about applying for a loan, making an insurance claim or trying to resolve a problem with a company.

There are perfectly sensible reasons for using technology in these situations.

It can process information quickly, identify mistakes and help people deal with large amounts of work.

Sometimes it may produce a better result than an overworked employee trying to get through hundreds of cases.

I don't think we should ignore those benefits.

But there is something different about being on the receiving end of a decision.

If a person tells you that your application has been refused, you can ask why.

They might not give you a satisfactory answer, but at least there is someone to question.

When a computer system plays an important part, the explanation can become harder to follow.

You may be told that the decision was based on an assessment of your circumstances.

That sounds reasonable.

But what does it actually mean?

Which circumstances mattered? Was the information correct? Did the system miss something?

And who can put it right?

Someone still has to be responsible

One thing I've noticed in discussions about AI is how easily we start talking about the technology as though it were another employee.

The AI reviewed the applications.

The AI identified the problem.

The AI recommended the decision.

That might describe what happened, but it can also make responsibility sound rather distant.

A computer doesn't have to explain itself to the person who has just lost an opportunity.

The organisation using it does.

And there is an important distinction between AI suggesting something and AI being allowed to act on that suggestion.

A system might flag an unusual insurance claim for a person to examine. That's very different from automatically refusing the claim.

It might help a manager review information. That's different from allowing it to decide that somebody should lose their job.

Those differences matter, particularly to the person affected.

The news is beginning to reflect the concern

In recent weeks, we've seen more attention being paid to AI in the workplace.

California has introduced laws addressing AI's role in employment decisions and workplace monitoring. Companies are also announcing job reductions alongside increased investment in AI, although that does not mean every job cut can be attributed to the technology.

There is a growing public discussion about where AI should be used and how much authority it should have.

I think that's a reasonable discussion to have.

But I also wonder how much of it reaches the ordinary person applying for a job or trying to understand why an important decision went against them.

Most people won't know which software a company uses.

They won't know how the system was trained, what information it considered or whether a human checked its recommendation.

And I don't think they should need to become technology experts simply to understand a decision that affects their lives.

Getting the answer right isn't the whole story

There is another side to this.

AI can make mistakes, but so can people.

An experienced employee might misunderstand an application. A manager might allow personal opinions to influence a decision. A busy department might overlook something important.

We shouldn't imagine that every decision was fair before AI arrived.

Used carefully, technology may help identify inconsistencies and give people better information.

That would be a good thing.

The concern is not simply that AI might get something wrong.

It is what happens afterwards.

Can the decision be explained?

Can someone challenge it?

Will a person with enough authority and understanding actually reconsider it?

Or will everyone simply assume that the computer must have had a good reason?

That last possibility troubles me more than the technology itself.

The person on the other side

When organisations talk about AI, the discussion often centres on speed, cost and efficiency.

Those things matter. Businesses and public services have limited time and resources.

But there is always someone on the other side of an important decision.

Someone waiting to hear about a job.

Someone trying to get an insurance claim settled.

Someone wondering why an application was refused.

To the organisation, it may be one decision among thousands.

To that person, it might be the decision they've been worrying about all week.

And even when the outcome is correct, being unable to understand it can leave people feeling that they were never properly heard.

I don't think AI has to make that worse.

It could help people receive quicker answers, more consistent treatment and fewer unnecessary delays.

But those benefits shouldn't mean losing the ability to ask a perfectly ordinary question.

Why was this decision made?

Perhaps that's something worth paying attention to as AI becomes more involved in everyday life.

Not just whether the technology can make a decision.

But whether the person affected can still understand it, question it, and find someone willing to listen.