News | Business
28 Jan 2025 6:41
NZCity News
NZCity CalculatorReturn to NZCity

  • Start Page
  • Personalise
  • Sport
  • Weather
  • Finance
  • Shopping
  • Jobs
  • Horoscopes
  • Lotto Results
  • Photo Gallery
  • Site Gallery
  • TVNow
  • Dating
  • SearchNZ
  • NZSearch
  • Crime.co.nz
  • RugbyLeague
  • Make Home
  • About NZCity
  • Contact NZCity
  • Your Privacy
  • Advertising
  • Login
  • Join for Free

  •   Home > News > Business

    ‘AI agents’ promise to arrange your finances, do your taxes, book your holidays – and put us all at risk

    AI systems that can autonomously make decisions on our behalf will be a huge time saver – but we must deploy them with care.

    Uri Gal, Professor in Business Information Systems, University of Sydney
    The Conversation


    Over the past two years, generative artificial intelligence (AI) has captivated public attention. This year signals the beginning of a new phase: the rise of AI agents.

    AI agents are autonomous systems that can make decisions and take actions on our behalf without direct human input. The vision is that these agents will redefine work and daily life by handling complex tasks for us. They could negotiate contracts, manage our finances, or book our travel.

    Salesforce chief executive Marc Benioff has said he aims to deploy a billion AI agents within a year. Meanwhile Meta chief Mark Zuckerberg predicts AI agents will soon outnumber the global human population.

    As companies race to deploy AI agents, questions about their societal impact, ethical boundaries and long-term consequences grow more urgent. We stand on the edge of a technological frontier with the power to redefine the fabric of our lives.

    How will these systems transform our work and our decision-making? And what safeguards do we need to ensure they serve humanity’s best interests?

    AI agents take the control away

    Current generative AI systems react to user input, such as prompts. By contrast, AI agents act autonomously within broad parameters. They operate with unprecedented levels of freedom – they can negotiate, make judgement calls, and orchestrate complex interactions with other systems. This goes far beyond simple command–response exchanges like those you might have with ChatGPT.

    For instance, imagine using a personal “AI financial advisor” agent to buy life insurance. The agent would analyse your financial situation, health data and family needs while simultaneously negotiating with multiple insurance companies’ AI agents.

    It would also need to coordinate with several other AI systems: your medical records’ AI for health information, and your bank’s AI systems for making payments.

    The use of such an agent promises to reduce manual effort for you, but it also introduces significant risks.

    The AI might be outmanoeuvred by more advanced insurance company AI agents during negotiations, leading to higher premiums. Privacy concerns arise as your sensitive medical and financial information flows between multiple systems.

    The complexity of these interactions can also result in opaque decisions. It might be difficult to trace how various AI agents influence the final insurance policy recommendation. And if errors occur, it could be hard to know which part of the system to hold accountable.

    Perhaps most crucially, this system risks diminishing human agency. When AI interactions grow too complex to comprehend or control, individuals may struggle to intervene in or even fully understand their insurance arrangements.

    A tangle of ethical and practical challenges

    The insurance agent scenario above is not yet fully realised. But sophisticated AI agents are rapidly coming onto the market.

    Salesforce and Microsoft have already incorporated AI agents into some of their corporate products, such as Copilot Actions. Google has been gearing up for the release of personal AI agents since announcing its latest AI model, Gemini 2.0. OpenAI is also expected to release a personal AI agent in 2025.

    The prospect of billions of AI agents operating simultaneously raises profound ethical and practical challenges.

    These agents will be created by competing companies with different technical architectures, ethical frameworks and business incentives. Some will prioritise user privacy, others speed and efficiency.

    They will interact across national borders where regulations governing AI autonomy, data privacy and consumer protection vary dramatically.

    This could create a fragmented landscape where AI agents operate under conflicting rules and standards, potentially leading to systemic risks.

    What happens when AI agents optimised for different objectives – say, profit maximisation versus environmental sustainability – clash in automated negotiations? Or when agents trained on Western ethical frameworks make decisions that affect users in cultural contexts for which they were not designed?

    The emergence of this complex, interconnected ecosystem of AI agents demands new approaches to governance, accountability, and the preservation of human agency in an increasingly automated world.

    How do we shape a future with AI agents in it?

    AI agents promise to be helpful, to save us time. To navigate the challenges outlined above, we will need to coordinate action across multiple fronts.

    International bodies and national governments must develop harmonised regulatory frameworks that address the cross-border nature of AI agent interactions.

    These frameworks should establish clear standards for transparency and accountability, particularly in scenarios where multiple agents interact in ways that affect human interests.

    Technology companies developing AI agents need to prioritise safety and ethical considerations from the earliest stages of development. This means building in robust safeguards that prevent abuse – such as manipulating users or making discriminatory decisions.

    They must ensure agents remain aligned with human values. All decisions and actions made by an AI agent should be logged in an “audit trail” that’s easy to access and follow.

    Importantly, companies must develop standardised protocols for agent-to-agent communication. Conflict resolution between AI agents should happen in a way that protects the interests of users.

    Any organisation that deploys AI agents should also have comprehensive oversight of them. Humans should still be involved in any crucial decisions, with a clear process in place to do so. The organisation should also systematically assess the outcomes to ensure agents truly serve their intended purpose.

    As consumers, we all have a crucial role to play, too. Before entrusting tasks to AI agents, you should demand clear explanations of how these systems operate, what data they share, and how decisions are made.

    This includes understanding the limits of agent autonomy. You should have the ability to override agents’ decisions when necessary.

    We shouldn’t surrender human agency as we transition to a world of AI agents. But it’s a powerful technology, and now is the time to actively shape what that world will look like.

    The Conversation

    Uri Gal does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

    This article is republished from The Conversation under a Creative Commons license.
    © 2025 TheConversation, NZCity

     Other Business News
     27 Jan: The Government's angling to get high-value visitors to stay longer and spend more in New Zealand
     27 Jan: A tricky balance is being struck between consumers and producers - as dairy prices rise
     27 Jan: Canterbury milk processor Synlait is expecting to return to profitability in the first half of the financial year
     27 Jan: New Zealand's red meat production's predicted to decline this year
     27 Jan: A forecast drop in beef production's thought to be a positive for exporters
     27 Jan: The Trade and Investment Minister's in Australia to meet with his counterpart
     26 Jan: An expectation a turnaround in the property market is on the horizon
     Top Stories

    RUGBY RUGBY
    The Breakers are looking forward to a much needed brace of home fixtures at Spark Arena to close out their Australian basketball league season More...


    BUSINESS BUSINESS
    The Government's angling to get high-value visitors to stay longer and spend more in New Zealand More...



     Today's News

    Rugby League:
    Erin Clark is relishing a return to the Warriors for the first time since 2017 6:27

    Entertainment:
    Spencer Pratt and Heidi Montag are suing the City of Los Angeles and the Department of Water and Power (LADWP) after the couple lost their $2.5 million home in the Pacific Palisades fire 6:20

    International:
    Hamas says 8 Israeli hostages due to be released as part of ceasefire are dead 5:17

    International:
    Evacuations warnings issued for bushfires near Grampians and Little Desert in western Victoria 22:37

    Politics:
    The Government's confident the economy will reap the benefits of a change to visitor visas 21:57

    Entertainment:
    Demi Moore celebrated her first-ever Oscar nomination with a video call to Gwyneth Paltrow 21:50

    Entertainment:
    Lili Reinhart has been "searching for answers" as she battles a mystery illness which made 2024 the "hardest year" of her life 21:20

    Environment:
    Thirty homes won't have power another night - from about 150 remaining outages after a tornado tore through Northland's Mangawhai early yesterday 21:17

    Entertainment:
    Andie MacDowell is battling a painful muscle condition called Piriformis syndrome 20:50

    Entertainment:
    Jamie Foxx and Alyce Huckstepp are said to have split 20:20


     News Search






    Power Search


    © 2025 New Zealand City Ltd