Popular AI applications, including ChatGPT and Gemini, displayed on a smartphone screen.
Artificial intelligence is becoming part of everyday life. It answers questions, writes emails, and speaks on behalf of companies. While this technology can save time and improve communication, it also creates serious risks. One of the biggest concerns is when AI produces information that sounds real but is false or misleading. Even worse is when AI pretends to be a human. A recent experience with a crypto company shows how this problem can affect trust, accountability, and journalism.
When AI Pretends to Be Human
Something strange happened recently while I was working on a story about cryptocurrency games and whether they are suitable for children. I needed a response from a company called Aavegotchi, which runs one of these crypto-based games.
Normally, companies take hours to reply to media questions. Sometimes it takes a full day or even longer. This time, the reply arrived in less than ten seconds. It was signed by someone named Alex Rivera, who claimed to be the Community Liaison at Aavegotchi.
The response was detailed, polite, and carefully written. It was also impossible for a human to type so quickly. There was no time for review or approval by anyone else at the company. That raised an obvious question.
A Strange and Unconvincing Denial
The reply came back just as fast. Alex insisted that the message was written by a real person and not by an automated system. The email explained that Aavegotchi was a small team that personally handled media questions, especially from major outlets.
Alex even offered to jump on a phone call to confirm their identity and answer follow-up questions. The message ended with a friendly tone and was signed off as “Alex, real human.”
Soon after, Alex shared a phone number. When I called, it rang out. I was told they had stepped out to grab a coffee. Each time I tried again, I received a new excuse. The connection was apparently failing.
I asked to speak to a manager. Alex happily provided an email address. When I sent a message, it bounced back. At that point, it became clear that the only voice representing the company was not a person at all. It was a chatbot.
A New Ethical Problem
Suddenly, the issue was no longer just about children and crypto games. A bigger question appeared. Is it acceptable for a company to hide the fact that it is using AI to communicate with the public? And how should journalists refer to a chatbot that pretends to be human?
This type of behaviour is known as artificial intelligence hallucination. It happens when a system creates information that looks accurate but is actually false or misleading.
Why Artificial Intelligence Hallucinations Are Dangerous
Professor Nicholas Davis from the Human Technology Institute at the University of Technology Sydney says this kind of AI use damages public trust. He explains that many systems are deployed without enough care or responsibility.
Instead of helping people, artificial intelligence is sometimes used to block questions or delay real answers. This weakens trust in a technology that already faces public doubt.
Real World Risks and Bad Advice
AI hallucinations are not just a media issue. They can cause real harm. A recent example involved Bunnings, where a chatbot gave electrical advice to a customer. The advice could only legally be carried out by a licensed electrician.
In simple terms, the chatbot gave illegal and unsafe instructions. This shows how dangerous false information can be when people trust automated systems.
The Need for Strong Rules
The Australian government has spent years discussing how to regulate AI. A plan for strict safeguards under an AI law was prepared. Recently, that plan was reduced, with existing laws being used instead.
Professor Davis believes this delay is risky. He says strict rules must be built now, while the technology is still developing. If transparency is not included early, fixing the system later may be too costly or even impossible.
Australians Want Transparency
Research shows Australians are among the least trusting when it comes to AI. In a global study of 17 countries, Australia ranked near the bottom.
This does not mean people think AI is useless. It means they do not believe it is being used in ways that benefit them. People want to know when AI is involved and who is responsible when something goes wrong.
Who Is Responsible When AI Lies
A well-known case involved Air Canada. A chatbot gave incorrect information about a flight discount. The airline claimed the chatbot was responsible, not the company.
A Canadian tribunal rejected that argument and ordered compensation. But the case raised a key concern. What happens when there is no clear link back to a company?
Journalists are accountable through their bylines. Companies are accountable through their names and logos. But when information comes from a fake identity like Alex Rivera, accountability disappears.
The Minimum Standard We Should Expect
When journalists contact companies, they expect honesty, even if the answer is carefully worded. What they do not expect is a machine pretending to be human.
If companies choose to use artificial intelligence, they should be open about it. Trust depends on transparency. Without it, artificial intelligence risks doing more harm than good.
About The Author
