AI Radar: Between Hallucinations and Governance – The Paradox of Trust in the Age of Artificial Intelligence

As AI models become more advanced, a contradictory reality is emerging on the technological horizon: more sophisticated systems are experiencing increasing rates of hallucinations, threatening reliability just when we need it most. This paradox is not just a technical challenge, but a warning of the urgent need for effective governance, as highlighted in recent UN reports and government initiatives around the world.

The Trustworthiness Crisis of Advanced AI Systems

The case of Cursor's AI robot, reported by S.Paulo Newspaper, is emblematic: a virtual assistant sent alerts to several customers about a policy change that simply did not exist. Even more alarming is the finding that the rate of hallucinations in new systems reached 79%, despite billion-dollar investments from companies like OpenAI and Google.

This phenomenon raises a crucial question: How can we trust AI systems for critical tasks when they are becoming less trustworthy as they get more powerful? It’s a paradox that I’ve been observing closely for years in my work with startups and technological innovation.

The Carnegie Mellon University experiment, reported by TechTudo, corroborates this concern: a company run exclusively by AI assistants failed to complete more than 75% of the tasks assigned to it. This result directly contradicts alarmist narratives about the mass replacement of workers by AI.

The crossroads of global governance

In parallel to these technical challenges, the political dimension of AI is highlighted by the report of the United Nations Development Programme (UNDP), cited by Economic Value and by Brazilian Post Office. The study is categorical: AI can both boost human development and deepen existing inequalities.

What strikes me about this report is the urgency with which it calls for active government intervention. It is not just about regulating, but actively direct these technologies towards uses that promote improvements in the overall quality of life. This vision aligns perfectly with what I have defended in my lectures and consultancies: technology, without ethical and social guidance, can easily amplify problems instead of solving them.

In Brazil, we see a practical example of this governance in the integration of AI into the judicial system. The Superior Council of Labor Justice launched version 1.06 of Chat-JT, integrating generative technology into the Electronic Judicial Process, as reported by Legal Consultant. This is a case that deserves special attention, as it brings AI into a context where accuracy and reliability are absolutely essential.

AI and business: from the billion-dollar economy to implementation challenges

While we debate issues of trustworthiness and governance, the private sector is rapidly moving towards AI adoption. Fuel distributor Vibra Energia reports impressive savings of around R$1 billion in fixed capital thanks to the implementation of AI in logistics and planning, according to the Economic Value.

This case is particularly interesting for two reasons: first, it demonstrates the transformative potential of AI for operational efficiency; second, the company is specifically targeting investments in female-founded startups, a move I celebrate for promoting diversity in the innovation ecosystem.

In my work accelerating and mentoring startups, I’ve seen a similar pattern: Companies that are able to strategically integrate AI often see significant efficiency gains, but they need to be aware of the risks of excessive dependence on systems that still have considerable flaws.

The future of work: continuous qualification as an imperative

The issue of AI’s impact on the job market continues to generate heated debates. On the one hand, we have the reassuring view of lawyer Ronaldo Lemos, who categorically states that “you will not lose your job to AI,” as reported by CNN Brazil. On the other hand, the observation of State that AI requires continuous qualification of all professional generations.

My experience with developing innovation ecosystems and training technology talent leads me to an intermediate position: We will not see a mass replacement of jobs by AI, but rather a profound transformation in the skills required for virtually every profession.

This is one of the reasons that motivated me to co-found Sirius, the first neo-university in Latin America focused on the innovative training of technology professionals. We need educational models that prepare people not only to use AI tools, but to develop the complementary skills that make us irreplaceable: creativity, critical thinking, emotional intelligence and the ability to collaborate.

AI in new territories: the case of gastronomy

A fascinating example of AI expanding into unexpected areas comes from gastronomy. As reported by Brazilian Post Office, AI is transforming the industry by optimizing processes and personalizing culinary experiences. What particularly interests me here is the delicate balance between technological innovation and cultural preservation.

Renowned chefs like Ferran Adrià are embracing AI as a creative tool, but they’re also expressing legitimate concerns about preserving culinary traditions. This tension between innovation and tradition is a microcosm of the larger challenge we face with AI across industries.

The Road Ahead: Guidelines for Businesses and Professionals

Given this complex scenario, what can companies and professionals do to successfully navigate the AI era? Based on my experience supporting more than 10,000 startups and attracting more than R$1 billion in investments, I share some guidelines that I have applied in my mentoring:

• Embrace AI with critical discernment: Implement AI systems where they truly add value, but maintain human oversight in critical processes.
• Invest in continuous training: Foster a culture of constant learning, especially in skills that AI cannot easily replace.
• Develop verification protocols: Create mechanisms to identify and mitigate hallucinations and errors in AI systems.
• Diversify your teams: Diverse teams develop more robust AI systems that are less prone to bias.
• Actively participate in the debate on governance: Companies must not only follow regulations, but contribute to their responsible formulation.

Conclusion: Navigating the paradox wisely

The current AI landscape presents us with a fundamental paradox: more advanced but less trustworthy systems; more powerful but also more risky. The key to navigating this context is not to blindly embrace the technology or to reject it out of fear, but to adopt a posture of cautious enthusiasm and active responsibility.

In my mentoring programs and speaking engagements on AI for business, I have emphasized that true competitive advantage lies not in simply implementing AI, but in implementing it strategically, ethically, and in alignment with fundamental human values. Companies that can balance technological innovation with social responsibility will be the true leaders of the next decade.

As I always tell the entrepreneurs I mentor: technology should be our ally in building a more prosperous and fair future, never an end in itself. This principle has never been more relevant than now, when we are faced with one of the most powerful tools ever created by humanity.