Mentor dos Nerds Home Generic Agents Can Be Your Company's Worst First Encounter with AI
Post

Article Artificial Intelligence

Generic Agents Can Be Your Company's Worst First Encounter with AI

A generic module fails to connect to different systems while adapted connectors pass through data, validation, and approval
A generic module fails to connect to different systems while adapted connectors pass through data, validation, and approval

The agent that promises to enter any company tomorrow may be precisely the agent that understands no company at all.

TL;DR

Generic agents aren’t necessarily useless. They can serve as a demonstration, prototype, or reusable foundation. The deception begins when that foundation is sold as a finished solution without discovering the company’s process, sources, exceptions, permissions, and responsibilities. The first experience usually fails in exactly the details the demonstration hides. The organization then concludes that “AI doesn’t work,” develops resistance, and may fall behind because of a poor implementation—not because of the technology’s actual limit. Run from anyone who arrives with the solution before understanding the problem.

Imagine the proposal.

A ready-made sales agent.

A ready-made financial agent.

A ready-made customer support agent.

Quick installation. A few integrations. A demonstration where everything happens at the right time and no customer decides to ask the question that wasn’t in the script.

It looks irresistible.

It may also be the worst first experience your company will have with artificial intelligence.

Not because agents don’t work.

Because “a generic agent for companies” may be a contradiction hidden behind a beautiful interface.

Company isn’t a sufficient noun for configuring a system.

The demonstration sells the part where everything looks alike

The common path is easy to demonstrate.

A message arrives. The agent identifies the intent. It queries information. It responds. Perhaps it updates a record. Everyone smiles at the screen.

The real problem begins later:

  • Which source is official when two systems disagree?
  • What does “active customer” mean in that company?
  • Who may grant a discount, and up to which limit?
  • Which information can support show to each user profile?
  • Which exception needs financial approval?
  • What happens when a document arrives incomplete?
  • Which action can be undone?
  • Who receives the case when the system doesn’t know?

The demonstration shows capability.

The operation demands context.

In that gap, many ready-made solutions turn into deception—even when the model is good, the interface works, and the vendor did not begin with the intention to mislead anyone.

The problem may be born from a promise larger than the architecture.

A generic job title doesn’t contain a process

“Sales agent” doesn’t describe a job.

Sales may involve prospecting, qualification, relationship history, commercial policy, calculation, inventory, deadlines, contracts, privacy, handoff to support, and dozens of exceptions that change from one company to another.

“Financial agent” is even more dangerous as shorthand. Which part of finance? Reconciliation? Collections? Invoice reading? Payment preparation? Forecasting? Approval? Each verb requires different data, authority, and controls.

“Human resources agent” can mean locating an internal policy or participating in a decision that affects rights and careers. Using the same name for both cases hides an enormous difference in risk.

An AI agent is a system capable of choosing next steps and using tools within an objective. The job title does not tell us which tools it receives, which sources it should trust, which decisions it may make, or where it must stop.

Without that, there is no digital job.

There is improvisation wearing a badge.

The generic agent fails precisely where the company is specific

It doesn’t necessarily fail at summarizing text, simple classification, or a common answer.

It fails when it encounters what makes a real business different from the sales slide:

  • a rule that was never documented;
  • two departments using the same term with different meanings;
  • an outdated record everyone learned to work around;
  • a rare but expensive exception;
  • a policy that depends on the requester’s profile;
  • an integration that works nine times and fails silently on the tenth;
  • a decision nobody wanted to own during project design.

The agent doesn’t know these things by intuition.

They must be discovered, transformed into context, and connected to the operation.

Sometimes even the company cannot explain them at first. That is why requirements gathering isn’t a bureaucratic meeting before installation. It is the work of making the process legible enough for any system—human, conventional, or AI-assisted—to operate it.

A poor first impression charges interest

After a poor implementation, the conclusion rarely stays with the vendor.

People say:

  • “AI makes up too much”;
  • “nobody trusts this”;
  • “we tried it and it didn’t work”;
  • “our company is different”;
  • “we’re better off continuing manually.”

They may be describing the system they received correctly.

The problem is carrying that experience into every future use.

There is a line of research called algorithm aversion, which investigates the tendency to reject an automated system after observing its errors. In experiments published by Dietvorst, Simmons, and Massey, participants began avoiding algorithmic forecasts after seeing them fail, even when those forecasts outperformed human ones on average.

The 2015 study addressed experimental forecasting tasks, not enterprise deployment of AI agents. It doesn’t prove that one bad chatbot experience will condemn all future adoption. I use it here as a limited lens for something worth considering: people don’t evaluate only an accuracy rate; they update trust when they see how the system fails.

An irresponsible implementation doesn’t waste only budget.

It may teach the organization to reject a capability it never had the chance to understand properly.

The company also learns the wrong behavior

When a ready-made agent is presented as magic, the team learns two equally poor options.

Either it should trust because “AI knows.”

Or it should reject everything because the first experience failed.

Neither teaches responsible use.

A mature adoption should teach people to:

  • recognize what the system was designed to do;
  • verify the source of an important answer;
  • distinguish suggestion, validation, and decision;
  • notice when a situation leaves the intended context;
  • correct the system without hiding the error;
  • route exceptions to the right accountable person;
  • measure whether the process actually improved.

The tool doesn’t enter only a technical architecture.

It enters a work culture.

If the first lesson is “click and hope,” the technology may look modern. The organizational learning isn’t.

A reusable foundation is different from a ready answer

I work with a reusable foundation myself.

It would be wasteful to rebuild version control, memory, permission management, action records, validation patterns, and automation components from scratch every time a system begins.

There are also automations that can be almost replicated when the process and tools are equivalent, requiring only configuration and integration adjustments.

The problem isn’t reusing engineering.

The problem is reusing conclusions.

A responsible foundation accelerates what is structural and keeps explicit what must be discovered:

Can be reused Must come from context
Memory and traceability structure Which information deserves to be remembered, and for how long
Tool and permission patterns Which systems the agent may access and with what authority
Validation mechanisms What a correct deliverable means in that process
Records, monitoring, and alerts Which failure matters, who needs to know, and when to act
Approval flows Who owns each decision and which limit requires escalation
Integration components Which data is canonical and how real disagreements are resolved

A ready-made product can mean mature components.

It shouldn’t mean imagined context.

What is usually missing from the box

An agent useful in production needs more than a model and a set of instructions.

It needs a harness: the system around the model that provides context, memory, tools, permissions, sources, validations, limits, and feedback loops.

In practice, that includes at least:

1. Source of truth

The agent needs to know which document, database, or system contains the official information. When sources disagree, precedence cannot be invented during the response.

2. Business vocabulary

Apparently simple terms—customer, proposal, pending, approved, completed—need the meaning used in that operation.

3. Tools and permissions

Reading is different from changing. Preparing is different from sending. Recommending is different from approving. Each action needs the least access required and, when there is a material consequence, an approval point.

4. Evaluation cases

Before production, the system needs to face a representative set of real situations: common path, missing data, contradictory information, exception, improper attempt, and tool failure. Evaluation isn’t asking the vendor whether the demonstration worked. It is comparing the result with acceptance criteria defined beforehand.

5. Observability

Observability is the ability to understand a system’s internal state from evidence such as event records, metrics, and execution traces. Without it, the agent may fail silently or produce a response whose path nobody can reconstruct.

6. A safe way to fail

The system needs to be able to abstain, request information, transfer the case, interrupt an action, and reverse what is reversible. “I don’t know” can be a security feature.

7. Decision owner

If nobody can answer who authorizes, who monitors, and who corrects, the agent wasn’t given autonomy.

It was abandoned.

The question isn’t “how much does it do in the demo?”

It is easy to impress by counting actions.

I prefer to ask:

  • In which situations must it not act?
  • Which context was used to configure the demonstration?
  • Which real cases were tested?
  • How does the system show the source of a claim?
  • Who defines and changes policies?
  • Which permissions are enabled by default?
  • What happens when an integration is unavailable?
  • Is there a test environment separate from the real operation?
  • How is a wrong action detected and reversed?
  • Under whose control do data and records remain?
  • Can the company change the model, tool, or vendor?
  • Who maintains the system when the process changes?

A good vendor should like these questions.

They help sell less fantasy and deliver more system.

Run from ready-solution vendors

Don’t run from someone who brings mature components, accumulated experience, or a reference architecture.

Run from someone who:

  • presents the solution before listening to the process;
  • calls a demonstration an implementation;
  • treats broad access as ease of integration;
  • promises autonomy without explaining authority;
  • measures success by conversation quality, not work outcomes;
  • cannot show how the agent is evaluated;
  • sells personalization as changing the name, logo, and opening message;
  • doesn’t discuss exceptions, security, records, cost, maintenance, or reversibility;
  • blames the user when the system encounters a situation that was never mapped;
  • creates dependency without offering control over context, data, and operation.

A truly mature solution isn’t afraid to reveal what still needs to be discovered.

Context comes before the agent

In its map function, the NIST AI Risk Management Framework includes the need to understand purpose, use context, users, impacts, human oversight, and the system’s intended scope. It is a broad voluntary framework; it doesn’t certify vendors or determine an agent architecture. Even so, it supports an important principle: risk and capability cannot be evaluated outside the context in which AI will be used.

Anthropic also recommends starting with the simplest solution and tailoring capabilities to the use case. This is an engineering reference from the company itself, not a universal rule. Its value is in the contrast with selling a universal agent: more complexity deserves to exist only when it improves an outcome that can be evaluated.

The agent shouldn’t arrive first and look for a function later.

The work comes first.

Then the architecture.

This is how I prefer to deliver

At i-9.ai, I can start from mature components, patterns, and automations.

But the delivery begins by understanding the operation: talking to the people who perform it, finding the bottleneck, gathering requirements, and mapping data, decisions, exceptions, and responsibility. Only then does it make sense to choose among process, integration, deterministic automation, AI assistance, an agent, or a bounded operational decision.

This is the same principle I presented in “Your company doesn’t need to figure out where to put AI”: technology enters after stuck work becomes visible.

If you received a proposal for a “ready-made agent” and want to understand what is inside it, get in touch. The conversation can start with the process, the risks, and what would need to be proven before putting any system into production.

Don’t let a poor implementation define the future

Perhaps your first experience with AI really was poor.

Perhaps the agent fabricated, got in the way, required more checking than the manual process, and disappeared when it met the first exception.

That is evidence about that design.

It isn’t proof of everything that can be built.

At the same time, don’t accept the promise that “this time it will work” as enough to try again. The second experience needs to begin with a better method, clearer criteria, and much more bounded authority.

Generic agents can make a company reject AI before it gets to know it.

The antidote isn’t believing more.

It is requiring the solution to stop being generic before it receives access to what is specific, valuable, and real.

Keep reading

Learn more

  • Intelligent agent: encyclopedic context on systems that perceive an environment and act to achieve objectives.
  • Requirements engineering: the discipline used to discover, analyze, and record a system’s needs, constraints, and criteria.
  • Principle of least privilege: the practice of granting only the access required for a function.
  • Observability: an introduction to records, metrics, and traces used to understand systems in operation.
  • Algorithm aversion: an overview of the concept and research on rejecting algorithmic systems after observed errors.

References and limits of use

The failure examples and warning signs are an authorial synthesis of architectural risks. They don’t accuse a specific vendor, demonstrate fraud, or replace a technical, legal, or security evaluation of a concrete solution.

The cover image is a synthetic editorial illustration created to contrast a generic module with an adapted, governed integration.

This post is licensed under CC BY 4.0 by the author.

Open conversation

Continue the conversation

Disagree, spot a gap, or have an experience that adds to the subject? Comment with your GitHub account. Do not publish personal data, credentials, or sensitive information.