How We Made 30 Years of AS/400 Knowledge Queryable in Slack

The situation

The system being modernized was a 30-year-old IBM AS/400 environment. It was built on Plexi, RPG programs, and a DB2 database whose field names had accumulated, layer over layer, for decades. Most of those names were cryptic by any modern standard, and the meaning behind them did not live in documentation. It lived in the heads of a small number of long-tenured developers.

That meant every meaningful question about legacy behavior, "what does this report actually do," "what triggers this field update," "where does this value come from," became a request for SME time. The modernization program's velocity was, in practical terms, a function of how much of those engineers' calendars were free that week.

The pace of modernization was governed less by engineering capacity than by the calendar of a handful of people who remembered why the system worked the way it did.

The challenge

Four parallel problems had to be solved at once:

1. Decades of undocumented behavior. Logic that mattered to underwriting, claims, and reporting was implemented across Plexi and RPG and was not explained anywhere outside the code itself.
2. Cryptic DB2 schema. Field names in the database did not map cleanly to business concepts. Anyone reading them cold could not tell what they meant or how they were populated.
3. SME bottleneck and bus-factor risk. A handful of engineers were the de facto interface to the entire legacy environment. Every modernization question competed for that interface, and the loss of any one of them would have been a material program risk.
4. Access vs. answers. Most people who had questions about legacy behavior did not need, and in a regulated environment should not have had, direct access to the legacy code or production data. They needed answers.

The approach

We built an AI agent purpose-built for legacy knowledge transfer, and we deployed it where the team already worked: Slack.

The operating model

A two-engineer ExeQut team worked directly with the client's modernization developers. The cadence was tight: discover the questions teams were actually asking, ingest and structure the legacy corpus, evaluate retrieval and generation behavior against real workflow questions, and iterate. The client's developers were partners in shaping what "good" looked like, not downstream consumers of a finished product.

The technical architecture

The agent was built AWS-native, which mattered for the security and compliance posture of a regulated insurance environment. Several foundation model families were evaluated on Amazon Bedrock, with the goal of choosing per task rather than committing to a single model up front. Amazon SageMaker was used as part of the broader AI stack supporting the solution.

Retrieval was layered. We used both Amazon OpenSearch and Amazon Kendra so that different question types (semantic, structural, lookup-style) could be served by the retrieval strategy best suited to them, rather than forcing one search pattern to handle everything. The corpus indexed against was the legacy codebase itself, the Plexi and RPG programs, together with the DB2 schema and the meaning behind its cryptic field names.

The point was not to put a chatbot on top of the codebase. The point was to make institutional memory queryable on demand.

Key infrastructure decisions

Three calls shaped the rest of the build.

The first was making AWS the home of the agent. For a regulated medical liability insurer, the security boundary, identity model, and operational tooling already existed. Standing the solution up inside that boundary was the path that did not introduce a parallel set of risks.

The second was the retrieval layering. Treating Kendra and OpenSearch as complementary, rather than choosing one, let us route the right type of query to the right index without overloading any single retrieval pattern.

The third was the access model: the developer asking a question did not need direct access to the legacy code or the database. The agent did. That separation was deliberate, and it made the tool safe to roll out broadly to product managers and engineers who would otherwise have been gated out.

Deployment and release approach

Slack was the only surface. No new portal, no new console, no new place to remember to go. Developers and product managers asked plain-language questions in the channels and threads they were already in: "what does this report do," "what triggers this field update," "where does this value come from." The agent grounded its answers in the indexed legacy corpus, including the DB2 schema, and returned them inline.

That delivery choice was as important as the model choice. A tool that lives outside the team's existing workflow tends to be used twice and forgotten. A tool that lives in the workflow tends to compound.

Meeting developers where they already worked turned the agent from a demo into a habit.

The outcome

The modernization team's day-to-day velocity stopped being a function of senior SME availability. Questions that previously sat in a queue waiting for a calendar opening could be answered immediately, in the channel where the work was happening. SMEs were freed to focus on the questions that genuinely required human judgment, rather than acting as a lookup service for things the codebase already knew.

Two effects followed from that. The bus-factor risk on the program shrank, because institutional knowledge no longer lived only in the heads of a small group of long-tenured engineers. And the pattern itself, AI as institutional memory, proved out. It is broadly applicable to any enterprise modernizing a mainframe or AS/400 environment, where the real obstacle is rarely the new stack and almost always the undocumented behavior of the old one.

What we took from it

A few things we would carry into any similar engagement:

1. Meet developers where they work. A Slack-resident agent gets used. A standalone portal does not. Distribution is part of the architecture.
2. Layer retrieval, do not pick one. Different questions want different retrieval strategies. Combining Kendra and OpenSearch let us serve semantic, structural, and lookup-style queries without compromising any of them.
3. Stay model-agnostic on Bedrock. Evaluating multiple foundation model families, rather than committing up front, kept us free to route the right model to the right task.
4. Decouple "needing to know" from "having access." In regulated environments, the agent can hold the access. The asker should not have to.
5. Treat institutional knowledge as a first-class asset. On legacy modernization programs, the tribal knowledge in a few people's heads is often the bottleneck. Naming that, and engineering around it, is its own deliverable.