A knowledge-grounded assistant that speeds up support resolution

Client Industry

Technology & SaaS · enterprise IT support

Key Challenge

Slow manual troubleshooting across scattered docs & logs

Core Tech Stack

Vertex AI · RAG · GCP Vector Search · LangChain

Deployment Model

Containerized FastAPI on Cloud Run

SoftmaxAI has developed an internal support assistant that is suitable for production use, enabling employees to deal with technical problems by posing their questions in ordinary language and obtaining answers that are based on the company's own documentation rather than being derived from error logs or dispersed sources. Using a retrieval-augmented process on Google Cloud, the most relevant sections are retrieved from a custom knowledge base through GCP Vector Search, after which a Vertex AI foundation model produces the response that is grounded in the documentation. The entire service operates as a containerized FastAPI application on Cloud Run, with LangChain handling the retrieval and generation processes and Langfuse being used to record observability and user feedback so that the quality of the answers can be measured rather than taken for granted.

• Troubleshooting meant manual digging. The technicians searched through the error logs and the various documentation in order to diagnose the problems that occurred repeatedly — carrying out a slow and reactive procedure for each query.
• Knowledge was scattered and siloed. The information needed to deal with a ticket was spread out over a number of references and there was no single searchable source, as a result of which the same problems had to be solved all over again.
• Answers drifted and stayed inconsistent. Responses differed from person to person and between sessions since there was no grounded source of truth, and it could not be guaranteed that an answer would represent the organization's real documents.
• Quality could not be observed. Since there was no instrumentation in place to show what had been requested, what had been retrieved, or whether an answer had been useful, it was impossible to assess or improve the support.

The assistant operates as a retrieval-augmented pipeline on Google Cloud, following a left-to-right data flow that starts with an employee's plain-language question, passes through serverless serving and LangChain orchestration, and then proceeds to grounded retrieval and foundation-model composition.

• Spec before code. We first worked out a written specification in accordance with the SoftmaxAI Method, the specification defining the knowledge sources, the manner in which information was retrieved, and the exact way in which the answers should be grounded before any production build was started.
• Retrieval-augmented grounding. LangChain manages a RAG pipeline which queries GCP Vector Search over the indexed knowledge base and then compiles the relevant passages, thereby ensuring that each response is constrained to the organization's own documents.
• Foundation-model composition. A Vertex AI foundation model creates clear and well-supported answers based on the retrieved context, converting the fragmented source material into a direct response to the employee's question.
• Instrumented and human-owned. Langfuse captures traces and user feedback throughout the pipeline so that the quality of the answers can be assessed and improved; throughout, each design decision remained human-owned.

• Plain-language self-service. Employees can ask questions in a conversational manner and receive a direct answer, thereby eliminating the need to manually check logs and search for documentation by providing a single point of resolution.
• Answers grounded in real documents. The responses are created using passages taken from the organization's own knowledge base, meaning that the advice is based on the source-of-truth material rather than being guesswork.
• Consistent, repeatable resolutions. Each query makes use of the same retrieval method, which provides employees with reliable answers rather than having them receive results that differ depending on who is on the other end.
• A measurable feedback loop. Langfuse's observability shows exactly what was queried, what was retrieved, and how the answers were delivered — enabling support quality to be continuously assessed and improved rather than taken for granted.
• Serverless scale on Cloud Run. Packaging the assistant as a containerized FastAPI service on Cloud Run allows it to scale elastically with demand without having to maintain any always-on infrastructure.