Tech Assist AI Chatbot

Enterprise AI Solution 6 min read

The Problem: Long Wait Times, Zero Visibility

At Macquarie Group, employees visiting Tech Assist kiosks faced 10-15 minute wait times for routine issues like password resets, Global Protect troubleshooting, and software downloads. There was no queue visibility, and Tech Assist teams were overwhelmed with repetitive queries that could be automated. This resulted in over 2,500 employee hours wasted monthly and significant productivity loss across the organisation.

The core issue was clear: employees needed instant, 24/7 access to tech support for common questions, while Tech Assist needed to focus their expertise on complex technical problems that truly required human intervention.

The Solution: AI-Powered Self-Service Support

I worked with a team of 5 to design and build Tech Assist AI (TAAI), an intelligent chatbot powered by Azure OpenAI GPT-4 that provides instant tech support by searching internal knowledge bases (Confluence and SharePoint). Using retrieval-augmented generation (RAG) architecture, TAAI answers questions in under 3 seconds with source citations, reducing wait times from minutes to near-zero.

How it works:

1. User asks a question

"How do I install Slack?"

2. System searches

Hybrid vector + keyword search across 10, 000+ internal documents

3. AI generates response

GPT-4 creates contextual answer using retrieved documentation

4. User receives answer

Response with source citations in 3 seconds

5. Smart escalation

If confidence is low or issue is complex, seamlessly hands off to Tech Assist and places the customer in the queue

Key Capabilities:

Instant answers with 3 second response time

24/7 availability via web, Microsoft Teams, and SharePoint

RAG pipeline with hybrid search (vector + keyword)

Source citations from Confluence and SharePoint docs

Smart escalation to human agents when needed

Live queue tracking for Tech Assist kiosk wait times

Design Mockup

Technical Architecture: Building for Scale and Accuracy

The core challenge was ensuring accurate, trustworthy responses at scale. I architected a comprehensive RAG pipeline alongside a fellow engineer in the team that prioritises retrieval quality and response reliability:

RAG Pipeline:

Tech Stack

Frontend: React 18, TypeScript, Tailwind CSS, WebSocket-based real-time chat
Backend: FastAPI (Python 3.11), LangChain for RAG orchestration
AI & Search: Azure OpenAI GPT-4, Azure OpenAI embedding models, Azure Cognitive Search
Data Layer: PostgreSQL for conversation history, Redis for caching and performance optimisation
Cloud Infrastructure: Azure App Service, Azure Key Vault for secrets management, Application Insights for monitoring

Business Impact

The business case for TAAI was compelling. By deflecting 40-60% of routine queries, the system would save approximately 2,500 employee hours per month while operating at a fraction of the cost:

$2,270

Monthly Operating Cost

Azure OpenAI, Cognitive Search, hosting

60%+

Resolution Rate

Queries resolved without human intervention

2,500

Hours Saved Monthly

Employee time recovered

24/7

Availability

Zero wait times, instant access

Beyond cost savings, TAAI provides data insights on common tech issues, enables continuous knowledge base improvement, and frees Tech Assist teams to focus on complex problems requiring human expertise.

Implementation & Go-to-Market Strategy

My role in the team focused on end-to-end product development from problem identification through technical architecture to launch strategy. This included:

Technical Implementation:

Designed complete system architecture from data ingestion to response generation
Ideated core RAG pipeline with confidence scoring, source attribution, and escalation logic

Multi-Channel Marketing Strategy:

Email Campaign

Organisation-wide announcement on Viva Engage

Physical Presence

Posters at 10 Tech Assist kiosk locations in Sydney with QR codes for instant access

SharePoint Integration

Hero banner on homepage with embedded chat widget

Champions Network

Early adopters to drive word-of-mouth adoption

Technical Challenges & Solutions

1. Ensuring Answer Accuracy Retrieval-Augmented Generation (RAG) systems can produce inaccurate responses if not properly grounded. The team implemented prompt engineering techniques into the solution to ensure GPT-4 generated responses only from retrieved enterprise knowledge, complemented by confidence scoring based on retrieval quality and response completeness. Source citations and escalation pathways were incorporated to improve transparency and enable users to verify AI-generated responses.

2. Handling Knowledge Base Scale Supporting knowledge retrieval across more than 10, 000 documents from Confluence and SharePoint requires an efficient search strategy. A hybrid retrieval approach combining semantic vector search with BM25 keyword search was designed, using reciprocal rank fusion to improve ranking quality. This enables users to retrieve both contextually relevant information and exact technical references with greater accuracy.

3. Balancing Performance and Cost Running GPT-4 at enterprise scale requires balancing response quality with operational cost. The solution incorporates Redis caching for frequently asked questions, conservative token limits and query routing strategies to minimise unnecessary API calls. These optimisations reduces infrastructure costs while maintaining a responsive user experience and high-quality AI responses.

4. Building User Trust and Adoption Successful adoption depends on building user confidence in AI-generated responses. TAAI prioritises transparency through confidence indicators, source attribution and seamless escalation to the Tech Assist team when required. A change management strategy reinforces that the AI assistant complements existing support teams rather than replacing them, encouraging adoption while maintaining trust.

Future Enhancements

The architecture was designed with extensibility in mind, enabling future capabilities including:

Multi-language support to improve accessibility across a diverse global workforce.
Voice-enabled interactions for hands-free technical assistance.
Proactive issue detection by integrating telemetry data to identify and resolve issues before users report them.

Key Takeaway

Tech Assist AI showcases an end-to-end approach to enterprise AI solution design, from identifying a business challenge and defining measurable success criteria, through to designing a scalable RAG architecture and planning adoption strategies.

The project demonstrates the ability to bridge business needs and technical execution by combining AI engineering principles, solution architecture, stakeholder alignment and change management to deliver responsible AI solutions that create measurable organisational value.