In my previous article, I shared why Semantic Kernel makes it straightforward to integrate AI into software without compromising security or reliability. This time, I’ll show how those principles worked in a real multi-tenant environment: Clobba Flex, a platform developed by Code Software.
Below, I’ll outline our high-level approach for turning Clobba’s Teams bot into a secure, AI-driven experience. In the next post, I’ll share code samples that detail how Azure Conversational Language Understanding (CLU) and Semantic Kernel come together under the hood.
Background: Clobba Flex’s Multi-Tenant, Role-Based Setup
Clobba Flex provides dashboards, call analytics, and license usage reports, among other features, for multiple customers on the same infrastructure. Each user may have different permissions, and no one should ever see data belonging to another tenant. Needless to say, they take security seriously and have even obtained an ISO 27001 certification, highlighting their commitment to data security. This means that while adding an AI chatbot was a logical improvement for analyzing data with natural language, we couldn't just plug in a large language model (LLM) without addressing these challenges:
Cross-Tenant Data Isolation: One user’s query must never show another tenant’s information.
Secure Handling of Customer Data: In a world where companies still block access to AI due to fears of data leaks and misuse for training AI models (which is understandable), we must assure our customers that their data will not leave our infrastructure.
Accurate Reporting: LLMs are not known for their precision; they can sometimes make things up, which can cause confusion or lead to incorrect decisions for our customers.
First Attempt: Direct LLM Access
A simple approach would have been to connect an LLM directly to Clobba’s database or a smaller set of data and let it extract information from there. However, this option doesn't address any of the three important points mentioned above, so it's not a viable solution.
Moving to Azure CLU
Our initial solution was Azure Conversational Language Understanding (Azure CLU) because it provided us with the security and precision we needed. Here’s how it worked:
Define Intents: For each query type—like “Most Busy Queue”—we trained an intent with around 30 example utterances. Examples of utterances for this case would be “What queue had the most calls this month” or “Identify the queue with the most calls today”. Azure CLU allows us to extract entities, for example, we can instruct it to detect queue names or date ranges.
Check Permissions: If CLU recognizes an intent, the system verifies the user’s role. Valid users get their data; everyone else is refused with a nice message indicating the permissions they’re missing.
Add Q&A: We supplemented with Azure Question Answering so users can ask about Clobba’s features and functionality.
You can get a better idea of this flow by looking at the picture below.
When a user asks a question, we send it to Azure CLU, which returns the identified intent along with a confidence score. We have a mapping between the intents and the required permissions needed to process that intent. If the user lacks the necessary permissions, we send a friendly message informing them to request those specific permissions and try again.
Pros of using Azure CLU
It’s fast! Each time you add or change intents you have to retrain the model and it takes a few minutes, but that’s not done very often. However, when sending a query to Azure CLU the intent is detected in 0.2-0.5 seconds.
It makes ensuring data security easy. This method keeps data secure by using Clobba’s existing role checks. Although it uses AI to train the model with those utterances, it doesn’t access customer data. Only the utterances and user questions are sent to Azure CLU, while the information from the database goes directly to the user, not to Azure CLU. This approach reassured customers about their data privacy concerns when using AI.
The cost is predictable. With Azure CLU, a Search Service is required, and while the Free Tier is available, it does not support Advanced Model Training for intents. This increases the risk of similar intents being confused, reducing accuracy.
To ensure higher precision, we use the Basic Tier of Azure AI Search with one instance, costing around $75/month. This enables Advanced Model Training in Azure CLU, allowing us to refine intent recognition and improve accuracy.
Cons of using Azure CLU
It was predictable but limited—adding new features meant retraining more intents, which was time-consuming.
This meant that the bot was pretty limited because the conversation looked like this:
As you can see, the user asks a question and the Clobba Teams bot responds straight to the point. The problem is that it's not 2015 anymore; we're in 2025. Nowadays, when people talk with a bot, they expect a ChatGPT-level conversation where they can ask for more details or have the bot explain it in a way that they can understand. How can we achieve this while providing accurate responses without risking a security breach?
Integrating Semantic Kernel and Azure OpenAI
To expand the bot’s intelligence, we introduced Semantic Kernel with Azure OpenAI:
Plugin-Based Structure
Each plugin handles a specific task (e.g., “Retrieve Most Busy Queue”, “Retrieve most busy agent”, “Retrieve all queues”, etc. ).
The LLM parses user queries and calls the correct plugin.
Inside that plugin, we check if the user is allowed to access that data using the same security model.
Enterprise Data Protection
- Azure OpenAI ensures that Code Software's customer data is not used for model training, aligning with ISO 27001 standards for secure data handling. The inputs, outputs, embeddings, and training data are not shared with other customers or OpenAI, nor are they used to improve models without our consent. The Azure OpenAI Service operates within Microsoft's Azure environment, ensuring data security and compliance with Azure's offerings.
Flexibility for Customers - we allow them to select how powerful the bot is
No Bot: Some clients opt out entirely.
Azure CLU Only: Deterministic approach with minimal overhead.
Azure CLU + Azure OpenAI: A more conversational experience under tight security constraints.
Here’s what a conversation looks like now:
As you can see, the conversation is more natural. What’s more, the user can now ask some difficult questions that Azure CLU could not interpret. An LLM is perfect at this because it can detect the context of the question and add the missing pieces without forcing the user to do that.
For example, an Azure CLU intent that requires a date range will not work unless the user specifies it.
E.g. “Show me the busiest queue for this month”
However, an LLM has the history of the conversation and knows that earlier I talked about a specific date range. Thus, the query will use that information - “January 2024”, “January 2025”.
Pros of using Semantic Kernel
Strict Security: Plugins enforce permission checks before any data retrieval, ensuring a user never sees another tenant’s info.
Maintained Precision: Rather than letting the LLM guess how to build SQL queries, each plugin executes vetted code paths.
Easy Feature Expansion: No massive retraining each time—just add a new plugin.
Natural conversation: It makes the conversation feel more natural, allowing users to speak as if they are talking to another person who understands the context.
Cons of using Semantic Kernel
It's slower than Azure CLU. Behind the scenes, Semantic Kernel first sends the response to Azure OpenAI for processing. The AI decides that plugin A must be used, and then the result from the plugin goes back to OpenAI, which returns the final response.This process can take between 1 to 5 seconds, depending on the complexity of the query, which is longer than Azure CLU. However, in my opinion, the results are worth the wait.
For debugging, it’s inherently unpredictable. Unlike compiled code, where you can reliably reproduce bugs and patch them, the non-deterministic nature of LLMs means there’s no guarantee you’ll even see the same issue twice. If an incorrect plugin call occurs, it might not happen the same way again, so you often have to accept it rather than fix it in the usual way.
Unpredictable costs. With Azure CLU, we generally know the monthly cost, but Azure OpenAI uses a pay-per-use model. Costs depend on the number of tokens processed for both inputs and outputs. This means pricing increases with usage, which can make it more expensive than CLU, but also more flexible. In a future article, I'll share strategies to save costs for both Azure OpenAI and CLU, including tips to optimize search usage and cut unnecessary spending.
Next Steps
In the upcoming article, I’ll share code samples that show how we connected Azure CLU, Semantic Kernel, and Azure OpenAI behind the scenes. You’ll learn how we enforce permissions and tackle the challenge of creating prompts that ensure precision.
