{"id":3824,"date":"2024-11-18T17:18:39","date_gmt":"2024-11-19T01:18:39","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=3824"},"modified":"2024-11-18T17:18:39","modified_gmt":"2024-11-19T01:18:39","slug":"customer-case-study-zipp-by-sticos-powered-by-semantic-kernel","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/customer-case-study-zipp-by-sticos-powered-by-semantic-kernel\/","title":{"rendered":"Customer Case Study: Zipp by Sticos &#8211; Powered by Semantic Kernel"},"content":{"rendered":"<p><a href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos.png\"><img decoding=\"async\" class=\"alignnone wp-image-3825 \" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos-1024x232.png\" alt=\"Image sticos\" width=\"706\" height=\"160\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos-1024x232.png 1024w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos-300x68.png 300w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos-768x174.png 768w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos-1536x349.png 1536w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/sticos-2048x465.png 2048w\" sizes=\"(max-width: 706px) 100vw, 706px\" \/><\/a><\/p>\n<p>Today we&#8217;re featuring <a href=\"https:\/\/www.sticos.no\/\">Sticos<\/a> on our Semantic Kernel blog to highlight their customer journey with AI and Semantic Kernel.<\/p>\n<h3><span style=\"font-size: 18pt;\">Who is Sticos<\/span><\/h3>\n<ul>\n<li>Company focused on helping Norwegian businesses, communes, accountants, and auditors understand regulations since 1983<\/li>\n<li>Sticos covers the market from Rv9 in Trondheim<\/li>\n<li>Sticos is a part of Visma company since 2021, with 127 employees, 70 of us are lawyers, accountants, and system developers<\/li>\n<li>We love building problem solving software for our customers who are accountants, auditors, SMEs and Public sector with over 7000 customers.<\/li>\n<\/ul>\n<p><strong><span style=\"font-size: 12pt;\">Our Vision:<\/span><\/strong><span style=\"font-size: 12pt;\"> Sticos is determined <\/span>to become the most customer driven company in the world. Together we make incomprehensible legislation easy and practical!<\/p>\n<p><span style=\"font-size: 12pt;\"><strong>What We Do:<\/strong> We combine legal expertise and technology to make it easy for businesses to comply with regulations and to make right and smart decisions.<\/span><\/p>\n<p><strong><span style=\"font-size: 14pt;\">Sticos\u2019 history in AI<\/span><\/strong><\/p>\n<p>In February 2017, before any LLM we launched a HR Chatbot. It could answer HR related questions, and you could even say to the @else mobile app that \u201cI am sick today\u201d. It would report a sick leave to the HR system and notify your closest leader. This was built using neural networks and self-made embedding models.<\/p>\n<p>Fast forward to 2023 the Open AI started releasing the large language models and generative AI, we had the know how to start using this straight away with information and self-made articles published in our encyclopedia named Sticos Oppslag.<\/p>\n<p>An easy prototype was created as a proof of concept where vector search was done in memory and the existing search engine was used. The resulting prototype worked great, and we could see that this would help our users find answers faster than in traditional search.<\/p>\n<p>We figured that we should go from prototype to production code with all bells and whistles. So, we contracted two from Visma Resolve to help us build out the Zipp API using Azure AI Search and Azure Open AI SDK. In November 2023 we released the first production version of Zipp. We saw that our code already was a bit bloated, and after listening to an <a href=\"https:\/\/www.hanselminutes.com\/921\/where-is-ai-taking-us-john-maeda\">Hanselminutes podcast episode <\/a>we learned about Semantic Kernel that was being made by Microsoft. We looked at it and realised that we could drastically simplify our code by using this enterprise ready AI framework. We also saw that our plans by expanding the API to support multiple AI pipelines with different skills would also be simplified. We decided to go ahead with switching Azure Open AI SDK with Semantic Kernel at this time.<\/p>\n<p>Using Semantic Kernel has simplified our code and made it more maintainable and futureproof.<\/p>\n<p><strong><span style=\"font-size: 18pt;\">Lessons from using Semantic Kernel for a Year<\/span><\/strong><\/p>\n<p>The technology is moving fast, and below are our learned during this year in production<\/p>\n<p><strong><span style=\"font-size: 14pt;\">A Kernel per API Request<\/span><\/strong><\/p>\n<p>For each API request we build a Kernel from scratch. We do this for multiple reasons.<\/p>\n<ul>\n<li>For scalability we had to load balance our requests to multiple Azure Open AI instances.<\/li>\n<li>To be able to gradually roll out new generative models we are able to configure which model is used per API client. This way we can activate a new and better model with a client with less API requests, see that it works and gradually roll out to the other clients.<\/li>\n<li>For GDPR reasons we have API clients that only want to have data processed inside EU, we can route these requests to Azure Open AI instances inside EU.<\/li>\n<\/ul>\n<p><strong><span style=\"font-size: 14pt;\">Experienced vs New AI Users<\/span><\/strong><\/p>\n<p>We have experienced that a user that has not much experience with generative AI tools starts querying the assistant using normal web search terms using key words to find an answer to the problem they are struggling with. For example, \u201cwhat are the requirements for stocktaking lists\u201d if you want the legal requirements for such lists for stores that are doing stock taking. Instead, we see that the more experienced users shift over to get the assistant to perform the task they are working on. From a more experienced user we see this instead<\/p>\n<p>\u201cI need to write a letter to my customers about the preparations they should make before stocktaking. Write a suggestion where you refer to the bookkeeping regulations\u2019 requirements for stocktaking lists.\u201d The assistant will instead write a full letter using the information found through our search service. This saves our more experienced users a lot of time! Since they get a head start at writing that letter and only need to change small details instead of writing it from scratch.<\/p>\n<p><strong><span style=\"font-size: 18pt;\">Zipp Powered by Semantic Kernel<\/span><\/strong><\/p>\n<p>We have created a product named Zipp which is a RAG based setup using Semantic Kernel as base framework. The product is one API with multiple client applications.<\/p>\n<p><strong><span style=\"font-size: 18pt;\">High Level Architecture<\/span><\/strong><\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-1.jpg\"><img decoding=\"async\" class=\"alignnone wp-image-3840 size-full\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-1.jpg\" alt=\"Image stico 1\" width=\"727\" height=\"207\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-1.jpg 727w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-1-300x85.jpg 300w\" sizes=\"(max-width: 727px) 100vw, 727px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-2.jpg\"><img decoding=\"async\" class=\"alignnone wp-image-3842 size-full\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-2.jpg\" alt=\"Image stico 2\" width=\"795\" height=\"556\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-2.jpg 795w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-2-300x210.jpg 300w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/11\/stico-2-768x537.jpg 768w\" sizes=\"(max-width: 795px) 100vw, 795px\" \/><\/a><\/p>\n<p><span style=\"font-size: 18pt;\"><strong>Sticos: Customer Facing solution with Semantic Kernel<\/strong><\/span><\/p>\n<p>The goal of the customer facing solution is to help our users to quickly find correct answers. Saving them time they can use on other important tasks. There are two client applications for this. Zipp which is a product and application which we own and an integration with Tripletex named Sticos Faghjelp. Both are chatbots where users can ask questions, they need answers for. Tripletex is a cloud-based ERP system with 400 000 users.<\/p>\n<ul>\n<li>Been in production since 6<sup>th<\/sup> November 2023<\/li>\n<li>Has 8,000 users across 6,000 companies<\/li>\n<li>Customer satisfaction score of 86 (Scale from 0-100)<\/li>\n<li>70 000+ questions answered so far<\/li>\n<li>The 30-day retention rate is around 70% with Zipp and 40% with Tripletex integration.<\/li>\n<li>We have seen questions at every hour of the day<\/li>\n<\/ul>\n<p><strong><span style=\"font-size: 18pt;\">Sticos Internal Tool<\/span><\/strong><\/p>\n<p>We have also made an integration with Zipp API through our Hubspot ticketing system. Our expert lawyers, accountants and auditors help our customers by answering questions directly to us. When a user sends in a question to hubspot, a ticket will be created. When a new ticket is made it kicks off a process that sends the question to Zipp and creates a note to the ticket that our experts can use as help to form a reply. This has been set up in such a way that it is an A\/B test. The control does not answer anything, and B is where a note is added with Zipp answer. This way we can measure the effectiveness of having this internal tool. After 16 000 questions we have obtained statistical significance and on average we are using 8% less time per ticket, which is around 1 FTE. As Zipp gets better we hope to improve this number as well.<\/p>\n<p>Furthermore, we have discovered that questions where Zipp is struggling to give answers actually have a significantly higher answering time than others. This indicates that if the question is difficult for Zipp it is also difficult for humans.<\/p>\n<p><strong><span style=\"font-size: 18pt;\">Future work at Sticos<\/span><\/strong><\/p>\n<p>Next year, Sticos will start making assistants for other Sticos Products which will not necessarily have a RAG setup but will use plugins far more extensively. Fetching our customers&#8217; own data and exposing it through an assistant make our users more effective in their daily work. Learn more about Sticos here: <a href=\"https:\/\/www.sticos.no\/\">https:\/\/www.sticos.no\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today we&#8217;re featuring Sticos on our Semantic Kernel blog to highlight their customer journey with AI and Semantic Kernel. Who is Sticos Company focused on helping Norwegian businesses, communes, accountants, and auditors understand regulations since 1983 Sticos covers the market from Rv9 in Trondheim Sticos is a part of Visma company since 2021, with 127 [&hellip;]<\/p>\n","protected":false},"author":149071,"featured_media":2302,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[42,1],"tags":[48,63,9,107],"class_list":["post-3824","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-customer-story","category-semantic-kernel","tag-ai","tag-microsoft-semantic-kernel","tag-semantic-kernel","tag-sticos"],"acf":[],"blog_post_summary":"<p>Today we&#8217;re featuring Sticos on our Semantic Kernel blog to highlight their customer journey with AI and Semantic Kernel. Who is Sticos Company focused on helping Norwegian businesses, communes, accountants, and auditors understand regulations since 1983 Sticos covers the market from Rv9 in Trondheim Sticos is a part of Visma company since 2021, with 127 [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/3824","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/149071"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=3824"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/3824\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/2302"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=3824"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=3824"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=3824"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}