Go-Live

In this interview Özlem Çetinoğlu, Debasmita Bhattacharjee, Kaue Coelho, Diana Andaluz, Francesko Molla, Lyudmyla Todoriko, Kristina Bicaj and Selman Özen provide deeper insights into the final phase of Dicobot's development.

How would you describe the development of Dicobot from starting in Alpha, through Beta, to going live??

Francesko Molla: It was like taking a simple sketch in a data scientist's notebook, and then bringing it to life into a fully functional product. Along the way, I got a taste of all sorts of areas – from marketing and software development to data engineering and product development. Plus, working alongside a team with such diverse skills was amazing. Sure, it was challenging at times, but the amount I learned and the experience I gained was incredible. I'm really grateful that the company gave me this opportunity and thankful to all my colleagues for the whole experience. 

Kristina Bicaj: For our team, this has been an exciting journey, where many aspects can only be approached in a work-in-progress mode. Dealing with the testing and feedback of a product still in its development phase requires flexibility and a focused approach. Nonetheless, effective planning of the roadmap and a stable, well-coordinated team were crucial for us. Scrum has been instrumental as a great development framework, providing a solid and structured foundation for collaboration across the entire organization while allowing agility in handling new topics, impediments, or unforeseen challenges. 

Debasmita Bhattacharjee, Kaue Coelho, Diana Andaluz: The design process for the Dicobot was agile throughout these different phases and included feedback sessions for further iterations. Two of the main milestones that come to mind were the implementation of the designed solution on the Dicobot page interface and the integration of the Dicobot frame into the company's career page on the website. 

Özlem Çetinoğlu: When I joined the Dicobot project, the main components were established and working. During these two months, two major improvements were made:

1) The bot’s response to user questions with more accurate answers and almost no hallucinations.

2) The deployment process, making the cloud architecture and codebase pipeline ready for a product release. 

Selman Özen: It was a great journey; we learned a lot and gained many new insights with our data product. The development of each milestone has adapted to the circumstances and expectations of everyone involved over the course of the project. Legal aspects can also be handled with an agile approach.

What key lessons have you learned from developing Dicobot? 

Francesko Molla: Learning from Dicobot's development was a bit like navigating murky waters. When we kicked off, there was a lot of excitement around Large Language Models, but also a lot of head-scratching. Over time, we learned that LLMs aren't a one-stop-shop and General AI isn't some kind of do-it-all wizard. We figured out that making something worthwhile meant mixing and matching different bits of General AI, and the combination of these bits and pieces is highly dependent on the use case. 

Debasmita Bhattacharjee, Kaue Coelho, Diana Andaluz: One of the biggest takeaways was the importance of testing and defining the key KPIs of the product up front, and implementing ways to measure them. 

Lyudmyla Todoriko: What I learned is that we want to take care of user/chatbot input (check if there is nothing unusual coming from user/chatbot input and filter/format anything weird). Next time I would think of something different and more general to deal with this instead of trying to solve one problem at a time. This is more related to the issue of links from chatbot, where we really need to have some properly formatted URLs. 

Selman Özen: It's been a long journey - and we've managed to address legal and privacy issues in almost everything.

What tips would you give to other companies if they also want to develop an AI chatbot?

Francesko Molla: Iteration is the key word. Developing the perfect solution from scratch is not possible, as the community of developers working with LLMs is effervescent and the landscape is ever-changing. Build a solution that it’s easily applicable to different scenarios, bring it to fruition, identify the challenges. Repeat. 

Özlem Çetinoğlu: Defining clear goals would be my first tip, they will also guide the development process. If the bot's answers are based on some data source, extracting high quality data from that source is very important to get better answers. Defining the user profile is crucial both for designing the user experience and for adapting the technical design to the additional load. Last but not least, keep in mind that AI chatbots are a rapidly evolving field, so be ready to adapt your product accordingly. 

Kristina Bicaj: Consider the implementation of a chatbot in the context of the overall system. While acquiring interfaces to e.g. language model providers is relatively straightforward, strategic and technical conceptualization requires extensive engagement with complex topics such as infrastructure and architecture requirements, legal and compliance aspects, product strategy, business and usage goals, and procedural implementation. 

Debasmita Bhattacharjee, Kaue Coelho, Diana Andaluz: Extensive user research and testing, defining a framework architecture and chatbot language based solely on user requirements. This allows a chatbot to be tailored to the needs of the use case. The chatbot may also differ based on business requirements, so aligning business and user values is essential. 

Lyudmyla Todoriko: From my perspective, I would say that talking to your bot and trying to get as many edge cases as possible to see if there are any bugs would be something I would follow the next time I work in an AI chatbot. 

Selman Özen: Bring all stakeholders together early on and let them participate in the process. Legal and privacy requirements change depending on the case and circumstances.

Debasmita Bhattacharjee, Kaue Coelho, Diana Andaluz: We are looking forward to learning more and getting smarter. We want to keep an eye on the user's single message feedback to better track user satisfaction. Other improvements would be to integrate Dicobot into other areas of the website and measure its performance. Also, to implement some of the visual design solutions and run AB testing and other testing formats. 

Francesko Molla: Think of software development not as baking a single cake, but as running a bakery. In a bakery, your task isn't just to bake one cake. You constantly adapt recipes based on customer feedback and changing trends, whether it's introducing a new flavor or modifying an existing one. The bakery is always evolving, and the work is never truly finished; there's always a new challenge or a recipe to improve. Similarly, in software development, it's not about just creating a one-time product. It's a continuous process of evolution and adaptation. 

Kristina Bicaj: Initially, we will analyze user behavior (of course in compliance with GDPR) and, based on that analysis and state-of-the-art technical requirements, establish a prioritized roadmap for further development.