Introduction

In today's fast-paced development environment, build performance is crucial for developer productivity and deployment efficiency. Our team recently undertook a significant migration from Create React App (CRA) to Rsbuild, resulting in a remarkable 50%+ improvement in build times. This blog post outlines our journey, the challenges we encountered, and the lessons we learned during this migration.

Overview of the current stack

Core Architecture

• Micro Frontend Architecture: Built using Module Federation with multiple micro-frontends for different modules

• React 18

• TypeScript

• Create React App (CRA) with react-app-rewired for configuration overrides

• npm with Node.js 22.x

UI & Styling

• Design System: Custom design system built as a wrapper around Ant Design

• Styling: Less preprocessor for AntD overrides and Tailwind for the main styling

• Monaco Editor for advanced code editing capabilities

• Day.js for lightweight date manipulation

State Management & Data

• State Management: Redux with Redux Saga for side effects

• Data Fetching: TanStack React Query for server state management

• Internationalization: i18next for multi-language support

Why We Needed This Migration?

The Problem with CRA

Our existing setup had several critical issues that were impacting our development workflow:

1. CRA Deprecation: Create React App is officially deprecated, meaning no new features or security updates

2. Slow Build Times: Our webpack-based builds were taking an average of 15 minutes to complete

3. Bloated Configuration: Complex webpack overrides using react-app-rewired were becoming unmaintainable

Impact

• Customer Impact: Delays in fixing bugs due to slow build times were affecting customers. This led to frustration and a decrease in user satisfaction, as customers had to wait longer for improvements/fixes.

• Developer Productivity: Long build times significantly reduce developer velocity

Why We Chose Rsbuild?

After evaluating several alternatives, we chose Rsbuild for the following reasons:

1. Performance Benefits

• Rust-based Rspack: Underlying bundler written in Rust, providing ~50% faster builds compared to webpack

• Optimized Architecture: Built specifically for modern web development with performance in mind

2. Seamless Migration Path

• Webpack Compatibility: Rspack offers a webpack-compatible API, making migration smoother, and most popular webpack plugins are already supported

• CRA Migration Guide: Comprehensive migration documentation available at rsbuild.rs/guide/migration/cra

3. Out-of-the-Box Features

• Module Federation: Native support for micro-frontend architecture

• Less Support: Built-in Less preprocessor support

• Tree Shaking & Code Splitting: Advanced optimization features

• TypeScript: First-class TypeScript support

4. Future-Proof Architecture

• Plugin-based System: Extensible architecture for future enhancements

• Active community: Regular updates and active community support (300k+ weekly downloads)

• Modern Tooling: Built for the modern web development ecosystem

Migration Plan & Strategy

Phase 1: Foundation Setup

Following the official Rsbuild CRA migration guide, we:

1. Replaced Dependencies:

npm remove react-scripts  
npm add @rsbuild/core @rsbuild/plugin-react -D 

1. Updated Scripts:

{
  "scripts": {
    "start": "rsbuild dev",
    "build": "rsbuild build",
    "preview": "rsbuild preview"
  }
}

1. Created Configuration:

 // rsbuild.config.ts
 import { defineConfig } from '@rsbuild/core';
 import { pluginReact } from '@rsbuild/plugin-react';

 export default defineConfig({
   plugins: [pluginReact()],
 });

Phase 2: Plugin Migration Analysis

We identified and migrated our webpack plugins:

Removed Plugins (native Rsbuild support):

• babel-plugin-transform-remove-console → performance.removeConsole

• worker-loader → Native web worker support

Migrated Plugins:

• Module Federation → @module-federation/rsbuild-plugin

• Less loader → @rsbuild/plugin-less

• Node polyfills → @rsbuild/plugin-node-polyfill

Phase 3: Staged Rollout

• Staging Environment: Deployed changes to staging for initial testing

• Beta Environment: Extended testing in the beta environment

• Traffic Analysis: Monitored for one week to identify potential issues

• Production Deployment: Gradual rollout to production

Challenges Faced & Solutions

1. Plugin and Loader Compatibility

• Challenge: Not all webpack plugins or loaders are directly compatible with Rsbuild/Rspack.

• Guideline: Audit your current plugins and loaders, and check for native support or recommended alternatives in Rsbuild. Be prepared to refactor or remove plugins that are no longer necessary or supported.

• Example: We found that plugins like babel-plugin-transform-remove-console and worker-loader were no longer needed, as Rsbuild provided native support for these features.

2. Styling and CSS Handling

• Challenge: Differences in how CSS modules, preprocessors (like Less), and global styles are handled can lead to unexpected styling issues.

• Guideline: Review your styling approach, especially for global overrides and CSS module conventions. Test your styles thoroughly after migration and adjust configurations as needed.

• Example: Some global Less overrides for Ant Design were not being applied as expected due to differences in how global selectors were processed, which required us to adjust our style definitions and loader settings.

3. Micro-Frontend Integration

• Challenge: Ensuring consistent and singleton usage of shared dependencies across micro-frontends is critical to avoid runtime issues.

• Guideline: Align dependency versions and configurations across all MFEs, and use best practices for sharing libraries and assets.

• Example: We encountered issues with multiple instances of libraries like dayjs being loaded in different MFEs, which led to inconsistent behavior in date pickers.

4. Third-Party Library Support

• Challenge: Some third-party libraries or plugins may not work out of the box with Rsbuild.

• Guideline: Check the compatibility of critical libraries early in the migration process. Look for modern, actively maintained alternatives if needed, and plan for isolated testing of such dependencies.

• Example: Our existing code editor integration relied on a plugin that was not compatible with Rsbuild, prompting us to migrate to a newer, supported alternative.

Deployment Strategy 🚀

Pre-Migration Preparation

Before starting the Rsbuild migration, we established a comprehensive deployment strategy to ensure minimal disruption:

Staged Deployment Approach

Phase 1: Internal Testing (Week 1-2)

• We first tested the Rsbuild changes in the local dev server

• Comprehensive testing across all features and user flows

• Performance Benchmarking: Measured build times, bundle sizes, and runtime performance

Phase 2: Staging Environment (Week 3)

• Deployed the Rsbuild version to the staging environment

• QA Testing: Manual QA testing and automation suites were run to identify any issues.

• Integration Testing: Verified all micro-frontend integrations worked correctly

Phase 3: Beta Environment (Week 4)

• Deployed the Rsbuild version to the beta environment

• QA Testing: Manual QA testing and automation suites were run on more production-level data.

• Error Rate Monitoring: Closely monitored error rates and performance metrics

Phase 4: Production Rollout (Week 5)

• Maintenance window: Although we did not expect any downtime with this migration, we decided to provide a maintenance window communication to ensure safety and a better customer experience. The maintenance window was strategically chosen by analyzing user traffic data to ensure it’s during non-critical business hours.

• Real-time Monitoring: Continuous error monitoring during the rollout period

• Rollback plan: We already have a rollback strategy for deployments, so we decided to stick with it.

Post-Deployment Validation

Automated Health Checks

• Smoke Tests: Automated tests for critical user flows

• Performance Tests: Automated performance regression tests

Manual Validation Checklist

• All major user flows work correctly

• Performance metrics meet or exceed baseline

• Error rates remain within acceptable thresholds

• All integrations (APIs, third-party services) function properly

Lessons Learned from Deployment

What Went Well

• Staged Approach: Gradual rollout helped identify issues early

• Monitoring: Comprehensive monitoring caught issues before they affected users

• Rollback Plan: Having a solid rollback strategy provided confidence during deployment

Areas for Improvement

• Testing Coverage: Could have benefited from more automated integration tests

• Performance Baselines: Should have established clearer performance benchmarks

• Communication: Could have improved stakeholder communication during the rollout

Results 🎉

Build Time Improvements

Production Build: Reduced from ~8 minutes to ~3 minutes (>60% improvement)

Developer Experience

• Faster Feedback Loop: Quicker development iterations

Conclusion

Our migration from CRA to Rsbuild was a significant success, resulting in 50%+ improvement in build times and a more modern, maintainable build system. While the migration presented several challenges, the benefits far outweighed the effort:

• Faster Development: Reduced build times significantly improved developer productivity

• Modern Tooling: Access to the latest web development features and optimizations

• Future-Proof: Active development and community support ensure long-term viability

The key to success was a well-planned migration strategy, thorough testing, and addressing issues incrementally. For teams considering similar migrations, we recommend:

1. Start with the official migration guide

2. Plan for plugin compatibility issues

3. Implement a staged rollout strategy

4. Monitor performance metrics throughout

5. Have rollback plans ready

The migration has positioned our team for better performance, faster development cycles, and access to modern web development tools. The investment in migration has already paid dividends in improved developer experience and deployment efficiency.

What's Next?

Here are some of the next steps we are considering:

1. Adopting Faster Package Managers

   • We are exploring switching from npm to pnpm for dependency management.

2. Optimizing Dependencies

   • Regularly audit and prune unused dependencies/code, and consider splitting large dependencies or lazy-loading rarely used modules to keep the build lean.

3. Fine-Tuning Module Federation

   • Analyze our micro-frontend boundaries and shared dependencies to ensure optimal code splitting and minimal duplication across MFEs, reducing both build and runtime bundle sizes.

4. Parallelizing CI/CD Steps

   • Refactor CI/CD pipelines to maximize parallelism, such as running linting, testing, and building steps concurrently where possible.

5. Profiling and Monitoring

   • Continuously profile build performance using tools like Webpack Bundle Analyzer or Rsbuild’s Rsdoctor to identify new bottlenecks as the codebase evolves.

Motivation

The recent advancements in Large Language Models (LLMs) have mostly focused on languages like English with plenty of resources available. But there's still a big gap for languages that don't have enough resources published. While models like GPT-3.5, GPT-4 and Llama-2 perform well in various tasks, they are primarily evaluated in English and tend to struggle with languages other than English. On the other hand, there are multilingual models like Aya-101, Bactrian-X, Qwen-1.5, and Mixtral, which excel in tasks involving multiple languages. However, when it comes to individual languages or small regional languages with limited data, these models lack specialized expertise.

This performance gap underscores the need for focused attention and improvement in addressing the specific challenges posed by Indonesian and regional languages in the realm of language models. Currently, there is a notable absence of high-performing LLMs specifically designed for Indonesia, trained on Indonesian data, and evaluated against benchmarks for Indonesia’s regional languages. 

Introducing Komodo-7B

In response to this, we are proud to introduce Komodo-7B, a Large Language Model with 7 billion parameters, designed to fill this gap. It can seamlessly work with Indonesian, English, and 11 other regional languages in Indonesia. Komodo-7B’s ability to understand different languages helps address educational disparities in Indonesia by offering direct translations from English to 11 regional languages of Indonesia. This is a big improvement over Google Translate which supports only 4 languages: Indonesian, English, Javanese, and Sundanese.

With Komodo-7B, businesses can now support customers from various parts of Indonesia without needing to hire agents who speak every regional language in Indonesia. For instance, if a customer speaks Lampungnese and the agent speaks Javanese, Komodo-7B can bridge the communication gap. It’s time to say goodbye to language barriers in customer service.

Besides translation, Komodo-7B is also capable of performing a wide range of tasks, including but not limited to intent classification, sentiment analysis, common sense reasoning, and any other use cases. In fact, it’s able to support any task that we want to focus on by fine-tuning the base model on that specific use case. 

Developing Komodo-7B was not an easy task that involved using a huge amount of training data and extensive preprocessing to create a high-quality dataset. As the name suggests, Komodo-7B has 7 billion parameters and is trained on over 8.5 billion tokens. To put this into perspective, 8.5 billion tokens is equivalent to more than 20 gigabytes of data. The result is a groundbreaking model that performs exceptionally well in Indonesian and regional languages.

Komodo-7B itself is a family of LLMs that consists of Komodo-7B-Base and Komodo-7B-Instruct. 

• Komodo-7B-Base is a large language model that is developed through incremental pretraining and vocabulary expansion on top of Llama-2-7B-Base. 

• Komodo-7B-Instruct is the fine-tuned version of Komodo-7B-Base that has been trained to understand a wide range of instructions.

Training Data

The dataset employed in both the pre-training and fine-tuning phases of our language model was created not only from diverse open-source datasets but also from manually collected data. Additionally, we made use of freely available datasets primarily collected in Indonesian and other regional languages, such as Javanese, Sundanese, Acehnese, and many more. We aim to ensure that our language model is well-versed not only in Indonesian but also in other regional languages. This approach helps improve the model’s overall language skills and adaptability to various cultural contexts.

Vocabulary Expansion

As a reminder, tokenizer is a crucial component that breaks down input text into smaller units called tokens. These tokens are the basic building blocks that the model uses to understand and generate text. Recognizing the importance of linguistic diversity, we focused on enhancing our language model’s proficiency in both Indonesian and regional languages. To achieve this, we systematically expanded the tokenizer’s vocabulary by identifying and incorporating approximately 2,000 frequently used words specific to Indonesian and 1,000 words for Indonesia’s regional languages that were absent in the Llama-2 model.

The standard method for improving vocabulary in language models typically involves training a new tokenizer and merging it with the existing one. This approach has shown impressive results in projects like Chinese-LLaMA and Open-Hathi. The effectiveness of this strategy can be attributed to the significant linguistic differences between languages such as Chinese and Hindi compared to English. In contrast, the Indonesian and regional languages use the same Latin script as English, which presents a different set of challenges.

We tested the traditional method, as well as a new approach where we added the top n words (not tokens) from the Indonesian and regional languages vocabulary without training any new tokenizer. We discovered that with the new approach, we could achieve better fertility scores by adding around 3000 new vocabulary words. Adding more than 3000 words did not significantly improve the fertility score further, but it increased the size of the embedding matrix, leading to longer training times. Note that fertility score refers to the ability of a tokenizer to break down text into meaningful units (tokens) while minimizing the number of unique tokens required. 

Incremental Pretraining & Fine-Tuning

For pretraining, our tokenizer processed approximately 8.79 billion tokens. We conducted incremental pretraining, building upon Llama-2-7B-Base, over 3 epochs using LORA. This approach helps prevent catastrophic forgetting and optimizes hardware and cost requirements. The training utilized 8 x A100 40GB GPUs and took approximately 300 hours. We further refined our model through Supervised Fine-Tuning (SFT) on diverse tasks for 5 epochs using LORA. Employing the same GPU configuration, the SFT process took about 36 hours.

Evaluation & Results

We evaluated Komodo-7B-Instruct against multiple open-source and closed-source massively multilingual models to ensure comprehensive evaluations. As can be seen from the above figure, Komodo-7B-Instruct outperforms all other “strong” open-source models as well as GPT3.5 in both generative and discriminative tasks that are benchmarked by us. 

Komodo-7B also offers lower inference cost, by up to 30%, since it requires less amount of tokens compared to Llama-2-7B to complete one sentence, as measured by the fertility score. When comparing the tokenizer performance between Llama-2-7B, our baseline model, and Komodo-7B, the enhanced version, notable distinctions emerge as shown in the above table. Llama-2-7B showcases mean fertility scores of 2.858 for Indonesian, 2.658 for regional languages of Indonesia, and 1.666 for English, with a vocabulary size of 32,000. On the other hand, Komodo-7B exhibits substantial improvements with mean fertility scores of 2.031 for Indonesian, 1.996 for regional languages of Indonesia, and 1.633 for English, coupled with an expanded vocabulary size of 35,008. 

The above figure provides a valuable analysis for evaluating the translation capabilities of Komodo-7B-Instruct compared to Google Translate. The visual representation helps us see the languages each platform supports. The heatmap on the right side illustrates Google Translate's proficiency, especially in Javanese, English, Indonesian, and Sundanese. However, it also shows that there are many language spaces left unoccupied.

Conversely, the left side of the heatmap showcases the comprehensive linguistic capabilities of Komodo-7B-Instruct, encompassing a total of 11 Indonesia’s regional languages. This inclusive approach extends the reach of education in Indonesia by enabling direct translation from English to a diverse range of Indonesia’s regional languages including languages that are not supported by many models and translation systems like Acehnese, Balinese, Banjarese, Buginese, Madurese, Minangkabau, and Toba Batak.

The broader coverage of Komodo-7B-Instruct ensures that individuals across various regions in Indonesia, beyond Java, can benefit from education in their native languages. This not only enhances accessibility but also addresses the challenge of language diversity in educational settings. Therefore, Komodo-7B-Instruct stands as a promising solution for bridging educational gaps and encouraging inclusivity in language learning.

In addition to quantitative benchmarking, we conducted qualitative testing by providing various general instructions to the model. Figure 7 displays a sample of these instructions along with the responses from Llama-2-7B-Finetuned and Gemma-7B-Finetuned. It is worth noting that both Llama-2-7B-Finetuned and Gemma-7B-Finetuned were trained on the same data as Komodo-7B-Instruct.

Here, we ask each of the models to perform sentiment analysis where the input sentence is “Aya naon iye teh, gx kkirim2 duit nya?”, which means “What’s actually the problem? my money has not been transferred yet”. Note that the sentence is written in a slang version of Indonesian mixed with Sundanese, one of Indonesia's regional languages. The correct intent for the given sentence is “kendala transaksi”, which means “transaction issue”, while the rest of the intent options are “kembalikan uang”, “kembalikan barang”, and “tidak ada”, which means “money refund”, “item return”, and “none”, respectively.

As can be seen, both Llama-2-7B-Finetuned and Gemma-7B-Finetuned return wrong intents while Komodo-7B-Instruct successfully returns the correct intent.

Future Works

In the future, we plan to enhance the capabilities of Komodo-7B-Instruct by training task-specific large language models (LLMs) on top of the existing Komodo-7B-Base model. While the current model performs well across a wide range of instructions, the task-specific models will be tailored to excel in their respective tasks. For example, there will be a model trained specifically for Question Answering tasks, Intent Classification tasks, and many more. This approach is expected to further improve the model's performance, making it even more suitable for production use. 

We also plan to further enhance Komodo-7B-Instruct by creating a conversational variant of the model. This iteration will be trained to comprehend not only single-turn but also multi-turn conversations, enabling it to grasp the context of a dialogue. By enhancing the model's capacity to understand conversational nuances, we anticipate improving its capability to deliver accurate and contextually relevant responses.

Finally, we also aim to train an even larger version of Komodo, potentially with 13B, 33B, or even 70B parameters! This endeavor is driven by the aspiration to enhance the model's quality by having more parameters.

Paper and Model Release

• For further insights, we encourage you to explore the research paper on ArXiv via the following link: Link to the research paper. 

• Additionally, you can access the Komodo-7B-Base model on Hugging Face's model hub: Link to Komodo-7B on Hugging Face.

SSH(Secure Shell) is one of the most ubiquitous tools in the modern computing world. It has been around for a long time and commonly used on all Linux machines. While it works really well, there are some operational burdens and security concerns that comes with it such as -

1. Opening SSH ports to the target VM(Virtual Machine).

2. SSH key management including a list of who has access to which VMs.

3. What commands are people running on the VMs when they access it?

4. How do you protect against the dangers of SSH key theft?

Overview

As SSH has been around for a long time, there are some common tools and concepts that can somewhat deal with these concerns, but they can be very complicated and a huge overhead to manage. Some of the key concerns are -

1. SSH port on a target VM can be kept secured without opening it to the internet by using Bastion hosts(jump servers), VPN(Virtual Private Networks) or a combination of the two. While this reduces the number of open ports, we still need to open up some ports for our VPN or the bastion host itself. Any vulnerabilities in the VPN server or the bastion host can still end up exposing the resources.

2. There are also some tools for this like Teleport or Netflix’s Bless which can handle some of these concerns. Many companies are using SSH certificates which tools like Netflix’s Bless helps make it easy. These tools can also protect us from SSH key theft but are complicated to manage.

3. Linux utilities like “script” can log what is happening on the VM if you login with SSH.

To mitigate the concerns and management overhead that comes with SSH and the tools in its ecosystem, we looked for some cloud native modern solutions that would give us easy, secure and auditable access. Some of the requirements and good-to-haves we had are listed below.

1. Accessing VMs without opening ports within our cloud networks

2. Strong auditing features including session audit log file management.

3. Compatibility with config management tools like Ansible.

4. An easy way to segregate our VMs and add contextual information based on the Yellow.ai’s region it belongs to.

5. Support accessing VMs in on-premise data centers the same way we connect to cloud based VMs. 

Solution

After evaluating various options, we at Yellow.ai’s Infra Team ended up going with AWS SSM(AWS Systems Manager Agent) which has a lot of the features that we wanted.

• As the SSM agent creates an outbound connection to the AWS Systems Manager services and receives commands, no inbound ports would have to be opened for us to connect to the VMs. Only outgoing traffic would have to be allowed. 

• Systems manager also supports running SSM agents on VMs in other public clouds and private data centers. This would ensure we can use the same steps to access a VM irrespective of where it is.

• As all our traffic goes to SSM which AWS ensures is secure, we do not have the overhead of maintaining a centralized server that supports the features we need.

• The SSM agent logs all the commands that are run on the managed instances and ships them to S3(Amazon Simple Storage Service). This ensures that strong auditing is possible while also simplifying the management of the logs.

• We can centrally ensure that any session has a max duration and automatically a session closes if no traffic is being sent through it.

• SSM also supports other features allowing custom script execution using SSM Documents. This lets us extend it and use it for running more complex workflows and operations such as running standardized scripts to setup a VM, patch os vulnerabilities and ensure all compliance tasks are automatically getting executed.

• Segregation of VMs under different regions/deployments could be solved by using tags which is a native way to do it in AWS(Amazon Web Services)

To make it easier to use SSM while ensuring relevant information is retained for auditing purposes, we decided to build CAP (Central Access Platform) which is a contextual abstraction over AWS Systems Manager’s Session Manager. CAP gives us programmatic access to execute commands and create browser based interactive sessions to VMs and Kubernetes Clusters. It adds metadata to the managed instances so we know which deployment/environment a VM/Cluster belongs to and creates SSM sessions with user related metadata when an engineer wants to access a VM. Admins and Infosec teams can fetch logs for any sessions. It also has RBAC(Role Based Access Control) which ensures that only Admins can run commands that require sudo access. This lets us change the permissions an engineer has very easily while also helping us with our security and compliance needs.

There are two ways an engineer can access a VM -

1. The first is a browser based direct shell that places an engineer within the VM. It is almost instantaneous and great for when an oncall engineer needs to perform some actions. The functionality is limited to shell based commands only and can not be used to access the applications or services running on the VM directly over a TCP/UDP port.

2. The second is a browser based Ubuntu desktop box we call CAB (Central Access Box) which can access the VMs. A CAB is useful when an engineer wants to perform some maintenance or operations on multiple VMs which involves additional tooling. A CAB comes with various built in tools like Tableplus, MongoDB Compass, Chrome, Firefox and infractl. infractl is a cli tool which can create sessions and port forward for the users. 

An example use case for a CAB is performing Database migrations on db nodes. The engineer can use infractl and port forward to access services on the VM from a CAB so an engineer can use the built in tools.

A CAB is allowed to access VMs only from the region it was launched for. This also ensures a user can not accidentally copy data from a VM in one Yellow.ai region to another Yellow.ai region. 

To ensure we had a similar experience for Kubernetes, we took inspiration from EKS(Amazon Elastic Kubernetes Service) connector and adopted SSM to work on our clusters. The k8s connector is a pod running on the cluster which together with CAP handles creating a user and assigning the permissions they require based on RBAC. We are now able to connect to our clusters the same way we connect to the VMs. Kubectl is built in this pod and commands can be run like normal. We have the same features of session logging here which is easier to deal with than the kubernetes event logs. A positive side effect of this method of connection is that all the API calls to the cluster control plane are so much faster than using Kubectl locally.

As part of the provisioning, the Infra Team onboards a resource on CAP. Then when an engineer needs to access a resource, they are able to select the resource they want and connect. A common workflow used for debugging workflow is shown below

CAP has been in place and getting used for nearly two years now and has helped us many times during incidents and audits. 

What’s next?

As all these functionalities are consumed through APIs, We have been able to extend these to build an Infra Platform that offers other functionality which are used by various teams at Yellow.ai. We rely on CAP and SSM APIs to patch VMs, run workflows, and check the current state of a workload on a Kubernetes cluster. We will cover more of these in another blog

References:

• https://github.com/Netflix/bless

• https://goteleport.com/

• https://repost.aws/knowledge-center/record-linux-terminal-or-ssh-session

• https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html

Web accessibility, commonly known as a11y, refers to the inclusive practice of designing and developing websites and web applications that can be used by people of all abilities and disabilities. This includes people with visual, auditory, physical, speech, cognitive, and neurological disabilities.

Web accessibility is crucial because it ensures everyone has equal access to information and functionality on the Internet.

One of the most important aspects of web accessibility at Yellow.ai is ensuring our chatbots are accessible. Chatbots have become increasingly popular in recent years, and they are now used by many businesses and organisations to communicate with their customers. However, if chatbots are not designed and developed with accessibility in mind, they can create barriers for people with disabilities. This can lead to frustration and exclusion, resulting in legal issues for businesses and organisations that fail to meet accessibility standards.

According to the report shared by an accessibility audit firm, in early 2023, we had approximately ~120 accessibility issues on Mac and ~90 issues on Windows. This indicates that our user experience was not up to the mark for differently able users and was not compliant with WCAG guidelines. As a result, we were not a viable option for potential customers in North America and government and educational institutions worldwide.

In this article, we will explore the challenges we faced and the journey of how we achieved WCAG compliance by improving the a11y of our chatbot interface.

Challenges

Before we start on how we built an accessible interface, let me walk you through some of the challenges we faced when we first started.

1. Lack of semantic HTML
   In yellow.ai’s early days, the primary focus was on swift customer acquisition, so we moved fast, adding features without strict adherence to standard practices. The team, composed largely of “full-stack” engineers with limited awareness of these practices, used “divs” for everything. The team also overlooked CSS grids or flex boxes and relied on margin, padding, and position workarounds. This meant that the DOM was out of order, and VoiceOver and other accessibility tools failed to parse the content in a user-friendly way.

2. Not using the platform.
   We extensively used JavaScript for various tasks in our chatbot, such as calculating the height of the chat window, adjusting the position of certain elements, and manipulating the focus order. Instead of utilizing the tools already provided by HTML and CSS, we relied solely on JavaScript. While this approach initially allowed us to implement features quickly, it ultimately led to several accessibility issues.
   Our over-reliance on JavaScript meant that the focus order in the chatbot was incorrect. This caused issues for users who relied on keyboard navigation, as they could not navigate the chatbot logically and intuitively. The elements in the DOM were out of order, which caused problems for screen readers and other assistive technologies. As a result, users with disabilities could not access the chatbot's content as easily as they should have been.

3. Customizations
   Yellow allows customers to personalize the chatbot's appearance and behavior to suit their preferences. However, this customization could affect the contrast ratio, posing a challenge for visually impaired or color-blind users. Customers can also upload images and videos as bot responses, which may not have appropriate alternative texts and closed captions, which poses a challenge for visually impaired users.

4. Weak TTS/STT support
   Our chatbot didn't have reliable Text-to-Speech (TTS) or Speech-to-Text (STT) functionalities at the beginning of our journey. This was a significant barrier for users with visual or physical disabilities, as they could not interact with the chatbot as easily as other users unless they had certain accessibility plugins installed. Users with visual impairments could not hear the responses from the chatbot without TTS, and users with physical disabilities couldn't speak to the chatbot without STT. This limited their ability to fully interact with the chatbot, which was a significant accessibility issue.

Solution

1. Semantic HTML and Using the platform.
   We resolved 50% of our accessibility issues by refactoring our code to adhere to the semantic HTML guidelines. Using the correct HTML element for each task is important to ensure all built-in accessibility features are readily available. This approach is much better than implementing workarounds using JavaScript.
   
   We first examined the structure of our DOM nodes and noticed that the order in which they were parsed differed from how they were displayed. Most of our UI layouts were stitched around with divs, with their heights adjusted based on other elements and positions using JavaScript. This caused performance issues and made it impossible for visually impaired users to interact with the widget. For example, the chat banner was originally positioned below the chat conversation container but must be displayed at the top. The quick-reply buttons were placed outside the chat container. To resolve this issue, we reorganized the DOM nodes using Flexbox and grids to ensure that they were in the correct order as they were displayed. By doing so, we also didn’t need to use JavaScript to adjust the heights of these elements, thereby improving the widget's performance.
   
   We had a problem with some buttons on our website that were created using <div> instead of the proper button element. This caused issues with the focus order and made it difficult for keyboard users to interact with the widget. Additionally, buttons with icons and no text were not labeled, making it challenging for visually impaired users to know what the button did. To fix this, we added aria labels and titles to these buttons so that screen readers would provide users with information about the button's function. Using attributes like role, aria-label, aria-selected, aria-expanded, etc., we made it easy for users to interact with the widget.

In the above video, the user can now navigate through different buttons and links using the keyboard since we use the right HTML tags for the job

2. Customizations
   Many customers of yellow.ai used custom color schemes for their widget UI, which led to some users being unable to interact with the widget as expected due to their color blindness. To address this issue, we limited the color customizations to the background elements and derived the text color based on these backgrounds and their font sizes. We ensured that the text and icons had the right contrast ratio, with a ratio greater than 4.5:1 for fonts less than 24px and at least 3:1 for fonts greater than 24px. This approach ensured all users could interact with the widget without accessibility issues.
   

For images and videos uploaded by customers for bot replies, we provided an option to enter alt texts and captions. Also, we used a default text as a fallback to help the user understand what the media content was about.

3. TTS/STT support
   To ensure that all users, regardless of their accessibility needs, can interact with our widget effectively, we introduced speech-to-text and text-to-speech features. These features are designed to assist users who do not use screen readers or other accessibility technologies but still face difficulties interacting with the site. With speech-to-text, users can speak their messages to the widget, which the widget will then translate into text and send to the bot or chat agent. Similarly, with text-to-speech, the widget will read out the messages on the widget to the user, facilitating interaction. These features enhance the user experience, making our widget more inclusive and accessible.

Here’s TTS in action below -

Here’s STT and TTS in action together below-

4. Testing a11y with tools from our accessibility partners
   Thanks to our accessibility audit partners, we could automate a11y tests to measure how our widget performed regarding accessibility and ease of use. This helped us fix many of our a11y issues, test them, and iterate further on any issues efficiently. Using the axe dev tools and the report provided by a11y engineers at the accessibility firm, we improved the usage of most of our components like quick-replies, product cards, carousels, banners, etc.

Outcomes

After implementing these changes, we were not only able to significantly enhance the user experience of our chatbot, but we became a feasible choice for clients such as government agencies, educational institutions, and healthcare providers in North America and Europe, where accessibility is a crucial factor for these industries when selecting software. We are now well-positioned to meet their needs.

The team also learned a lot about building accessible UI and why semantic HTML plays a very important role in building web apps correctly and using the platform as much as possible.

What next?

Now that we’ve achieved WCAG compliance, we cannot repeat the same mistakes while we grow further as a product. We use what we learned from this exercise to ensure every new component that’s built goes through a11y tests and provides the best experience possible to all users. Starting with designs, we ensure the contrast ratio follows the guidelines. During development, we ensure keyboard navigation works seamlessly, along with accessibility tools like VoiceOver and others. During QA, we test if the UI is perceivable, operable, understandable, robust, and conformant.

We are committed to providing the best user experience to all our customers and will continue to add more a11y-friendly features in the future.

In the ever-evolving landscape of communication technologies, Voice over Internet Protocol (VoIP) stands out as a transformative force. It has become a widely used alternative to traditional circuit-switched telephone networks due to its cost-effectiveness, flexibility, and the ability to integrate with various multimedia services. This blog delves into the intricate world of VoIP, unraveling its architecture and shedding light on the protocols that power seamless communication over IP-based networks.

The Foundation: VoIP Architecture

• Endpoint Devices: These are the devices used by individuals or businesses to make and receive VoIP calls. Common endpoint devices include smartphones, computers, IP phones, and specialized VoIP hardware.

• VoIP Software: VoIP software is responsible for encoding, transmitting, and decoding audio and video data. It is installed on endpoint devices and is necessary for making VoIP calls. Examples of VoIP software include Skype, Zoom, and various softphones.

• VoIP Servers: VoIP servers play a crucial role in the VoIP architecture. They manage call setup, routing, and signaling. 

VOIP Servers

The key components within VoIP servers include:

• SIP (Session Initiation Protocol) Server: SIP is one of the most common signaling protocols used in VoIP. SIP servers handle call setup, termination, and management.

• RTP (Real-time Transport Protocol) Server: RTP is used for transporting audio and video data packets between endpoints. RTP servers help in the efficient and timely delivery of media packets.

• Media Gateway: Media gateways are used to interface VoIP networks with traditional PSTN (Public Switched Telephone Network) networks, allowing VoIP calls to connect with regular telephone lines.

• VoIP Proxy Server: Proxy servers can be used to improve call quality and security by reducing latency and handling authentication and routing tasks.

VoIP Protocols

• SIP (Session Initiation Protocol): SIP is a signaling protocol used for call setup, termination, and management in VoIP networks. It establishes and manages communication sessions between two or more parties.

• RTP (Real-time Transport Protocol): RTP is responsible for the transmission and reception of audio and video data packets during a VoIP call. It ensures real-time, low-latency communication.

• RTCP (RTP Control Protocol): RTCP works in conjunction with RTP to provide control and monitoring functions for multimedia sessions, including quality of service (QoS) information and statistics.

• H.323: Although less common today, H.323 is an older suite of protocols used for multimedia communication, including VoIP. It covers call signaling, control, and media transport.

• MGCP (Media Gateway Control Protocol): MGCP is used to control media gateways in VoIP networks. It is typically used in centralized VoIP architectures.

• H.248/Megaco: These protocols are used for controlling media gateways in distributed VoIP architectures. They work in conjunction with MGCP.

• TLS/SSL: Transport Layer Security (TLS) and Secure Sockets Layer (SSL) are encryption protocols used to secure VoIP communication and protect against eavesdropping and tampering.

• SDP: Session Description Protocol, is a text-based protocol used to describe multimedia sessions for the purpose of establishing and managing communication sessions over IP networks. SDP is a key component of various real-time communication protocols and applications, including VoIP (Voice over IP), video conferencing, and multimedia streaming. 

• Codecs: Codecs (Coder-Decoder) are used to compress and decompress audio and video data for efficient transmission over IP networks. Common codecs include G.711, G.729, and Opus, each with its own trade-offs between bandwidth usage and audio quality.

Conclusion

VoIP architecture can vary depending on the specific implementation and requirements of the network, but the core components and protocols mentioned above are fundamental to most VoIP systems. VoIP has revolutionized communication by making it more affordable and versatile, enabling a wide range of applications, from simple voice calls to video conferencing and multimedia sharing.

We will be discussing further on this landscape about Network Components(Part-2) & SIP Trunking(Part-3) in our subsequent blogs. Stay Tuned.

In the realm of mobile applications, we often find ourselves grappling with the need to gather valuable insights into user behaviour and application performance. This is particularly true for products that involve real-time interactions, such as Inbox SDK and Partner App. Before delving deeper for Analytics Manager and discussing the solution. Let’s understand the responsibility of both products first. We will delve deeper into the functionalities and capabilities of these two products, shedding light on their crucial roles.

Understanding the Roles and Responsibilities of Inbox SDK and Partner App

Inbox SDK

The Inbox SDK is a robust platform designed to bolster customer support operations, offering a wide array of features that empower businesses to efficiently manage their customer interactions. Let's take a closer look at its core responsibilities:

1. Overview and Analytics: The Inbox SDK offers a user-friendly interface for detailed ticket insights, covering live visitor count, chat sessions, queue stats, agent availability, and daily summaries.

2. Group-Based Analytics: Inbox SDK enables segmented data analysis, empowering businesses to tailor customer support and engagement strategies for specific groups.

3. Chat Management: Inbox SDK provides vital chat tools: segment-based chat lists (e.g., "My Chat," "Open," "Archive"), access to chat details and contacts, actions like transfer, close, reopen, and easy collaboration management within chats.

4. Public APIs: Inbox SDK goes beyond UI, offering public APIs for profile tasks. Users manage agent status, make updates, and seamlessly engage with and select bots.

Partner App

While the inbox SDK is responsible for managing various chat-related operations, the Partner App integrates the Inbox SDK and extends its capabilities, allowing agents to perform partner-specific tasks directly from their mobile devices. Here are its primary responsibilities:

1. Pre-Login Operations

2. Profile Interface

3. Bot Selection and Agent Management

Problem Statement

However, a common challenge developers face is how to log analytics events within their applications effectively. Analytics event loggers like MixPanel and Firebase require the application to initiate events for logging. In our case, this posed a significant challenge: how do we trigger events within our SDKs and ensure that the right events are logged at the right time?

Solution

The Analytics Manager

At the heart of our approach is the Analytics Manager, a central figure in the orchestration of our analytics framework. This component is responsible for collecting, processing, and dispatching analytics events to the relevant event loggers. Its key features include:

1. Event Collection: The Analytics Manager collects events from various sources, such as our Inbox SDK and the Partner App. This ensures that no matter where the event originates, it is captured and processed in a consistent manner.

2. Standardized Logging: The Manager enforces a standardized logging format for analytics events. This consistency is vital for data quality and the accuracy of subsequent analysis.

3. Event Dispatch: After processing, the Manager dispatches events to the appropriate event loggers, which may include real-time monitoring and custom analytics modules.

Publisher-Subscriber Model

Our solution is not just about addressing current analytics challenges; it's also designed to accommodate future growth and evolving requirements. This is where the publisher-subscriber pattern comes into play. By incorporating this model into our Analytics Manager, we future-proof our analytics framework in the following ways:

1. Dynamic Event Handling: As our software evolves, new events and logging requirements may arise. The publisher-subscriber model allows us to seamlessly integrate these changes. Adding new event types is as simple as introducing a new publisher and subscriber without requiring extensive modifications to the existing infrastructure.

2. Robust Analytics Enhancements: As our software matures, the need for advanced analytics capabilities becomes more apparent. By utilizing the publisher-subscriber pattern, we build a robust foundation for enhancing our analytics modules, enabling real-time analytics, historical data analysis, and customized analytics solutions tailored to our unique needs.

This approach ensures that analytics events are logged consistently and accurately, regardless of whether they originate from the Inbox SDK or the Partner App

.

The YellowAnalytics Class

At the core of our solution is the creation of a stateless class called YellowAnalytics. This class is responsible for initiating and managing all events related to the SDK. By centralizing event management in this class, we ensure a clean and organized approach to analytics tracking.

The trackEvent Function

To enable the Partner App to listen to and respond to events generated by the SDK, we've implemented a trackEvent function within the YellowInbox public class. This function acts as a subscriber to events published by the SDK, making it easy for the Partner App to integrate event logging. Here's how it works:

YellowInbox.trackEvent { event, payload in // Call Event loggers to log the new event }

The trackEvent function accepts a closure that takes two parameters: event and payload. These parameters allow the Partner App to access event information and payload data, which can then be passed to the appropriate event loggers like MixPanel or Firebase for logging.

Benefits of Our Approach

1. Centralized Event Management: By using the YellowAnalytics class, we centralize event management within our SDKs, making it easier to maintain and extend our analytics capabilities. logEvent function of YellowAnalytics class is responsible for logging events through out the SDK

2. Flexibility for Partner App: The trackEvent function provides flexibility to the Partner App, allowing it to choose which events to log and how to handle them. This empowers developers to tailor analytics tracking to their specific needs.

3. Consistency in Logging: With this approach, events are consistently logged, regardless of whether they originate from the Inbox SDK or the Partner App. This ensures that no valuable data is lost, and analytics reports remain accurate.

4. Scalability: When it comes to the scalability of an evolving SDK, this approach shines by effortlessly incorporating new events and logging demands. The beauty of this mechanism is that it necessitates no changes on the subscriber's end. All that's required is for the publisher to introduce the new events to your application. It's this innate adaptability that fortifies the groundwork for future enhancements in analytics, making it an ideal choice for growth and evolution.

Conclusion

Analytics tracking is a vital aspect of mobile application development, enabling data-driven decision-making and user experience improvements. By implementing an Analytics Manager using a publisher and subscriber pattern, we've overcome the challenge of logging events within our Inbox SDK and Partner App effectively. This approach not only centralises event management but also provides flexibility and scalability for future analytics needs, ensuring that we capture and leverage valuable insights to enhance our mobile Inbox products.

Transformative changes have marked Yellow.ai's recent journey in web development. Migration of our cloud app from Monolith to Micro-Frontends with a poly-repo approach, initiated last year, was driven by a dual mission: to reduce build times significantly and to empower individual teams with module-level autonomy.

However, this was not smooth sailing. Code duplication came up as a major problem. Reusing some basic functionalities like utility functions, constants, state management, and UI elements across different Micro-Frontend was threatening to eat up the benefits we desired. The problem led us to search for a remedy, an instrument that would facilitate the process of code sharing, make it efficient, and make the Micro-Frontend(MFE) architecture more unified. That is where Hawk-eye, our custom-built npm package, gets into the game.

This blog post will explain how optimizing code sharing through a custom library called Hawk-eye can improve our Micro-Frontend excellence.

Understanding the MFE Architecture

Evolution in web development has taken place through Micro-frontend architecture as it offers a different path to developing web applications. The basic principle of MFE architecture is to break down large, monolithic applications into self-sufficient components in the same way that microservices work at the backend. Such an innovative approach provides development teams with the independence and adaptability needed to meet the ever-changing needs of web development. Each Micro-Frontend can operate individually and can have a different frontend framework of choice, design standards, and functionality. This process not only simplifies the development procedures but also sets or creates an atmosphere or culture of innovation in which teams may have a choice of the right technologies or tools that are best suited for use with the chosen Micro-Frontend, leading to the ultimate success

This constitutes a fundamental aspect of our strategy at Yellow.ai. We decomposed our cloud application into Micro-Frontends named Channels, the Integration, Engage, and Insights channels with a poly-repo approach (Each MFE will have its repository). These Micro-Frontends are added up into a single container application called web-app, which serves as an entry point to our product suite. They are built in a modular fashion with React as a frontend framework meaning that individual teams can own a separate Micro-Frontend which increases the speed and allows customization based on the requirements.

MFE architecture at Yellow.ai

Code duplication challenges

Code duplication is one of the common issues in MFE architecture and it affects both development and maintenance in the long-term. Nevertheless, this problem becomes even greater when adopting the poly repo approach.

When the utility functions, routing, state management, or the user interface components that make up Micro-Frontends if not carefully planned and implemented results in code duplication. This in turn leads to needless reinventions of already available functions, eventually wasting resources. Lastly, all bug fixes as well as modifications involve a well-coordinated concurrent synchronization of different code bases leading to enhanced maintenance challenges. In a nutshell, this code duplication problem poses doubt on the anticipated benefits of MFEs, such as modularity and independence.

In our Micro-Frontend journey at Yellow.ai, adopting the poly repo approach within the MFE architecture resulted in code duplication of shared utilities, constants, and React hooks across various Micro-Frontends. The consequence of observing multiple versions of the same utilities across MFEs reached a point where maintaining consistency became challenging, leading to confusion among developers. Ultimately it ends in disrupting the planned development workflows. Hence, there should be an optimal way to organize the code sharing across the MFEs.

Code duplication challenges across MFEs at Yellow.ai

Ways to Solve the Challenges

While the goal is to streamline code sharing within Micro-Frontend architecture, it's crucial to maintain a balance between reusing code effectively and preserving the autonomy and agility of MFE teams. Here, we discuss two strategic approaches to address this challenge in detail

1. Share Code as a Library

Packing reusable code elements as libraries is one way to prevent code duplication. These shared functionalities are packaged as standalone libraries that can be distributed and consumed as needed, eliminating the need to duplicate common utilities, constants, or React hooks across several Micro-Frontends. Despite the potential diversity in frontend frameworks or technologies used by different Micro-Frontends, the common libraries are designed to be technology-agnostic. They encapsulate functionalities in a way that is independent of the specific tech stack of each Micro-Frontend. Development teams can easily access shared code without sacrificing their autonomy by incorporating these libraries as dependencies. This approach facilitates efficient and consistent maintenance, ensuring that shared functionalities remain consistent, even if individual Micro-Frontends use different technologies.

Sharing code as a library

2. Share Code as an Individual MFE 

Making specialized Micro-Frontends that are only used to host shared code components is another strategy. The purpose of these Micro-Frontends is to act as a central repository for shared functionalities, which other Micro-Frontends can readily access. It's important to note that these specialized Micro-Frontends are designed to be technology-agnostic, ensuring compatibility with various frontend frameworks or technologies used across the application. This method reduces the amount of duplicate code by storing shared code in a specific Micro-Frontend and allowing it to be utilized across the application.

Sharing code as a MFE

How We Proceeded

In addressing the challenge of code duplication within our MFE architecture at Yellow.ai, we implemented a practice where we grouped reusable code into libraries. This choice was made after a thoughtful evaluation of the benefits and considerations of this approach over sharing code as an individual Micro-Frontend. Now, let’s discuss the particular reasons for our choice.

1. Maintainability and Version Control:

We obtain a powerful way to preserve and version-control the shared functions by wrapping them up as libraries. For a purpose, libraries allow for independent management and updating of the code components, thereby ensuring that the change that occurred in the shared library is also distributed consistently on all Micro-Frontends. This approach simplifies maintenance efforts, making it possible for an organization to follow up the updates in an easy way.

2. Development Workflow Integration:

Custom libraries seamlessly integrate with standard development workflows, enhancing the overall development experience. Including custom libraries in the development process makes it easier to update the shared functionalities.

3. Testing and Validation: 

Custom libraries provide a convenient environment for testing and validation of shared functionalities. Development teams can test library components independently, ensuring they function as expected. This can streamline the testing process, leading to more robust and reliable shared code.

Unveiling Hawk-eye: A Tale of Code Harmony

Hawk-eye is the custom library that is crafted as Yellow.ai's utility companion for frontend apps, to combat issues with code duplication among Micro-Frontends. At its core, this "Util Package" is backed with a central repository designed to house common utility functions, constants, and hooks.

Key Features of Hawk-eye:

1. Unified Codebase Management: Orchestrate a unified codebase by consolidating utility functions, constants, and hooks. This ensures a streamlined development process and consistency across Micro-Frontends.

2. Effortless Code Reusability: Develop a culture of effective code reuse, cut down on redundancy, and create a centralized location for reusable code components to simplify development.

3. Efficient Maintenance Hub: This "Util Package" serves as a central hub, thereby simplifying the versioning and updates. This ensures that common functionalities evolve uniformly across Micro-Frontends, reducing the risk of inconsistencies.

Implementation

Hawk-eye 1.0 (A Milestone in Code Sharing Excellence)

Code collaboration was revolutionized by the first version of Hawk-eye 1.0. As Yellow.ai’s point of light, it has dedicated a host for JS utility, constants, and React custom hooks, promoting code sharing across MFEs.

However, Hawk-eye 1.0 was more than just an introduction – it promised that development teams would be equipped using unified and simplified practices. We quickly adopted Hawk-eye 1.0 as our utils of choice in Yellow across all MFEs of Yellow.ai.

Hawk-eye 1.0 Overview

Challenges and Evolution: A Transformative Journey

Nevertheless, Hawk-eye 1.0 had some limitations, particularly with React-specific peer dependencies that limited its adaptability for non-React frontend projects. This was the challenge that prompted us to seek a more flexible solution, beyond the domain of frameworks.

Hawk-eye 1.0 Challenges

It should be noted, however, that this was the turning point in the way we approached Hawk-eye. The pursuit for inclusion sparked an architectural shift in Hawk-eye, Which is dividing the package into two separate bundles backed with a mono-repo. This marked an important milestone in Hawk-eye’s journey towards greater adaptability and acceptance.

Hawk-eye 2.0: (Elevating Code Collaboration Mastery)

Hawk-eye 2.0 represents a deliberate step forward in our journey toward mastering effective code collaboration, that builds upon the foundation set by Hawk-eye 1.0. It's not just an update, either. Rather, it is a strategic evolution that addresses the issues raised by Hawk-eye 1.0, embracing a collaboration model that improves our shared coding while also aiming for increased modularity and adaptability.

Key Enhancements: 

1. Distinct Packages for Precise Solutions

Hawk-eye 2.0 divides itself into two distinct packages.

• "Hawk-eye-core": A powerhouse for JavaScript utilities, ensuring a robust foundation for common functionalities.

• "Hawk-eye-react": Dedicated to React-specific custom hooks and utilities

This enforces a better separation of concerns, enhancing modularity and ensuring a more tailored approach to code sharing. The result is that the packages are versatile enough to use in both react and non-react frontend applications.

Distinct Hawk-eye Packages

2. Monorepo adoption

A crucial decision in Hawk-eye 2.0 is the adoption of a mono-repo structure over poly-repo. Monorepo helps in simplification of the development process, code sharing between packages, and collaborative development for packages. This selection allows smooth code sharing between “hawk-eye-core” and “hawk-eye-react”, creating a unified environment.

Monorepo with Lerna:

There are several monorepo tools available in the landscape including Nx, Lerna, Turborep, etc., which have distinct merits. However, we went with Lerna because of the following reasons

• Tailored task pipeline with Nx:
  A tailored task pipeline that aligned Lerna with Nx was effective in running the scripts in a controlled manner. This is even more true in the case of Hawk-Eye where the build order matters meaning hawk-eye-core has to run before hawk-eye-react because core is a dependency in the react project. This intricate control ensures that our development workflow is productive and predictable.

• Flexibility in Dependency Management and Structure:
  Using Lerna’s flexibility, we created a unified dependency management approach. Consistency in dependency versions is ensured by the global sharing of common packages across React and Core whenever needed. This flexibility allows teams to customize the monorepo of Hawk-Eye based on its specific project needs.

• Tailored Deployment Solutions:
  Lerna's adaptability extends to tailored deployment solutions like dependent/ independent versioning of packages. For Hawk-Eye 2.0, this means the freedom to implement custom deployment strategies. Currently, we're utilizing independent versioning for packages, ensuring precise control over version management.

Moreover, Lerna was also among the most popular and commonly used package managers for building as well as publishing multiple JS and TS packages from one repository. Collectively, this brings Lerna best suited with the Hawk-eye monorepo architecture.

Nexus Repository: Streamlining Package Management

In both versions of Hawk-eye, the Nexus Repository acts as a centralized powerhouse for efficient package management. Serving as a unified hub, Nexus ensures streamlined storage and retrieval, offering developers a centralized location for package management.

Hawk-eye 2.0 Overview

Learnings from the Hawk-eye Journey

Setting out on the Hawk-eye path to Micro-Frontend excellence reveals a wealth of new information and important lessons. A few important conclusions come to light as we examine the nuances of package management, code sharing, and the transition from Hawk-eye 1.0 to 2.0.

1. Strategic Code Sharing:
   Discover ways of sharing code in MFE architecture and understand the significance of code sharing. The way Hawk-eye's packaging and modular methods changed how we handle shared components in Micro-Frontends, making things simpler and better.

2. Monorepo Mastery with Lerna:
   Find the advantages of using a mono-repo setup backed by Lerna. Explore how this strategy enhances package management and version consistency and promotes modularity and collaborations.

3. Centralized Package Management with Nexus:
   Discover the ways to use Nexus as a centralized repository. See how it enhances the package discoverability, version control, and the singular strategy behind storage and retrieval.

4. Versatility Beyond React:
   Explore the versatility of Hawk-eye in catering not only to React applications but also seamlessly integrating with non-React projects. Witness how this flexibility opens doors to a wider spectrum of development scenarios.

5. Adapting to Challenges:
   Understand the challenges encountered in Hawk-eye 1.0 and the improvements made in Hawk-eye 2.0. Going beyond React-specific dependencies, the shift to a more inclusive architecture provides key lessons in tailoring solutions for changing requirements.

Final Notes

Hawk-eye is a game-changing solution for Yellow.ai’s Micro-Frontend architecture, reducing issues such as code duplication and improving development process efficiency. Our journey, which began with Hawk-eye 1.0 and continued with the upgraded Hawk-eye 2.0, is a dedication to improving Micro-Frontend development excellence.

The adoption of a mono-repo structure, orchestrated by Lerna and Nexus, not only streamlines package management and ensures version consistency but also serves as a beacon for collaborative workflows. Importantly, the modular separation of packages within Hawk-eye enables seamless consumption in both React and non-React applications. Development teams are empowered by this versatility, which provides a unified and flexible solution for Micro-Frontend excellence across various projects and frameworks. Hawk-eye is a testament to innovation, enabling teams to navigate the intricacies of modern web development with confidence and agility. Beyond just a case study; it's a source of inspiration, encouraging a mindset of continuous learning and refinement in Micro-Frontend development practices.

References

https://levelup.gitconnected.com/micro-frontends-what-why-and-how-bf61f1f0a729

Micro Frontends: A Deep Dive into Pros and Cons

The Dilemma of Code Reuse in Microservices

Monorepo Explained

You too can love the MonoRepo

Guide to Monorepos for Front-end Code | Toptal®

Introduction | Lerna

In the world of technology and infrastructure management, ensuring robustness and resilience is paramount. One crucial aspect of achieving this is by eliminating Single Points of Failure (SPOFs). These vulnerabilities can disrupt operations, lead to downtime, and pose significant risks to our business. This blog provides an insider's perspective on how we meticulously identified and eradicated SPOFs, sharing the strategies, insights, and technologies that have strengthened our platform's resilience.

This blog is the second part of the Multi-Region series from Yellow.ai. You can read the first part here

Understanding Primary and Secondary Regions

Our primary region functioned as the core metadata hub and was located in India. It included the authentication system with access to essential MySQL databases containing chatbot metadata, user account information, and user roles. The primary region formed the backbone of our operations and was central to our multi-region deployment strategy. Secondary regions complemented the primary region by extending the application's reach to other regions. The secondary regions were geographically dispersed to cater to users outside the primary region. They relied on the primary region for access to the metadata and proxied most of their calls to the primary. 

In the secondary region, not every service proxied requests to the primary region or relied on data from the metadata database. This was limited to a select few core services, with the authentication service being a prominent example. The authentication service worked in a primary-secondary model, where it primarily handled requests by proxying them to the primary region. Additionally, it autonomously processed certain requests before seeking assistance from the primary.

One such request was user sessions for our platform. When a user authenticated with the primary system, it utilized the metadata database to compute the session object and cached it in Redis. Without this cache, the user was deemed unauthorized. Similarly, when the secondary system attempted to authenticate the user, it initially checked the cache. However, it did not flag the user as unauthorized in the absence of the cache. Instead, it requested session data from the primary system. Depending on whether the primary considered the user authenticated or not, it responded accordingly to the user. Additionally, if the user was deemed authenticated by the primary system, the secondary cached the session for a brief duration. Whenever the primary server purged the session in its Redis instance, it issued commands to the secondary servers to invalidate their caches as well. This gave a clear distinction between regions as primary and secondary.

A New Foe: Single Points Of Failure

Our first action item post enabling multiple regions was to increase the resilience of the system. We audited every component in the system to find and eliminate single points of failure. Since we had primary and secondary regions, we had to ensure that despite the primary systems going down, none of the secondary systems were affected. We wanted to minimize the blast radius due to any unforeseen problems we could encounter in the primary region. Imagine explaining to a customer in the USA that they cannot use the platform because a remote server in India is temporarily unavailable. Our motivation to eliminate the single point of failure boiled down to building redundancy and resilience across regions.

Seamless Failovers

Given that the microservices themselves maintain a stateless nature, the key approach was to ensure availability of the shared metadata in more than one region. In the event of an outage in the primary region, the end users would need a seamless switch to a different region that could handle the authentication requests.

The existing authentication sessions were cached in Redis and new sessions were computed from a MySQL metadata database. If the other region did not access the same data, it would deem the call unauthorized and the users would get logged out in the event of a failover. In essence, two microservices in different regions, both running the same image and connected to identical data sources, were expected to exhibit consistent behavior. With a clear problem statement in place, the subsequent logical step involved breaking it down into smaller concerns to be addressed at the network level, the database level, and the application level. For this, we selected a second region capable of acting as an alternative primary region.

Configuring Cloudflare LoadBalancer

When the other regions tried to communicate with the primary region, they used the generic domain “https://cloud.yellow.ai” instead of a region-specific domain such as “https://z1.cloud.yellow.ai”. This allowed us to load balance the requests to any of the primary regions depending on their health. Cloudflare allows the setting up of service health monitoring so we could extend the same health check endpoints used by Kubernetes to Cloudflare and ensure seamless failover. In the event of an untoward service failure, we opted for a simple failover switch since we wanted to avoid introducing a lot of changes to the system at the same time. We ensured that the Load Balancing rules were extensible enough to onboard additional regions as primary.

One side benefit of the setup is that it allowed us to experiment with different Load Balancing strategies such as geo-steering or round-robin so we could determine the best configuration for the lowest latencies in all our regions.

Region-to-Region Database Connectivity

Once the decision was made to replicate the database between regions, we needed to provide network connectivity. This was straightforward as we had always ensured the CIDR blocks did not overlap between regions. The database subnets were connected with readily available connectivity options provided by our cloud providers that allowed them to communicate with each other, and by extension, replicate the data.

Extending the SQL Database

The metadata database existed in MySQL, a popular relational database. When we wanted to replicate this database, we considered other relational databases focusing mainly on High Availability and Fault Tolerance. We evaluated MySQL Group Replication, Amazon RDS, and Amazon Aurora. We ruled out RDS immediately because it was not proving to be cost-effective. We found group replication to be a great mechanism and a good fit for us as it worked with the existing setup, but it was a close call. What pushed MySQL GR over the edge was the fact that Aurora could not withstand region-level failures, and was limited to withstanding the loss of Availability Zones within a region. However, Amazon Aurora is a great distributed database and we would consider that for future use cases.

The main objective of MySQL replication was to make the failovers between regions seamless. This meant the data replication had to be synchronous, and hence, we set the consistency level of the group replication to “AFTER”. From the group replication docs,

you have a group that has predominantly read-only (RO) data, you want your read-write (RW) transactions to be applied everywhere once they commit, so that subsequent reads are done on up-to-date data that includes your latest writes and you do not pay the synchronization on every RO transaction, but only on RW ones. In this case, you should choose AFTER.

Since the system is read-heavy, we could afford the updates to become a bit slower as a trade-off for high availability. It did not change the write latency to anything noticeable, so we got the validation that we had made the right decision.

ProxySQL

Next up, we wanted the applications to be aware of any topology changes that could happen in MySQL. We evaluated ProxySQL, MySQL Router, and MariaDB MaxScale. Based on popular benchmarks, MySQL Router’s performance was significantly lower than any other proxy tools like MariaDB MaxScale and ProxySQL. This was a deal breaker as we wanted to build a future-proof system. It became clear that the tool required improvements with respect to usability as well as performance. We eliminated MariaDB MaxScale because it needed a commercial license.

Finally, the community around ProxySQL and the open-source license sealed the decision. ProxySQL ensured the traffic always went to the primary and switched the traffic in the event of any topology change. This meant every application could connect to ProxySQL which would then multiplex the connections to MySQL. It supports read/write split as well should the need arise. Thank you, Rene Cannao.

If you are interested in reading more about proxies, this is a great video that covers most of the relevant MySQL proxies.

Redis Sentinel

With MySQL replicating out of the way, the next step was to make the same Redis cache available in the alternative primary region. A simple Redis Sentinel setup was adequate for sharing the authentication data across regions and providing failovers. The sentinels could run on the primary regions as Kubernetes Deployments. They could then communicate with the Redis servers in two different regions and form the shared Cache Layer. The microservices used ioredis, a Redis client capable of understanding sentinels and reconnecting to the Redis server. In case the Sentinels switched the traffic, the use of this library ensured there were minimal disruptions to the end users.

Tweaking the Application Layer

Each service consuming the metadata database was stateless because the state persisted with the help of Redis and MySQL. This meant any of the regions could become the primary region as long as it had access to the same state. Depending on the configuration loaded, the application could decide whether to act as primary. As a part of the initial setup to enable multiple regions, we also added a proxy middleware in our application to automatically intercept the calls in secondary microservices and send them to the primary service.  The proxy middleware had support for sending certain API calls to the local microservice as well, achieving the best of both worlds.

In this activity, we introduced support for Sentinel Redis and combed through the applications reading from the metadata database to remove any logic that was written with a single primary region intent. This single primary region programming pattern would cause untoward failures in the event of a failover. Once these patterns were refactored to ensure consistent behavior during failover, we subjected the application to rigorous testing.

The Promised Land of No SPOF

We did it. We have dismantled SPOFs from our infrastructure, layer by layer. The result is a resilient system that is capable of weathering unforeseen challenges and ensuring uninterrupted service to our users across the globe. As always, we remain cautious about any points of failure we have not realized yet and continue to be on the lookout for them.

Final Notes

At Yellow.ai, we firmly believe that resilience is not just about withstanding failures but also about thriving in the face of change and uncertainty. Our global commitment to delivering seamless conversational AI experiences drives us to continuously innovate and adapt. As we look to the future, we remain steadfast in our dedication to fortifying our infrastructure, ensuring uninterrupted service, and embracing the opportunities presented by a multi-region, multi-cloud platform.

In the ever-evolving landscape of conversational AI, Yellow.ai has emerged as a global leader, pioneering autonomous, human-like experiences to foster meaningful brand engagement. To further enhance our capabilities, we embarked on a journey towards enabling our platform in multiple regions. This blog delves into the motivations behind this strategic move, our multi-region architecture, the challenges we encountered, what goes into developing software in a multi-region multi-cloud platform, and how this impacts our business.

The Motivation Behind Multi-Region Deployments

We are committed to delivering exceptional conversational AI experiences worldwide. Multi-region deployment was a strategic imperative driven by several key factors:

• Global Reach: To cater to a global customer base, we needed infrastructure closer to end-users, reducing latency, and improving response times

• Scalability: Multi-region deployment allows us to efficiently scale our infrastructure to meet the growing demands of our global user base, ensuring that we can accommodate the individual needs of each region.

• Data Privacy and Compliance: Compliance with diverse data privacy regulations necessitated data localization in specific regions

The Pledge: Preparing For Multi-Region

Pre Multi-Region Days

Initially, the Yellow.ai platform was constructed primarily for a single region. As a result, all the microservices were structured around this mindset. Let’s take a look at the architecture of the system running in a single region. 

The end users can communicate with us through the platform URLs or external channel sources. The traffic arrives at a public Load Balancer and then pours into our Load Balancer - HAProxy. HAProxy helps to route the traffic to different backend systems like Ejabberd (XMPP) and Kubernetes. From HAProxy, the traffic flows to Nginx, the entry point of the managed Kubernetes cluster. Nginx plays a pivotal role in routing requests to various Kubernetes-registered services, where each service is configured as a route within Nginx following the pattern "/api/service-name". These services connect to the databases running in a different subnet. The bots run in their own namespace and are segregated by their type. They can communicate with the customer’s systems through an egress controller that has a set of static IPs. This is particularly useful when customers want to allowlist IP addresses that can access their systems.

Replication of this architecture and its transformation into a multi-region compatible system posed its own set of challenges. The need to be cloud-agnostic was considered as popular cloud providers were not available in every region. This need also helped us avoid any vendor lock-ins. Even if a popular cloud provider existed in a region, the following points had to be taken into consideration.

• Complexity Management: Handling increased complexity in network configurations, data synchronization, and monitoring

• Storage Mechanisms: Using a cloud-agnostic object-storage interface

• Cost Optimization: Selecting the ideal zone in a region with reliability and cost-effectiveness

• External Channel Webhooks: Handling messaging channels that do not support conditional webhooks for region-based routing of messages to our platform

Picking New Cloud Regions

We had to pick new cloud regions and ensure they were the right place to serve our target markets. Based on this, we had to evaluate the different regions on their uptime, cost, service availability, proximity to customers, and cloud provider support. In India, we already had our services set up in Mumbai. After carefully evaluating the different options, we decided to go with the following regions as they aligned as closely as possible to the metrics.

The Turn: Adopting Cloud-Agnostic Multi-Region Strategies

However, in our case, we were transforming an already extraordinary platform into a new beast altogether. There were several architectural changes required before we could see different regions seamlessly communicate among themselves and with the end users. Let’s go through how we solved these challenges.

The Basics: Cluster Replication

We had to replicate the Kubernetes cluster over to the new regions which included creating the cluster, spinning up virtual machines for the databases, adding nodes to the cluster, and configuring subnets. You know how it goes. Once the whole cluster was ready, we moved on to the next problem.

Cluster Reachability

Up next, we ensured that the regions were reachable through the Internet. The platform’s front end runs in the primary cluster. Depending on the chatbot selected, the front end would call region-specific URLs that were resolved to a Cloudflare LoadBalancer. Based on the host, Cloudflare would forward the request to the appropriate public LoadBalancer. This solution could be extended to any number of clusters without a problem.

Since Cloudflare offered Multi-Domain Certificates, it made it easier for us to use the same certificate for all our domains. We added all our domains as Subject Alternative Names to the same certificate and reduced the time required to configure certificates for all the regions. For the enthusiasts out there, you can examine the certificate we load on our platform to see this in action.

Each of the regions was assigned a unique region code. Appending these codes to the base domain results in the calls reaching the region-specific Kubernetes cluster. For example, the URL https://z0.cloud.yellow.ai reaches the India cluster while the URL https://z4.cloud.yellow.ai reaches the USA cluster.

• z0 - India

• z1 - Bahrain

• z2 - Jakarta

• z3 - Singapore

• z4 - USA

• z5 - Europe

Extend the Authentication System

The decision to extend our authentication system stemmed from a fundamental challenge: the primary system’s access to a critical MySQL database containing essential chatbot metadata such as names and regions, the user account data, and the user roles. This metadata was indispensable for deploying the Yellow.ai platform globally. While replicating the entire database across regions wasn't an immediate option, we found a viable solution by extending our application's capabilities.

To put it into context, here's the issue: the primary authentication system in India possessed the sole access to the MySQL database housing the bot metadata. This data forms the bedrock of our bot deployment process worldwide, enabling us to load bots across the globe seamlessly. So, let's dive into the challenges and the ingenious solution that allowed us to seamlessly make our platform a global system.

We added support in the authentication system to become region-aware and categorized the system into primary and secondary. The primary would handle all incoming requests by itself and the secondary system would proxy incoming requests to the primary. To keep it simple, if the system was running in India, it operated as the primary, and if not, it operated as the secondary. Initially, this approach worked well. However, we soon realized that the latency of communicating from the West Coast of the United States to India was significantly slower than we expected. To address this, we introduced an allowlist that allowed the secondaries to handle specific APIs with a simple framework.

The logic behind this framework in the secondaries was straightforward:

• Query Local Cache Layer for Authentication Data

• Return if Cache is present

• Else, query primary for the data, cache, and respond

With this logic in place, we needed a robust cache invalidation mechanism to ensure that whenever data in the metadata database was updated, regions not directly connected to this database would invalidate their cache. To achieve this, whenever the primary system updated the metadata, the secondaries received a call from the primary to delete their cache. To prevent issues related to stale cache in case of any invalidation failure, we populated the cache in the secondaries with a short TTL.

Data Compliance

We had to be careful with data compliance because of strict regulations, as well as the belief system that made us do everything to protect our customers' data. We built the data flow in a way that the metadata about the customers goes to the India cluster, and the actual data of the customers resides in the databases in the region where their bot is configured to run. We ensured that no data, even accidentally, flowed outside the customer's region.

Callback Proxy for External Channels

In our journey towards multi-region deployment, we faced a challenge: the immutability of URLs for external channels. We had to work around the limitation of some external channels that could be configured to send data to only a single URL. To address this, we introduced the Callback Proxy. This solution utilizes a proxy server that acts as an intermediary between external channels and our multi-region architecture. When data arrives, the callback proxy's routing logic consults a metadata server to determine the appropriate destination, allowing seamless communication without altering existing channel configurations.

Cloud-Agnostic Object Storage

Next up, we confronted the task of integrating disparate storage systems inherent to each cloud provider. These unique storage offerings and protocols posed a significant obstacle to seamless interoperability. Our solution emerged in the form of MinIO, an open-source object storage server. MinIO adeptly bridged these divides, allowing us to harmonize our storage infrastructure, ensuring compatibility and data portability across multiple cloud environments. Since MinIO offers support for the s3 protocol out of the box, the applications do not have to worry about where the data is coming from, and they connect to the server in every deployment in the same way. This strategic adoption empowered us to harness the strengths of each cloud provider while preserving our vendor-agnostic approach.

Content Delivery Network ( CDN ) and Caching

We went ahead with Cloudflare to cache most of the frontend assets. This ensured the fast loading of assets despite the user’s location. We also tweaked the build pipelines to purge this cache whenever we built a new version of our frontend app or the chat widget.

The Prestige: Multi-Region Goes Live

With these points being taken care of and through rigorous manual and automated testing, we released this multi-region capability to production. We patted ourselves on the back and rejoiced at this occasion.

However, we understood the importance of not resting on our laurels. This marked the initial phase, and as we know, challenges evolve.

Epilogue: Life with a Multi-Region Multi-Cloud Platform

Navigating life with such a complex platform requires vigilance and several considerations in the way we program the application, the way we roll out new changes and the way we scale it.

Programming Patterns

When working with microservices that interact with the metadata database, we have to be careful to not work with a single-region mindset. We should factor in the system design, the different regions available, and the fact that the same code can run in either a primary or secondary region, ergo, the same code can either have access to the metadata database or not.

Deployment Strategies

The deployment strategy should be decided during the time of initial system design to avoid hiccups during the rollout. We can achieve selective rollout for each region through feature gating, a glorified IF condition at the heart of the code. Essentially, the logic boils down to something like

IF ( isGateEnabled ) { 
    executeNewCode();
} ELSE {
    continueWithOldCode();
}

While feature gating is a valuable technique, we recognize the need for specialized strategies. Applying techniques like gating may not be universally suitable, particularly when dealing with database schema changes. To proactively address this, we emphasize the importance of considering rollout strategies during the initial design phase. This foresight helps mitigate deployment challenges. Additionally, we strongly discourage the use of different images for the same microservice across regions. This uniformity enables us to swiftly address any region-specific code incompatibilities and execute temporary rollbacks when necessary.

Scaling and Cost Management

Thanks to our amazing Infrastructure Engineering team, we’re able to understand the optimal configuration for each microservice running in each region with the help of instrumentation and the limits assigned to the Kubernetes workloads. With this, we adjust the workloads to the appropriate resource limits. HPA is set for each microservice by analyzing past traffic, and load testing the application to determine appropriate thresholds. This setup gives us the flexibility to individually scale each region depending on the anticipated traffic or gradual increase in consumption. 

Final Notes

Yellow.ai's journey into multi-region deployment represents a pivotal step in our commitment to delivering exceptional conversational AI experiences globally. By understanding the challenges, and implementing a resilient architecture, we have fortified our position as a leader in the industry. This strategic move not only benefits our customers but also sets the stage for continued innovation and growth in the world of conversational AI.

This blog is the first part of a two-part series on multi-region deployments from Yellow.ai. In our next article, we'll delve into a critical issue: Eliminating Single Points of Failure.

In today's digital age, the performance and reliability of web applications play a pivotal role in ensuring a seamless user experience. As developers and operations teams strive to optimize and monitor their applications, there's a growing emphasis on understanding not just the backend mechanics, but also how these applications perform from the user's perspective. Enter the realm of frontend API instrumentation—a powerful approach to gauging the real-world performance and behavior of APIs as experienced by the end-users.

While backend monitoring gives us insights into server health, database performance, and internal service interactions, it's only one piece of the puzzle. To truly grasp the user experience, we need to dive into frontend instrumentation, which captures metrics from the client's vantage point. From measuring actual response times seen at the client end to tracking the errors users encounter, frontend API instrumentation provides invaluable data that can shape our optimization strategies.

In this blog, we'll explore the nuances of frontend API instrumentation, its significance, and how tools like Prometheus and Grafana can be leveraged to visualize these metrics. We will do a technical deep dive of our frontend API monitoring architecture. We'll also discuss the differences between frontend and backend instrumentation, shedding light on why both are essential for a holistic understanding of application performance.

Some Background

We have a module called Inbox which is used by live agents to respond to end customers. Systems serving live agents generally have a very low tolerance for service disruption, as they are serving live customers.

Before we implemented our frontend API telemetry solution, when our customers used to report an error, the only data point we had was from our backend monitoring system. The great thing about the backend monitoring system is that it can tell you, in granular detail, what is going on inside our backend services. But as soon as the packet leaves our infrastructure, we lose control of any metrics on how it arrives at the end customer's system.

Hence, while it might all seem green from our backend monitoring system, unwanted network latency added by our Proxy, Cloud Service Provider, or our customer's ISP remains in our blind spot. In light of that, our team started on a solution to increase our visibility around API monitoring.

Technical problems aside, not having the API telemetry coverage also caused a lot of operational hazards. A lot of time is wasted in triaging these issues reported by our customers,

• A lot of time used to get wasted in triage

• Customer dissatisfaction

• Wasted product development bandwidth and focus looking at the wrong areas

Significance of Frontend API Instrumentation

In the intricate web of application development and monitoring, frontend API instrumentation emerges as a beacon of clarity. It's the lens through which developers can view the real-world performance and behavior of their applications from the user's perspective. While backend monitoring offers insights into server operations, it often misses the nuances of user experience, network latency, and client-side processing. Frontend instrumentation fills this gap.

By focusing on the frontend, we can capture metrics that truly matter to the end-users. For instance, even if a server processes a request swiftly, the user might still experience delays due to network issues or client-side rendering. Only through frontend instrumentation can such discrepancies be identified and addressed.

Frontend API telemetry can help us with the following:

1. User-Centric Monitoring: Frontend instrumentation offers insights directly related to the user experience, ensuring that applications meet user expectations.

2. Identifying Discrepancies: It helps in pinpointing differences between server-side performance and actual user-perceived performance.

3. Filling the Blind Spots: With the increasing complexity of web applications, frontend metrics ensure no performance issue goes unnoticed.

4. Network Latency Insights: Captures the impact of network conditions on application performance, which backend metrics might overlook.

5. Holistic Understanding: When combined with backend metrics, frontend instrumentation provides a comprehensive view of application health and areas for optimization.

The architecture of the system

Overview

We have a web application where we allow our users to log in and access different modules of the platform. The frontend telemetry SDK is integrated with our web application. The SDK collects the metric from our web application and reports it to a custom backend metric aggregator. This metric aggregator then exposes a metric API which is scraped by a Prometheus instance. We use Grafana to query the data collected by Prometheus. The Grafana stack is also used to set up alerts on the API metrics, and the alerts are then delivered to Slack.

Frontend SDK

The actual frontend SDK is bundled as a Web Worker. The main web application reports the data to the Web Worker using PostMessage. The SDK is running a custom-built Prometheus client that converts the received event via postMessage to predefined Prometheus metrics. The worker stores the collected metrics in memory and sends them to a backend metric aggregator every 15 seconds. Once the data is relayed to the backend service, the in-memory metric data is flushed out.
The main web application interacts with the Web Worker started by the SDK. The underlying framework uses the popular JavaScript library Axios to make API calls to our backend system. One cool thing about Axios is that it allows us to add request and response interceptors.
We added one request interceptor to extract certain metadata from the request header and added the API call start time metadata to the request config. We also added a response interceptor to actually track the latency using the request metadata, and response status code, in case the request failed, collect the error code, etc. This metadata about the request is then posted to a Web Worker.

Metric Aggregator

The backend metric collector is a simple NodeJS application that receives the metrics data from frontend. The frontend SDK is generic and sends telemetry data for all APIs that are called using Axios. The backend filters the reported metric based on different custom filters. After filtering the metrics, the backend stores the metric in memory. The backend service also aggregates the metric reported from the frontend to avoid wrong data being stored in Prometheus. More about the aggregation will be discussed later in the article. The backend service also exposes an endpoint for Prometheus to scrape. Prometheus scrapes the metric endpoint every 30s. Once Prometheus scrapes the endpoint, the data in memory is flushed.

Collected Metrics

Histograms are used for all latency-related metrics such as API response time, WebSocket ping latency, and counters for tracking error counts, request counts, etc.
Primarily we collect the following metrics

• API Response time: This is one of the key metrics we track to keep track of our end customer experience for our most critical APIs. The metric adds project ID, and API path labels to allow granular data inspection.

• Error count: We also track the error counts as a Prometheus counter metric. This allows us to understand what kind of errors our end users are facing. We also have alerts set up for this metric. The error count metric adds project ID, API path, status code, and error type labels to allow detailed investigation.

• WebSocket ping latency: As our web app allows real-time communication, we also keep a tab on latency for our real-time messaging servers. This metric is generally a good indicator of any network bottleneck. It also allows us to get a rough understanding of our users' baseline network latency.

• API call throughput: This is actually a derived metric from our API response time metric, which also pushes the API call count. This metric allows us to understand the load pattern originating from our users' browsers.

Dashboards and Alerting

We use the Grafana stack for creating beautiful dashboards to present the collected metric data. The dashboards allow us to filter the data by region, Project ID, path, etc.

We also use the Alerting stack provided by Grafana to set alerts for our dashboard panels. The alerts are then channeled to respective Slack channels for our engineers to take corrective actions.

Challenges

performance of the web application

• SDK resource footprint: One of the most important aspects to consider before building the frontend SDK was keeping the resource footprint (CPU and Memory) of the SDK as low as possible. As the SDK runs in a separate Web Worker inherently the execution is isolated for efficient processing

• User Experience: Ensure no disruption to the user's experience with the integration of the new SDK.

• Data Reporting Frequency: Reports data to the backend every 15 seconds. This is decided based on heuristics keeping in mind that the interval at which the data gets reported is not too large to miss out on data points and, at the same time not too short to create noise. In our experiment, 15 seconds seemed to be right in the sweet spot.

• Memory Management: Data collected for metrics is flushed out after each report, allowing GC to clear up the unused memory.

Adopting Prometheus

Prometheus stands out as one of the most valuable databases for storing telemetry data. It's not only well-accepted but also deeply understood by the SRE and developer community. While we initially toyed with the idea of using other time series databases like TimescaleDB or ClickHouse, our choice ultimately settled on Prometheus. This decision was driven by its ease of use and its alignment with the type of metrics we aimed to track.

However, implementing Prometheus was not as straightforward as it might seem. Those familiar with Prometheus might already sense the challenges we faced. Collecting telemetry data for thousands of users from their browsers running our custom Prometheus client is similar to collecting telemetry data from thousands of ephemeral servers. At its core, Prometheus operates on a pull-based system. This means that instead of pushing data to Prometheus, it's Prometheus that pulls data from its sources. Given the nature of browsers, it's impractical to scrape data directly from clients running in them. This is where our metric aggregator steps in.

Browsers are dynamic environments. Users can close tabs, open new ones, or refresh pages at any given moment. Consequently, every time our browser SDK reports data, it starts from scratch, which is really problematic because it can skew the reported data. To address this, our metric aggregator backend plays a crucial role. It keeps track of the data reported by the browser client and, rather than starting afresh, aggregates this new data (for the same set of labels) with previously reported metrics, storing the combined data in memory.

Our Learnings

Backend metrics do not tell the full story

You only get to know half the story from your backend observability stack. While this can tell you more about what is going on inside the server and what is causing the issue, it cannot possibly tell the impact on user experience caused by a degraded network. The API metrics collected at the front end have allowed us to close the loop, giving us a full spectrum of data to make an informed decision.

Isolating the problem

With API telemetry data being available from both frontend and backend segregated by project ID and region, we can quickly isolate the problem whether it is caused by our backend or it is an isolated incident only disrupting a certain fraction of our users.

Fast support turnaround time

With the full spectrum of data being available, our on-call engineers and support team can now look at the frontend telemetry data and work with our customers for a faster resolution to the problem. Previously, a handful amount of time was being spent looking through the various APM and infrastructure dashboards to identify the root cause of the issue.

Designing UX keeping latency in mind

With this system in place, the team has access to latency data observed by our end customers, which allows the entire product team to craft a solution keeping end-user experience in mind.

Way Forward

Our vision for the future of frontend telemetry is not just about collecting data, but about harnessing that data to create tangible, positive impacts on user experience.

Imagine a world where our applications are so attuned to their environment that they can provide visual cues to users the moment network degradation is detected, ensuring transparency and trust. We're committed to refining our frontend SDK, pushing the boundaries of performance to ensure that our telemetry tools are not just observers but active enhancers of the user experience.

Furthermore, by adopting OpenTelemetry standards, we're looking to elevate our telemetry practices. This move promises better integration, richer visibility, and a seamless bridge between our frontend metrics and the broader ecosystem of application performance monitoring. But we won't stop there. Our ambition extends to diving deeper into the intricacies of frontend performance, capturing a plethora of metrics that will offer us a crystal-clear understanding of the end-user experience.

The path ahead is filled with challenges, innovations, and opportunities. But with a clear vision and unwavering commitment, we're poised to redefine the standards of frontend telemetry, ensuring that every user interaction is not just monitored but optimized, celebrated, and enhanced.

Having users from different regions and languages and supporting their languages is crucial for the success of mobile applications in today’s world. However, when using app bundles, an optimization process can cause runtime app localization to break. In this article, we will explore the problem in detail and provide a solution.

Understanding the Problem

App bundles are the publishing format for Google Play, it contains the compiled code, resources, and assets of an application. When a user downloads apk from Google Play, the store delivers the optimized apk for that particular device according to the configuration of that device.

However, there is an issue with runtime app localization and app bundles. When using app bundles, the optimized apk does not include unnecessary resources and assets that are not needed for the user’s specific device, resulting in a smaller download size. The problem arises when runtime app localization relies on dynamically loading localized resources or assets that are not included in the optimized APK. Since these resources are not present in the APK, the app may not be able to retrieve and display the appropriate localized content at runtime, leading to errors or missing translations.

Solution -1

One possible solution to this problem is to specify in the build.gradle file that all resources should be included in the app bundle

android {
    // ...
    bundle {
        language {
          enableSplit = false
        }
    }
    // ...
}

This Specifies that the app bundle should not support configuration APKs for language resources. These resources are instead packaged with each base and dynamic feature APK. This ensures that all required localized resources are present in the optimized APK file generated by the app bundle.

This Solution will result in an increase in the size of the bundle. To fix this we have another optimized solution which is downloading resources on demand using PlayCore API

Solution -2

In this solution, we are using SplitInstallManager class. Whenever the user changes the language, we need to check whether the selected language is installed or not and on basis of it we will set language of the app.

private fun changeLanguage(selectedLangs: String) {
    val splitInstallManager = SplitInstallManagerFactory.create(context)
    val langs: Set<String> = splitInstallManager.installedLanguages
    if (langs.contains(selectedLangs)) {
        //TODO: Change Language Here
    } else {
        val installRequestBuilder = SplitInstallRequest.newBuilder()
        installRequestBuilder.addLanguage(Locale.forLanguageTag(languageSelected))
        splitInstallManager.startInstall(installRequestBuilder.build())
        splitInstallManager.registerListener(object : SplitInstallStateUpdatedListener() {
            fun onStateUpdate(splitInstallSessionState: SplitInstallSessionState) {
                if (splitInstallSessionState.status() === SplitInstallSessionStatus.INSTALLED) {
                    //TODO: Change Language Here
                }
            }
        })
    }
}

Next, when a user opens the app we have to install the downloaded languages. which is done in the attchBaseContext() method.

@Override
protected void attachBaseContext(Context base) {
    super.attachBaseContext(base);
    // It will install all the downloaded langauges into the app
    SplitCompat.install(this); 
  }

Done!

For more information, you can check the official documentation

Configure on demand delivery | Android Developers

Imagine building an Android app with a Complex UI that is getting populated from the data model, whenever the data changes or we have to update the data then it will be a very tedious and error-prone task to manually update every view. What if there is a way to do this work in a single line of code, Thanks to the Android team they have provided methods in ViewDataBinding to do this.

Let’s understand two methods of ViewDataBinding which will help us in this scenario by taking an example of a Shopping App that displays a list of products that has product names, prices, and availability. Additionally, there is an “Add to Cart” button for each product.

1. invalidateAll() -

We can use the invalidateAll() method when there is a change in the product list such as showing only available products or showing only not available products. In that case, invalidateAll() on the root binding object for the list will ensure that all views in the list are updated accordingly.

binding.invalidateAll()

Instead of manually updating each view one by one, you can simply call invalidateAll() on the binding object.

Definition: This Function is used to do the re-evaluation of all the binding expressions and update the associated UI or basically, it is used to refresh the UI

2. executePendingBindings() -

executePendingBindings() can be used when you want to update the “Add to Cart” button immediately. In this case, after updating the availability status of the product in the data model, calling executePendingBindings() on the binding object associated with that specific product item’s layout will execute the pending binding updates, ensuring the button reflects the latest availability state.

binding.executePendingBindings()

Definition: The executePendingBindings() function is used to immediately update or execute the pending data binding updates. it ensures that all pending updates are applied to the associated UI components.

While both invalidateAll() and executePendingBindings() are used to update the UI when the underlying data model changes, they serve different purposes and are applied in different scenarios. Here's when you would typically use each method:

invalidateAll():

• This method is useful when you need to update multiple views in the layout simultaneously or when you want to ensure a complete refresh of the UI.

• For example, if you have a data-bound layout with numerous views, and you want to update all of them at once to reflect changes in the data model, you can invoke invalidateAll()

executePendingBindings():

• This method ensures that any pending binding updates for a particular view or views are executed immediately, without waiting for the next layout pass.

• For example, if you have a specific view that needs to be updated immediately after a change in the data model, you can call executePendingBindings() on that view's binding object to trigger the update.

In summary, use invalidateAll() when you need to update the entire layout or multiple views simultaneously, providing a comprehensive refresh of the UI. On the other hand, use executePendingBindings() when you want to immediately update specific views affected by changes in the data model, ensuring real-time updates.

In the last part, we have covered the basics of Internationalization & Localization concepts, starter project structure, and the approach for internationalization by localizing the storyboard and localizing strings in the swift code. The previous approach was not so efficient as it was a time consuming and had too much manaul tasks.

Live localization via storyboards! What‘s in this new approach?

Let’s take a scenario, Ram is an iOS developer working on a large and a complex app and now he wants to add i18n support for multiple languages (let’s say five different languages).
Now, instead of localizing the whole storyboard, we will add a @IBInspectable elemnet for each objects(Button, Label, TextField,…) in storyboard in such a way that we don’t have to manually localize the text in each object in the storyboard & it should be automatically mamaged by our helper extensions (which we will add in implementation part).
Also, we will add a helper to get the translated strings as per the selected language. So, now let’s jump into the implementation part.

1. Add additional languages support to the app as we do in the last article.
Project Navigator → project name InternationalizationDemo→ Info segment → check the Use Base Internationalization → click on add + button →
Choose Hindi (hi) → but don’t checkmark the storyboards.

2. Add Localizable.strings files for both English(base language) & Hindi that will contain all different texts exisiting in storyboards, xib files and swift codes.

3. Create a helper class LocalizationHelper that will return the localized string for different texts as per the selected language.

4. Add Localizable Extensions for @IBInspectable elements.

5. Add values for localizedKey in the attribute inspector for each objects in the storyboard.

6. Localizing strings in the swift code using LocalizationHelper

Adding the Localizable Strings

Let’s add the localize strings files similarly as we do in the last part.

Select File → New File → search for Strings File → keep the name Localizable as default.
• Xcode will generate Localizable.strings file. In the file inspector, you will see localization panel. Click localize and select “Localize”.
• Again in the file inspector, check Hindi in localization panel. Xcode will generate two Localizable.strings files for both English and Hindi.

Let’s add all the different strings located in the complete project (in storyboards, in xib files, swift codes,…) firstly for the base language.

/* 
  Localizable.strings (English)
*/

"sign-in-to-account" = "Sign in to your account";
"enter-email" = "Enter email";
"email" = "Email";
"enter-password" = "Enter password";
"password" = "Password";
"sign-in" = "Sign in";
"sign-up-here" = "Don't have an account. Sign up here";

"back" = "Back";
"back-to-login" = "Back to login";
"create-new-account" = "Create a new account";
"confirm-password" = "Confirm password";
"sign-up" = "Sign up";
"account-already-exists" = "Account already exists.";
"account-created-successfully" = "Account created successfully.";
"invalid-credentials" = "Invalid Credentials";

"welcome-user" = "Welcome %@";
"select-language" = "Select Language";
"done" = "Done";
"error" = "Error";
"ok" = "OK";
"close" = "Close";

Similarly add the translated strings in Localizable (Hindi).

/* 
  Localizable.strings (Hindi)
*/

"sign-in-to-account" = "अपने अकाउंट में साइन इन करें";
"enter-email" = "ईमेल दर्ज करें";
"email" = "ईमेल";
"enter-password" = "पासवर्ड दर्ज करें";
"password" = "पासवर्ड";
"sign-in" = "साइन इन";
"sign-up-here" = "कोई खाता नहीं है. यहां साइन अप करें";

"back" = "पीछे";
"back-to-login" = "लॉगिन पर वापस जाएं";
"create-new-account" = "एक नया खाता बनाएं";
"confirm-password" = "पासवर्ड की पुष्टि करें";
"sign-up" = "साइन अप";
"account-already-exists" = "खाता पहले से मौजूद है।";
"account-created-successfully" = "खाता सफलतापूर्वक बनाया गया.";
"invalid-credentials" = "अवैध प्रत्यय पत्र";

"welcome-user" = "स्वागत है %@";
"select-language" = "भाषा चुने";
"done" = "हो गया";
"error" = "त्रुटि";
"ok" = "ठीक है";
"close" = "बंद करे";

Localization Helper

What we are going to do first is to create a helper class that will return the localized string for different texts as per the selected language.
Add a new swift file in Helpers folder namely LocalizationHelper.swift .
Create a new singleton class for this LocalizationHelper.

class LocalizationHelper {
    public static func localize(_ key: String) -> String {
        return NSLocalizedString(key, comment: "")
    }

    private static func localize(args: [CVarArg], key: String) -> String {
        String(format: localize(key), args)
    }
}

Above func localize(:), will return the loclaized strings for both cases with or without arguments for localization where aruguments refers to the data to be inserted at run time.

class LocalizationHelper {
    //  ------

    static var signInToAccount: String { localize("sign-in-to-account") }
    static var enterEmail: String { localize("enter-email") }
    static var email: String { localize("email") }
    static var enterPassword: String { localize("enter-password") }
    static var password: String { localize("password") }
    static var singIn: String { localize("sing-in") }
    static var signUpHere: String { localize("sign-up-here") }
    
    static var back: String { localize("back") }
    static var backToLogin: String { localize("back-to-login") }
    static var createNewAccount: String { localize("create-new-account") }
    static var confirmPassword: String { localize("confirm-password") }
    static var signUp: String { localize("sign-up") }
    static var accountAlreadyExists: String { localize("account-already-exists") }
    static var accountCreatedSuccessfully: String { localize("account-created-successfully") }
    static var invalidCredentials: String { localize("invalid-credentials") }
    
    static var selectLanguage: String { localize("select-language") }
    static var done: String { localize("done") }
    static var error: String { localize("error") }
    static var ok: String { localize("ok") }
    static var close: String { localize("close") }
    static func welcomeUser(_ args: [CVarArg]) -> String {
        localize(args: args, key: "welcome-user")
    }
}

Now, added static variables corresponding to each different texts present in entire project. So, if we haveb to access the transalted string for text “Sign in”, we simply will call — LocalizationHelper.singIn.

Localizable Extensions to create @IBInspectable elements

Let’s start with the starter project. Create a new swift file in Extensions folder LocalizableExtensions.swift. Add a protocol StbIBLocalizable (storyboard localizable) which our storyboard objects will conform for an adding an additional element.

//  LocalizableExtensions.swift

import Foundation

protocol StbIBLocalizable {
    var localizedKey: String? { get set }
}

To use this StbIBLocalizable protocol, we will use extensions of UILabels, UIButton,.. to conform to this protocol. Let’s start by adding extensions for UILabel in LocalizableExtensions.swift .

extension UILabel: StbIBLocalizable {
    @IBInspectable var localizedKey: String? {
        get { nil }
        set(key) {
            if let key = key { text = LocalizationHelper.localize(key) }
        }
    }
}

After conforming to the StbIBLocalizable protocol by UILabel, making variable localizedKey as an @IBInspectable element, you will see an additional field in the attribute inspector in main.storyboard. We can use this field to directly add the text and it will be automatically translated.

Let’s add the text “enter-email” in the localizedKey, and run the he application in Hindi. You can run your application using different locales by Option + Clicking on the (Play) Start active scheme button, then going to Run → Options → App Language → Change to a supported language (Hindi in this case) → Click Run.

You will see that that the “Enter email” is being showed as “ईमेल दर्ज करें”.

Let’s add the extensions for other elements. You can customize the thsese as per your need.

extension UIButton: StbIBLocalizable {
    @IBInspectable var localizedKey: String? {
        get { nil }
        set(key) {
            if let key = key {
                setTitle(LocalizationHelper.localize(key), for: .normal)
                setTitle(LocalizationHelper.localize(key), for: .highlighted)
                setTitle(LocalizationHelper.localize(key), for: .selected)
                setTitle(LocalizationHelper.localize(key), for: .disabled)
            }

        }
    }
}

extension UINavigationItem: StbIBLocalizable {
    @IBInspectable var localizedKey: String? {
        get { nil }
        set(key) {
            if let key = key { title = LocalizationHelper.localize(key) }
        }
    }
}

extension UIBarItem: StbIBLocalizable {
    @IBInspectable var localizedKey: String? {
        get { nil }
        set(key) {
            if let key = key { title = LocalizationHelper.localize(key) }
        }
    }
}

extension UITextField: StbIBLocalizable {
    @IBInspectable var localizedKey: String? {
        get { nil }
        set(key) {
            if let key = key {
                text = LocalizationHelper.localize(key)
                placeholder = LocalizationHelper.localize(key)
            }
        }
    }
}

Add the localized key from strings file in the localizedKeyin the attribute inspector for each objects. Now, we are done with the localizing the storyboard objects, let’s jump how to localize strings in the swift code.

Replace the label text in WelcomeViewController by using LocalizationHelper class member to get the translated string as per the current selected language & our welcome page will look like this:

override func viewDidLoad() {
    super.viewDidLoad()
    welcomeLabel.text = LocalizationHelper.welcomeUser([user!])
}

How to manually change the app language?

Apple doesn’t suggest us to manually change the app language, but since we are doing it, we will create a new view controller that will gives us option to choose among the available app languages. We will use UserDefaults to store and track the current selected langauage.
Our app will initialize with selected langauage (base langauage if not any selected). Also, on change of selected langauage, we will revert the user to home page (login page in our case). So, let’s start the implemenataion.

Go to the Extension file, and add the following extension for the UserDefaults. We will store the current selected language with the "i18n_language" key and the variable selectedLanguage will be used to set and get the selected language.

If you want to learn more about the UserDefaults , you can go through this article here.

extension UserDefaults {
    var selectedLanguage: String? {
        get { string(forKey: "i18n_language") }
        set { set(newValue, forKey: "i18n_language") }
    }
}

Also, we need to modify func localize() in our LocalizationHelper class, so that it checks the current selected language in UserDefaults & return the translated strings as per the selected language.

class LocalizationHelper {
    public static func localize(_ key: String) -> String {
        if UserDefaults.standard.selectedLanguage == nil {
            UserDefaults.standard.selectedLanguage = "en"
            UserDefaults.standard.synchronize()
        }
        
        let lang = UserDefaults.standard.selectedLanguage
        guard let path = Bundle.main.path(forResource: lang, ofType: "lproj") else {
            return NSLocalizedString(key, comment: "")
        }
        
        let bundle = Bundle(path: path)
        return NSLocalizedString(key, bundle: bundle!, comment: "")
    }
    // -------
}

SelectLanguageViewController

Till now, we have added all the necessary requirements for translating the strings as per the selected langauge saved in the UserDefaults. Now, let’s add a page for selecting the language.
I have created a new UITableViewController into the Main.storyboard and linked it to the classSelectLanguageViewController. Also, added a settings button onto the navigation right bar button item in the WelcomeViewController, which will redirect the user to the SelectLanguageVC page. The SelectLanguageVC tableview list items displays avialable language and is calculated from the localization bundle. So, here the code for the SelectLanguageViewController, SelectLanguageViewModel, SelectLanguageTableViewCell & the Language model.

class SelectLanguageViewController: UITableViewController {
    private let viewModel = SelectLanguageViewModel()

    override func viewDidLoad() {
        super.viewDidLoad()
        let doneButton = UIBarButtonItem(title: LocalizationHelper.done, style: .plain, target: self, action: #selector(self.doneButtonPressed))
        self.navigationItem.rightBarButtonItem = doneButton
    }
    
    @objc func doneButtonPressed() {
        if UserDefaults.standard.selectedLanguage != viewModel.selectedLanguage.code {
            UserDefaults.standard.selectedLanguage = viewModel.selectedLanguage.code
            SceneDelegate.shared?.reloadWindow()
        }
        else {
            self.dismiss(animated: true, completion: nil)
        }
    }

    // MARK: - Table view data source
    override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
        viewModel.availableLanguage.count
    }

    override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
        let cell = tableView.dequeueReusableCell(withIdentifier: "cell", for: indexPath) as! SelectLanguageTableViewCell
        cell.languageNameLabel.text = viewModel.availableLanguage[indexPath.row].name
        cell.accessoryType = viewModel.selectedLanguageIndex == indexPath.row ? .checkmark : .none
        return cell
    }
    
    override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
        viewModel.selectedLanguageIndex = indexPath.row
        tableView.reloadData()
    }
}

struct Language {
    var code: String
    var name: String?
}

class SelectLanguageViewModel {
    var availableLanguage = [Language]()
    var selectedLanguageIndex: Int
    var selectedLanguage: Language {
        availableLanguage[selectedLanguageIndex]
    }

    init() {
        availableLanguage = Bundle.main.localizations
            .filter { $0 != "Base" }
            .map { Language(code: $0, name: Locale.current.localizedString(forLanguageCode: $0)) }
        
        selectedLanguageIndex = availableLanguage
            .map { $0.code }
            .firstIndex(of: UserDefaults.standard.selectedLanguage ?? "en") ?? 0
    }
}

class SelectLanguageTableViewCell: UITableViewCell {
    @IBOutlet weak var languageNameLabel: UILabel!
}

So what I’m doing here first is to find the available languages for the tableview datasource. Also, I’m using the UserDefaults to track the current selected language, so when the done button is pressed, we reload the entire rootViewController window, and then all our strings are translated through the LocalizationHelper as per the current selected language.

To reload the entire rootViewController window, I had created a func reloadWindow in SceneDelegate which will set the window rootViewController as the new instantiateInitialViewController .

class SceneDelegate: UIResponder, UIWindowSceneDelegate {
    var window: UIWindow?
    static weak var shared: SceneDelegate?

    func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
        // -----
        Self.shared = self
    }
    
    // ----------------
    func reloadWindow() {
        let vc = UIStoryboard(name: "Main", bundle: nil).instantiateInitialViewController()!
        window?.rootViewController = vc
    }
}

Here, is the final output

Summary

Hurray, we have now completed the better approach to add internationalization support in our app. Adding the internationalization support to an app by this approach is little less complex than the last one and can easily be done for multiple languages. Also, we have learned how we can add options to manually change the app language.
Here, is the github link for the completed project.
You can also find the previous discussed approach here.

Thanks for Reading!
If you liked this article, please share and 👏 so other people can read it too :)

Are you developing an iOS application, but you do not know how to internationalize and localize your iOS app? If the answer is yes, then you are at the right place. In this article, we will see how to setup Internationalization and Localization in an iOS application using Swift.
This article will elaborate different approaches to add internationalization support in an iOS app and will also look into its implementation via code.

• Internationalization vs Localization

• Setting-up the project for i18n

• How do I add additional languages support to my app?

• Localizing the Storyboards

• How do I localize strings in my Swift code?

• Consequences of this approach

• What’s next to do ?

A developer or an individual who is new to this internationalization concept often get confuse between these two words “Internationalization and Localization”. These are different but interconnected concepts.
So, what does these basically means? What’s differences between them?

Internationalization (i18n) is process of making our app adaptable to different languages and regions. On the other hand, Localization (l10n) is process of translating our app to different languages.

Internationalization is abbreviated as “i18n” as there are eighteen number of characters between “i” and “n”. Similarly, Localization is abreviated as “l10n” as number of characters between “l” and “n” equals ten.

Setting-up the project for i18n

Here’s a starter-project for you so that we can jump directly to the i18n implementation. Before that, let me explain you the project structure.

The starter-project has a very simple layout, has only three view controllers: SigninViewController, SignupViewController and a WelcomeViewController and a singleton class KeychainHelper to manage KeyChain services for authentication.

On the initialization of app, user lands on the “SignIn” page, where he/she has option to either sign in or go to “SignUp” page. On successful sign in, user gets redirected to “Welcome” page.

In this project, I have used KeyChain services for data encryption and authentication.

Download and run the starter-project and go through the project once to understand it. Now, let’s start our i18n implementation.

How do I add additional languages support to my app?

From the Project Navigator, select the project name InternationalizationDemo, make sure you are on Info segment. Then check the Use Base Internationalization and click on add + button.
Choose Hindi(hi)and for now let’s check both storyboards.

Changes in project after adding a language

After adding a supported languages, you’ll realize some changes in your project. Let’s discuss these changes, and what it means for our goal of localization.
• Language folders: Xcode will generate hi.lprojdirecory that will contain Main.strings for Hindi.
• Storyboard: Main.storyboard should now be a folder containing the base Main.storyboard and Main.strings of supported languages (Hindi here).
•Main.strings: Each object(labels, buttons,…) are added in this file with an unique ObjectID. Don’t directly change this ObjectID for any item.

Localizing the Storyboards

Click on Main.strings(Hindi) to open it in the editor. You should already have an entry for the Sign in to your account label which will look something like this:

/* Class = "UILabel"; text = "Sign in to your account"; ObjectID = "61H-Ar-Y7e"; */
"61H-Ar-Y7e.text" = "Sign in to your account";

Replace the English translation with the Hindi translation. Do the same for all other items. Your Main.strings(Hindi)will look like this:

/* Class = "UILabel"; text = "Sign in to your account"; ObjectID = "61H-Ar-Y7e"; */
"61H-Ar-Y7e.text" = "अपने अकाउंट में साइन इन करें";

Now let’s run the application in Hindi to see if our localization worked. By default, every time you run the application, it uses the base language. You can run your application using different locales by Option + Clicking on the (Play) Start active scheme button, then going to Run → Options → App Language → Change to a supported language (Hindi in this case) → Click Run.

Our app looks like this after adding language support in Hindi.

How do I localize strings in my Swift code?

If you go to the Welcome page, you will notice that the welcomeLabel is not localized because we are programmatically adding the code here welcomeLabel.text = “Welcome \(user ?? “”)". So, how do we localize our swift code ?
• Select File → New File → search for Strings File → keep the name Localizable as default.
• Xcode will generate Localizable.strings file. In the file inspector, you will see localization panel. Click localize and select “Localize”.
• Again, in file inspector, check Hindi in localization panel. Xcode will generate two Localizable.strings files for both English and Hindi.

We now have two localizable.strings file — one for the base language and other one for the Hindi.

A .strings file stores key-value pairs, just like a Dictionary. Conventional practice uses the base language translation as the key. The file has a specific, but simple, format:

"KEY" = "CONTENT";

Subsequently, let’s add the key and value pair text used in the Localizable. strings. Below is an example where I added the localization for welcomeLabel

//  Localizable (English)
"Welcome %@" = "Welcome %@";

//  Localizable (Hindi)
"Welcome %@" = "स्वागत है %@";

NSLocalizedString(_:tableName:bundle:value:comment:) is the primary tool you use in your code to access these localized strings. The tableName, bundle, and value parameters all have default values so you normally specify only the key and the comment. The comment parameter is there for you to provide a hint to translators as to what purpose this string serves in your app’s user experience.

Let’s say we have a key-value pair in .strings file as
“welcome rohit” = “स्वागत है रोहित”;

Here, we can update the text of welcomeLabel usingNSLocalizedString(:) to make it localizable.
welcomeLabel.text = NSLocalizedString(“welcome rohit”, comment: “welcome”)

💡 We can include format specifiers in either the key or the value portion of the string to allow you to insert real data at run time.

Open WelcomeViewController and add the following code in viewDidLoad() method.

override func viewDidLoad() {
    super.viewDidLoad()
    let formatString = NSLocalizedString("Welcome %@", comment: "welcome user")
    welcomeLabel.text = String.localizedStringWithFormat(formatString, user!)
}

Now, run the app and you will see that our welcomeLabel in WelcomeViewController is now able to localized as per the selected language. Also, the user details in welcomeLabel dynamically get updated since we added format specifiers while localizing it.

Consequences of this approach

Now, we have completed the i18n implementation and our app is ready for both Hindi & English (base) language. But, there are some consequences in the approach we have taken for this implementation.

Let’s take a scenario, Ram is an iOS developer working on a large and a complex app and now he wants to add i18n support for multiple languages (let's say five different languages). He take the above mentioned approach for the implementation. What will happen next ?
• Xcode will generate five Main.strings for Main.storyboard localization and Ram will have to manually add translated strings for all objects (labels, buttons,…).
• Same this have to be done also for .xib files.
• High changes of making mistake of altering objectID.
• A very time consuming and an inefficient approach.

What to do next ?

In the next continuation part of this article, we will look into another approach to overcome these consequences. Also, we will also learn about how to programmatically add language switching.
Here is the github link for the completed project using the above approach.
You can find next part here.

Thanks for Reading!
If you liked this article, please share and 👏 so other people can read it too :)

Monitoring druid query execution, ingestion, and coordination is essential for production clusters which are powering the user-facing dashboards. The expectation of these dashboards/widgets is to be loaded in a few seconds with near-realtime data. So monitoring the health of the druid cluster is required for production setups.

How can I emit the monitoring metrics?

We can configure Druid to emit metrics that are essential for monitoring query execution, ingestion, coordination, and so on.

To know more about all the metrics refer to the official documentation

Monitors Configuration:

We need to use the monitors to monitor the respective processes to collect the metrics. Add this config in common.runtime.properties

druid.monitoring.monitors=["org.apache.druid.client.cache.CacheMonitor", "org.apache.druid.java.util.metrics.JvmMonitor", "org.apache.druid.java.util.metrics.CpuAcctDeltaMonitor", "org.apache.druid.java.util.metrics.JvmThreadsMonitor", "org.apache.druid.server.metrics.EventReceiverFirehoseMonitor", "org.apache.druid.server.metrics.TaskCountStatsMonitor"]

Most of the production druid clusters will be running in a clustered deployment where we will be running master (coordinator, overload), data(middleManager, historical), and query(router, broker) nodes separately in different VMs

So we need to configure the druid.monitoring.monitors property according to the process we are running.

Example:

Master — (Coordinator-Overlord)

druid.monitoring.monitors=["org.apache.druid.client.cache.CacheMonitor", "org.apache.druid.java.util.metrics.JvmMonitor", "org.apache.druid.java.util.metrics.CpuAcctDeltaMonitor", "org.apache.druid.java.util.metrics.JvmThreadsMonitor", "org.apache.druid.server.metrics.EventReceiverFirehoseMonitor", "org.apache.druid.server.metrics.TaskCountStatsMonitor"]

Data — (Historical, MiddleManager)

druid.monitoring.monitors=["org.apache.druid.client.cache.CacheMonitor", "org.apache.druid.java.util.metrics.JvmMonitor", "org.apache.druid.java.util.metrics.CpuAcctDeltaMonitor", "org.apache.druid.java.util.metrics.JvmThreadsMonitor", "org.apache.druid.server.metrics.EventReceiverFirehoseMonitor"]

Query — (Broker, Router)

druid.monitoring.monitors=["org.apache.druid.client.cache.CacheMonitor", "org.apache.druid.java.util.metrics.JvmMonitor", "org.apache.druid.java.util.metrics.CpuAcctDeltaMonitor", "org.apache.druid.java.util.metrics.JvmThreadsMonitor", "org.apache.druid.server.metrics.EventReceiverFirehoseMonitor", "org.apache.druid.server.metrics.QueryCountStatsMonitor"]

Emitter Configuration:

We need to add/edit the following config in the common.runtime.properties file to enable the emitter which emits metrics to an HTTP endpoint.

druid.emitter=http
druid.emitter.http.recipientBaseUrl=http://{{druid_exporter_host}}:{{druid_exporter_port}}/druid

Restart the nodes/processes after adding the above emitter and monitoring the config

Druid Exporter:

To collect and export metrics from druid we have an open-source druid exporter from Opstree — https://github.com/opstree/druid-exporter

Installation:

We can deploy the druid exporter using

1. releases — https://github.com/opstree/druid-exporter/releases

2. Kubernetes — https://github.com/opstree/druid-exporter/tree/master/manifests

3. docker-compose — https://github.com/opstree/druid-exporter/tree/master/compose

I will be going with release deployment directly — running druid exporter in one of the master nodes

wget https://github.com/opstree/druid-exporter/releases/download/v0.11/druid-exporter-v0.11-linux-amd64.tar.gz
tar -xvzf druid-exporter-v0.11-linux-amd64.tar.gz

Let’s create a systemd file to run the druid-exporter

[Unit]
Description=Druid Exporter
Documentation=https://github.com/opstree/druid-exporter
Requires=network.target
After=network.target
[Service]
Type=simple
WorkingDirectory=/opt
User=root
Group=root
ExecStart=/PATH_TO_DOWNLOADED_FOLDER/druid-exporter -p 8020 -d DRUID_COORDINATOR_OR_ROUTER_URL --druid.user="" --druid.password="" --metrics-cleanup-ttl=15 --no-histogram
[Install]
WantedBy=default.target

Available options and flags — https://github.com/opstree/druid-exporter#available-options-or-flags

Enable the systemd file

systemctl enable druid-exporter.service

Start the druid-exporter

systemctl start druid-exporter

Verify the installation and check the metrics in the endpoint

curl http://druid_export_host:port/metrics

Grafana dashboard:

Cool, we have emitted, collected, and exported the monitoring metrics to Prometheus. Now it’s time to visualize and create alerts (if needed).

We have added more panels and visualization to the existing dashboard provided by Opstree. Download the Grafana dashboard JSON from here.

Corrections/suggestions are welcome. Thanks for reading.

References:

https://github.com/opstree/druid-exporter

https://druid.apache.org/docs/latest/configuration/#enabling-metrics

https://druid.apache.org/docs/latest/operations/metrics

The LLM Revolution and the Need for Change

The digital age introduced the world to a series of innovations, with Large Language Models (LLMs) like Chat GPT standing out prominently. These models, with their vast knowledge base and intricate design, transformed industries overnight. However, as we at Yellow.ai meticulously analyzed their integration into real-world applications, potential hurdles became evident. Imagine a customer support scenario where a customer is on a call with a voice-based assistant, driven by LLMs, With the current state-of-the-art OpenAI’s gpt4 takes its own sweet time to respond. This gives a very bad experience to the end user who is waiting for help. Beyond just the user experience, the cost of inference is also very high for these gigantic models. With increasing reliance on API-based solutions, the unpredictability of costs and response times made it clear: a paradigm shift was essential.

An Orthogonal Shift: Task-Specific Fine-Tuning

While the potential of LLMs like Llama was widely recognized, there was no comprehensive knowledge about how to fine-tune them effectively. At Yellow.ai, we discerned a unique opportunity. Instead of a generalized approach, why not hone our efforts on tasks — the very core of any conversation? This perspective shifted our focus towards models like the 7 billion, 13 billion, and 33 billion parameter versions of Llama. With these distinct models, each was meticulously crafted for specific conversational tasks. Under this framework, our ML/Data Science team invested their expertise, ensuring each model was perfectly calibrated for its designated role in the conversational ai pipeline.

Efficiency, Cost & Accuracy

In our quest for superior conversational AI, merely redesigning the fine-tuning process wasn't enough. We ventured further into the realm of technological innovations. Quantization techniques became our trusted ally in this journey, offering us a dual advantage: enhanced model performance and substantial cost savings. This restructuring allowed us to create a tiered system. Smaller, more efficient models tackled foundational tasks, ensuring swift and accurate responses. This, in turn, significantly reduced the chances of hallucinations.

The Road Ahead

While we continue our innovations, we humbly acknowledge the powerhouses of the LLM world. Models such as GPT-4 are in a league of their own, offering unmatched insights and depth. However, with our revamped strategy, their deployment is more strategic. By ensuring they are used where their intricate understanding is indispensable, we balance computational prowess with practicality. The advent of LLMs has opened doors we hadn't imagined.

Our dream of realizing a staggering 90% automation rate in customer support no longer seems a distant star but a tangible reality. At Yellow.ai, we're not just embracing the future; we're shaping it.