Here at Kiwi.com, we use Smartlook to verify bugs and issues that our customers report. It also helps us evaluate some of the A/B tests, and our UX research team uses it to understand customer behavior better.
Smartlook is used mainly by testers on our quality assurance team. Our task is testing the Kiwi.com flight booking portal to improve user experience. We review bugs and user experience issues and report them to our development and UX teams.
If a support representative receives a ticket from a customer and believes it’s related to a bug or issue with the interface, they report it to our quality assurance team. We analyze it further and watch the Smartlook video recording of the customer to see what exactly happened. If we confirm it is a bug or a UX issue, we report it to the appropriate team and send them a link to the customer’s recording in Smartlook. Sometimes, the UX team uses Smartlook directly to analyze A/B test results and how new features are used.
Smartlook allows us to verify customer issues in a way that wasn’t possible before. When a customer reports an issue, it’s often difficult to analyze what exactly happened based on just logs or the customer’s brief description. Customers will often say stuff like “Your booking page doesn’t work, so I wasn’t able to book a flight” without providing details. Now, we can simply watch a video recording of that particular customer in Smartlook to see what exactly happened and identify the issue right away. Thanks to Smartlook, we’ve improved the discovery of bugs and UX issues and now achieve faster case resolution times.
There is also a collateral added value in Smartlook. While we watch recordings of particular customers to review an issue they reported, we often discover unrelated UX issues. When watching customer recordings, we noticed, for example, that customers often forgot to fill in their title (Mr./Mrs.) while entering their personal info, which results in a form error when the customer attempts to complete their booking. Thanks to Smartlook, we found out about this and reported it to the UX team, who then improved the interface.
Our quality assurance team verified customer issues using log files from developers, which often didn’t give us such good insights into customer behavior. The review of customer issues was a lengthier process. We didn’t have any evidence for the customer if they were disputing what exactly happened during the flight booking, as the logs were too technical to be used in customer communication.
Our QA testers save approximately eight minutes with each customer issue. We reveal around 10 bugs per month, which we would likely not discover without Smartlook. I also believe Smartlook helps decrease ticket volume for support because we can solve customer issues faster.
Other products weren’t able to record all visitors because they used visitor traffic sampling. Smartlook was the only solution with always-on recording. It was also the only service able to record all elements of our complex website, such as, for example, our dynamic map for flight search.