What to do if your Shopify automated testing flow fails
What is Shopify automated testing?
Automated website testing tools for Shopify allow merchants to set their online store QA testing on autopilot. Think of it like a team of robotic QA testers that are always testing key customer journeys to ensure your store is functioning correctly.
When you automate Shopify with tools like Uptime, you gain back valuable time that can be spent growing your business. For example, just last year Uptime’s Shopify automated testing saved merchants over 3K hours of manual testing – in that time merchants could learn over 6 languages!
Setting up Shopify monitoring
With Uptime, setting up Shopify automated testing is easy and only takes a few minutes to set up. It’s truly the best way to test Shopify.
Out of the box, you’ll get real-time Shopify status page monitoring and third party app status page monitoring. For setting up automated QA test flows you can simply follow our 3-step guide. If you need some ideas of what test flows to record, make sure to check out our ecommerce website testing blog post.
What to do when an automated test flow fails
If one of your test flows fail you will receive Uptime alerts (and Slack if you have it connected) with useful information to help start understanding the issue.
Here are some helpful tips to assess a test flow failure:
Check the test flow screenshot
When an automated QA test flow fails we will take a screenshot at the exact point the test failed. This is a great starting point to begin to assess why there was a failure in the first place. For example, if a test flow was ensuring an upsell could be added to cart, but the screenshot doesn’t show the upsells appearing, this may be an indication of what is causing the test failure. This is exactly how the team at Prismfly debugged a test flow failure on their client's store.
To view a screenshot of the test failure, head over to your automated test flow tab in the Uptime app, hover over the test flow that recently failed, and click the ‘View screenshot’ button.
Test the flow yourself live
After reviewing the screenshot, it can be helpful to double-check the test flow by clicking through the same steps yourself live on the online store. This will help you confirm if it’s just a passing issue, or if something is actively broken.
For example, when Uproot Clean received an Uptime alert about their post-purchase upsell test flow failing, they immediately when to the ‘Thank you’ page to test the functionality themselves, at which point they realized it was not functioning properly.
Check the advanced debugging information
When a test flow fails in Uptime we provide some helpful advanced debugging information to help you better understand why a test failed.
This advanced debugging information can be found above the screenshot image in the Uptime app. Here are the types of advanced debugging info you can expect to see and descriptions of what they mean:
Error: Unable to locate element by its selectors: this error means we tried to click a specific button or element on your online store but were unable to successfully do so. Additionally, with this error the Uptime app provides a list of the elements we tried to click on – this gives you a head start in trying to debug what was un-clickable.
Error: Completed the flow at an unexpected URL: this error means that Uptime expected the test flow to end at one location, but instead it ended at another location. An error like this may occur when navigating the website, for example, if a test flow was navigating to the ‘shirts’ collection but ended up on the ‘pants’ collection instead.
Re-record if necessary
Sometimes, an adjustment on your store will cause a test flow to fail, however, the actual functionality of the store is still working. In this case, it may be necessary to simply re-record the test flow.
As we mentioned at the beginning of this blog post, recording automated test flows with Uptime is very easy and only takes a minute. Here’s a simple 3-step guide to record automated test flows for your Shopify online store.
Get some peace of mind
Uptime’s goal is to give you peace of mind knowing someone is always monitoring your online store. If your team wants to get some peace of mind, install the app now for free or book time with us and we can help you get started!
