- Decide what metrics to track in your app
- Determine how you will create variations
- Learn key design and development considerations for Android and iOS experiments
To succeed in the world of cross-platform and mobile optimization, companies like Facebook, LinkedIn, and Airbnb need to be able to learn quickly, iterate, and respond to their mobile user behavior.
In this article, we will show you how you can do the same for your mobile apps by using Optimizely Experimentation.
With Optimizely Feature Experimentation, you can experiment and push content without repeated submissions to the App Store, thereby compressing the app development cycle and making your platform more responsive to user behavior.
Define your mobile goals
Optimizely Experimentation on mobile enables experimentation that will equip you with the data you need to maximize the business value of your app. To do this, you should identify what key metrics you want to optimize through testing. Here are the most common ones that testing can influence:
| Metric | Why It is Important | How You Can Test |
|---|---|---|
| Daily Active Users (DAUs) | DAUs measure your engagement levels and retention. They are an indicator of your app's stickiness and the lifetime value (LTV) of your users. | Changes to UX, messaging, and features can make your app stickier and increase DAUs. |
| Downloads / Registrations | Downloads and registrations serve as the top-of-funnel metric for user engagement, and should correlate with DAUs. | Experiment with your app's icon, name, keywords, and screenshots in the app store to promote downloads. Try simplifying or eliminating your in-app registration flow. |
| Retention | DAUs divided by downloads measures your retention. | If you see a high download rate compared to DAUs, you have a clear reason to experiment with retention features. |
| In-App Purchases (IAPs) or ecommerce | IAPs (or purchases in an ecommerce app) are a direct source of revenue and a clear conversion metric. | Experimenting with prices and pricing models can help optimize IAPs, especially for freemium apps. Ecommerce apps also have a variety of options to drive purchases. |
| Average Rating & Reviews | Ratings and reviews quantify what users think of your app and serve as a leading indicator of lifetime value (LTV). | Test different timing and prompts for your ratings and reviews. |
| Taps and Views | For apps that contain CTAs, forms, or ads, taps and views can serve as indicators of engagement, or even revenue. | Experiment with changes that make it easier for users to tap CTAs, complete forms, or highlight ad content. |
| Offline Conversions | Some apps have offline conversions, such as redemption of app-exclusive offers that occur outside of the app. | Changes to messaging and promotions can affect offline conversion numbers. |
| Sessions | More time in-app and more times in-app have a strong correlation to higher conversion rates. | Three session goals are automatically added to your experiments: Sessions, Average Session Length, and Sessions Per Visitor Per Week. |
Design considerations for app testing
After you have identified key metrics for how to measure the success of your tests, consider how you might test experiences that increase user delight and reduce friction. When testing on mobile devices, use these guidelines to shape your hypotheses:
Design with the physical “sweet spot” in mind. Only 1/3 of the screen real estate is effortlessly accessible by the thumb. Test to identify the ideal placement for the elements that drive your key conversion metrics.
Know your audience. If you have loyal users on the web, they will likely intuit your app's functionality but expect more agility. For new or mobile-first users, keep the interface simple, focus on key actions that will drive repeat usage. Explore delivering tailored experiences to each group.
Focus on NUX (new user experience). Consider how users will be introduced to your app when they load it for the first time.
- Social login. Including social login options (Facebook, Google, etc.) can decrease barriers to entry.
- Make users feel invested early. Mobile users are less likely to type multiple inputs on a small screen. Get them engaged early through discovery and engagement.
- Reduce friction and increase interactivity. With less screen real estate than the web, mobile users appreciate search, filtering, image browsing, and other focusing features early in the funnel.
- Experiment with different flows. For example, your app may include an instructional mode, labeled icons, or full, ungated content on first sign-in. It woul d be worthwhile to see which experience works best for retention.
Respect the tap, embrace the swipe, and consider the scroll. On the web, clicks are a key way of interacting with content, but in apps, you have the luxury to think through other ways of prompting user interactivity. Touch screens offer the unique ability to swipe and pinch content, which many users find engaging. Touch screens also offer a more natural scrolling experience than desktops do, and users may be more willing to scroll on mobile devices. Discover which flow works better for your users at key interaction points —more scrolling, tapping to new views, or gestures.
Less is (still) more. On mobile devices, you do not have the luxury of designing for the big desktop screen —so embrace the constraints! Decluttering is already important on the web, but designing for a mobile-first experience means stripping content to its bare minimum. Focus on exposing your main actions and making them simple to perform.
Balance focus and exploration. Apps often feature deliberate process flows that aim to guide users through a series of tasks. When designing for the small screen, delight your users with an experience that is focused enough to provide direction, but open enough to drive exploration and repeated use. Focused experiences that constrain user freedom may drive abandonment due to limited functionality, while open and complex flows may confuse users with competing calls to action and a lack of direction.
Functional considerations for app testing
Most popular apps deliver both task-based functionality and information-seeking content, and users can have different intents based on their source. Mobile users require both because information-seeking is a task that bridges the web and app experiences. This instills confidence in users that they can achieve as much, if not more, through the streamlined app interface, making them more loyal to the app, which leads to more loyalty to the business.
The process flow and milestones you aim to have users achieve should be a central focus of your app testing program. To achieve the optimal flow, constantly test:
- How and when the app asks for permissions
- The length and steps of your registration flow, and how much content is gated before registration
- How and where ads are featured
- Number, sequence, and format of calls to action throughout your app's flow
By focusing on functionality, not just design, you can drive key insights about how users engage with your app.
<%= heading %>
