AWS Device Farm FAQs

AWS Device Farm allows developers to increase application quality, time to market, and customer satisfaction by testing and interacting with real Android and iOS devices in the AWS Cloud. Developers can upload their app and test scripts and run automated tests in parallel across 100s of real devices, getting results, screenshots, video, and performance data in minutes. They can also debug and reproduce customer issues by swiping, gesturing, and interacting with a device through their web browser.

AWS Device Farm is designed for developers, QA teams, and customer support representatives who are building, testing, and supporting mobile apps to increase the quality of their apps. Application quality is increasingly important, and also getting complex due to the number of device models, variations in firmware and OS versions, carrier and manufacturer customizations, and dependencies on remote services and other apps. AWS Device Farm accelerates the development process by executing tests on multiple devices, giving developers, QA and support professionals the ability to perform automated tests and manual tasks like reproducing customer issues, exploratory testing of new functionality, and executing manual test plans. AWS Device Farm also offers significant savings by eliminating the need for internal device labs, lab managers, and automation infrastructure development.

AWS Device Farm supports native and hybrid Android, iOS, and web apps, and cross-platform apps including those created with PhoneGap, Titanium, Xamarin, Unity, and other frameworks.

AWS Device Farm tests are run on real devices. The devices are a mixture of OEM and carrier-branded devices.

Please see our getting started guide.

AWS Device Farm works on Internet Explorer 9 or later and the latest versions of Chrome, Firefox, and Safari.

Your web applications will be tested in Chrome on Android and Safari on iOS.

AWS Device Farm supports files up to 4 GB.

No instrumentation or source code is required to use the built-in tests. Android apps can be submitted as is. iOS apps should be built with “iOS Device” as the target instead of a simulator.

Apps and test packages are automatically removed after 30 days. Logs, video recordings, and other artifacts are stored for 400 days. You can also choose to delete files and results at any time through the AWS Device Farm console or API.

After test execution completes, we perform a series of cleanup tasks on each device, including uninstallation of your app. If we cannot verify uninstallation of your app or any of the other cleanup steps, the device will be removed and will no longer be available.

While we continue to add additional cleanup steps and improve the cleanup process, it is possible for data to persist between sessions in some cases, especially if you make use of the device system outside the context of your app. For this reason, and because AWS Device Farm captures video and logs of activity taking place during your use of each device, it is recommended that you avoid providing or entering sensitive information such as account info (E.g., Google Account, Apple ID), personal information, and other security-sensitive details during your automated test and remote access sessions.

On iOS, we replace the embedded provisioning profile with a wildcard profile and resign the app. If you provide it, we will add auxiliary data to the application package before installation so the data will be present in your app’s sandbox. Resigning the iOS app results in the removal of certain entitlements. This includes App Group, Associated Domains, Game Center, HealthKit, HomeKit, Wireless Accessory Configuration, In-App Purchase, Inter-App Audio, Apple Pay, Push Notifications, and VPN Configuration & Control.

On Android, we resign the app. This may break functionality that depends on the app signature, such as the Google Maps Android API. It may also trigger anti-piracy and anti-tamper detection available from products such as DexGuard. For built-in tests, we may modify the manifest to include permissions required to capture and save screenshots.

AWS Device Farm has a large (and growing) selection of Android, iOS, and Fire OS devices. We add popular new devices as they are released by manufacturers. We also add new devices as new OS versions are released. See the list of available devices.

We currently have international devices from India. We use market data and customer feedback to continuously update the fleet. If you would like to see a device that isn’t in our fleet, please let us know.

For Automated Testing, devices are selected through a collection called a device pool. Some curated device pools are provided automatically, but you can create your own pools, too. During execution, tests will be run against all devices in the specified pool that are compatible with your application and tests. For Remote Access, you select the desired device based on make, model, carrier variant, and operating system version. You can then optionally upload apps and other data as well as configure other device settings. Device Farm then locates an available device matching your request and displays the device’s display in your browser. You can then interact with the device and capture screenshots and video.

Yes, test devices will have a number of apps pre-installed by the device manufacturer or carrier.

Yes. All devices have a WiFi connection with Internet access. If your systems are internal (that is, behind a corporate firewall), you can allow the IP range 54.244.50.32-54.244.50.63. All device traffic will come from those IPs.

While you can't test actual carrier connections, you can simulate connection types and conditions using the network shaping functionality. When scheduling a run, you can select a curated network profile like "3G" or "Lossy LTE," or you can create your own, controlling parameters like throughput, jitter, and loss. All WiFi traffic from the device will be shaped and manipulated for the duration of your tests according to the profile you choose. You can also simulate dynamic environments by changing network parameters from your test scripts.

No, devices do not have carrier connections and cannot make phone calls or send SMS messages.

Yes, you can use the device cameras, both front- and rear-facing. Due to the way the devices are mounted, images and videos may look dark and blurry.

The built-in compatibility test suite allows you to install, uninstall, launch, and run Fuzz on the app.

Fuzz will perform fuzz testing on your UI immediately after launch. It streams random user input (touches, swipes, keyboard input) in a rapid fashion to your app. You can configure the number of events, the delay between events, and the seed used to randomize events. Using the same seed across test runs will result in the same sequence of events.

AWS Device Farm supports testing on Native, Hybrid, and Web apps for the Android and iOS platforms. The following frameworks are supported by default: Appium Java JUnit, Appium Java TestNG, Appium Python, Appium Ruby, Appium NodeJS, Instrumentation (Including JUnit, Espresso and most instrumentation-based tests), and XCTest (Including XCUI and KIF). For more information and a complete list of supported frameworks, visit our documentation.

You can run tests written in Appium Java JUnit, Appium Java TestNG, or Appium Python.

We’re always evaluating frameworks to support. Please contact us.

If you use one of the supported automation frameworks, you are in full control and can decide when to take screenshots. Those screenshots are included in your reports automatically.

Yes, Google Play Services is installed on devices that support it. The services are updated as new versions become available.

No, devices do not have an active Google account.

AWS Device Farm supports frameworks like Espresso and Robotium that have record and playback scripting tools.

No, AWS Device Farm will automatically replace a provisioning profile and resign your app so it can be deployed on our devices.

No, but you can download the logs and symbolicate the stack traces locally.

Yes, if you use ProGuard. If you use DexGuard with anti-piracy measures, we are unable to re-sign the app and run tests against it.

Although devices have access to the Internet, we make no guarantee that ads will be displayed. We recommend that you remove ads from the builds tested on AWS Device Farm.

Yes. If you’re using a client-server framework like Calabash, Appium, or UI Automation, you can access the Internet and execute limited shell commands from the host.

You can provide a .zip archive up to 4 GB in size. On Android, it will be extracted to the root of external memory; on iOS, to your app’s sandbox. For Android expansion files (OBB), we will automatically place the file into the location appropriate to the OS version. For more information, see the Developer Guide.

Yes, you can select multiple apps and the order in which to install them. These dependent apps will be installed before your tests begin.

Yes, in order to test your upgrade flow, you can upload and install an old version of your app before the new version is installed and tested.

Yes, you can supply latitude and longitude coordinates that will be used to override a device’s GPS.

Yes, you can provide a locale (for example, “en_US”) to override the default locale setting on a device.

Tests are immediately queued for execution and usually start within minutes. If one or more devices are not available, test execution for those devices will remain queued until they become available. Testing on the other devices in your test run will continue.

The maximum time allowed is 150 minutes. 

Yes. We have a plug-in for the Jenkins continuous integration environment and a Gradle plugin compatible with Android Studio. AWS Device Farm also provides programmatic support for all console features, including setting up a test and downloading test results through an API. For more information, see the AWS Device Farm API Reference. In addition to the API, you can access AWS Device Farm from the AWS SDKs.

AWS Device Farm test reports contain pass/fail information, crash reports, test logs, device logs, screenshots, videos, and performance data. Reports include both detailed per-device data and high-level results like the number of occurrences of a given error. Remote Access results contain logs and a video of the session.

AWS Device Farm reports include complete logcat (Android) and device logs (iOS), as well as logs from the device host and specified test framework.

If you write data to logcat (Android) or the device log (iOS), those log entries will be included in the report. AWS Device Farm does not collect any non-standard logs or other artifacts, although you may transfer files via your test script using the device's or device host's Internet connection.

Pricing is based on device minutes, which are determined by the duration of tests on each selected device. AWS Device Farm comes with a free trial of 1000 device minutes.* After that, customers are charged $0.17 per device minute. As your testing needs grow, you can opt for an unmetered testing plan, which allows unlimited testing for a flat monthly fee of $250 per device.

Your first 1000 device minutes are provided free of charge. This is a one-time trial and does not renew. Once your trial allocation is depleted, you will be billed at the standard rate of $0.17 per device minute.

A device minute is the billing unit. Device minutes are a measurement of the time it takes (in minutes) to install, execute, and uninstall your app and tests on every device you have selected for your test run. The unit price is constant regardless of the device, test, or application type. Device minutes are only billed for tests that complete without any device or system errors. Similarly, for Remote Access sessions, device minutes are measured from the time it takes to prepare a device to your specification to completely removing any apps and data you placed on the device.

You can dynamically grow and shrink your usage according to your business needs without any upper limits or up-front commitments.

Your first 1000 device minutes are provided free of charge.* This is a one-time trial and does not renew. Once your trial allocation is depleted, you will be billed at the standard rate of $0.17 per device minute.

Unmetered plans allow unlimited testing and remote access starting at $250 per month. Unmetered pricing is based on the number of device slots you purchase for each usage type (i.e. automated test or remote access) and device family (i.e. Android or iOS) and are priced at $250 per slot per month. Device slots correspond to concurrency.

For instance, if you purchase ten automated test Android device slots and schedule a run on 100 Android devices, Device Farm will execute your tests on up to ten devices at a time until all tests are completed on your selected devices. Purchasing more slots would enable you to get your results faster. Regardless of how many tests or remote access sessions you have in a month, you are billed at the flat rate of $250 per device slot per month. You can cancel your subscription for one or more device slots at any time and the cancellation will take effect at your next renewal date (the day of the month that you purchased your first active device slot). You can purchase slots from the AWS Device Farm console, AWS Command Line Interface (AWS CLI), or AWS Device Farm API. To learn more see our docs.

You can add device slots at any time and they will be available to you immediately. You can also cancel your subscription for one or more device slots at any time and the cancellation will take effect at your next renewal date (the day of the month that you purchased your first active device slot).

Yes. When creating a run, you can choose to make use of your unmetered device slots or use metered device minutes instead. Because concurrency is not limited on metered billing, this gives you the flexibility of running tests faster than would otherwise happen using your device slots.

A private device is a physical instance of a phone or tablet that is exclusive to your account. Private devices can have custom, static configurations and run custom OS images. Each device is deployed on your behalf and removed at the end of your subscription.

Each private device under your account is considered a private device subscription. The monthly subscription price is tiered on the cost of the device and starts at $200/month. After the minimum subscription period, you can choose to cancel your subscription at any time. Please contact us for more information.

Yes. When selecting devices for a test run or remote access session you will see your private devices as well as public devices. You can also create device pools comprised of both private and public devices. For more information about private devices, please contact us.

Selenium is an open-source framework that automates web browser interaction. You can learn more about Selenium here.

Device Farm enables you to execute your Selenium tests on different desktop browsers and browser versions that are hosted in the AWS Cloud. Device Farm follows a client-side execution model for Selenium testing i.e., your tests execute on your own local machine but will interact with browsers hosted on AWS Device Farm through the Selenium API.

To get started, please see our getting started guide here.

All browsers are hosted on EC2 Windows instances which run on Microsoft Windows Server.

You can see the list of desktop browsers and browser versions supported here.

You can see the list of Selenium desired capabilities that Device Farm supports here.

Device Farm generates console logs, web driver logs, action logs, and video recordings of the entire test to help you troubleshoot test failures.

Yes. Device Farm supports testing web apps on real mobile devices using Appium. Visit our developer guide for Appium Web Testing to learn more. Please note that for testing on real mobile devices, Device Farm follows a server-side execution model and hence you will need to upload your tests to the service.

You can find all limits for this feature here.

Pricing is based on instance minutes, which are determined by the duration of tests on each selected browser instance. You will be charged $0.005 per browser instance minute.

An instance minute is the billing unit for Desktop Browser Testing on Device Farm. Instance minutes are a measurement of the time it takes (in minutes) to execute your tests on every browser instance you have selected for your test run. The unit price of $0.005 is constant regardless of the browser or browser version you have selected. We do not charge you for the time it takes to launch, initialize, or teardown the EC2 instance that host the browser.