Selected Papers

To identify papers that perform beyond-GUI exploration, we started from existing surveys on testing approaches for mobile software [1,2,3]. In addition to GUI-based exploration techniques, these surveys include techniques related to application functional testing, testing for concurrency, security, performance, and energy consumption, which typically involve alterations to the application or the ecosystem it interacts with, i.e., the framework, the device or external components like servers. We excluded approaches which (a) require additional human input to guide the exploration, such as bug reports or pre-computed app models, and (b) only applicable to a specific goal, e.g., thread manipulation for data race detection, which would not map to generic app exploration.

Next, we performed forward snowballing, to augment our list with papers published after the surveys’ timeline, using the same inclusion/exclusion criteria. At the end of this process, we identified 22 relevant papers listed below. We characterized the app exploration approach along four dimensions:

Invocation Strategy: mechanisms tools employ to launch any given activity and trigger activation locations, notably:

• Direct Activity Invocation: sending intents to directly jump start an activity through utilities like ADB. To trigger even non-exported activities, tools can modify the Manifest file to expose all activities.
• Callback Code Injection: directly invoking callbacks through static or dynamic instrumentation.
• Framework Event Generation: customizing the Android OS to inject events at the framework layer, i.e., invoke the same methods within the framework that are triggered by the hardware layer.
• Code Extraction: generating a new application containing only one activity and none of the original application entry points, to target a given statement, e.g., startActivity.

Navigation Strategy: mechanisms tools employ to navigate past activation guards, given that the enclosing location is triggered. We identify four main implementation paradigms:

• Forced Branching: modifying control flows of a program to steer execution towards a desired point of interest.
• Variable Manipulation: manipulating code variables, to ensure execution takes the desired path, by intercepting Android framework APIs and/or arbitrary variables.
• Device Manipulation: interacting with configurable and/or stateful aspects of devices without modifying code (e.g., time, location, sensor readings).
• Server Simulation: monitoring network traffic to simulate server interactions accurately (in specific use cases).

Value Generation Strategy provides inputs necessary for invoking callbacks and activities, and for generating data to steer navigation. We identified three kinds of strategies:

• Fixed: using random or default/manually-crafted values
• Heuristic: integrating app- or domain-specific information (e.g, using natural language processing)
• Symbolic: using a constraint solver to generate semantically meaningful values

Value Generation Type corresponds to the different types of data that can be generated for a given strategy, notably Generic Primitives, Generic Objects and Specialized.

For completeness, we also mark which of the selected papers perform GUI Exploration by sending GUI and/or System Events (in addition to their main exploration strategy). Additional tool-specific information, notably availability, targeting strategy, invasiveness and date published is in an Excel sheet.

[1] Kong, Pingfan, et al. "Automated testing of android apps: A systematic literature review." IEEE Transactions on Reliability, 2018.

[2] Luo, Chu, et al. "A survey of context simulation for testing mobile context-aware applications." ACM Computing Surveys (CSUR), 2020.

[3] Júnior, Misael C., et al. "Dynamic testing techniques of non-functional requirements in mobile apps: A systematic mapping study." ACM Computing Surveys (CSUR), 2022.