AppForge package

Submodules

AppForge.appforge module

class AppForge.appforge.AppForge(runs: str, base_folder: Path = WindowsPath('runs'), use_docker: bool = False, emulator_id: str | None = None, sdk_path: Path | None = None, bench_folder: Path | None = None, docker_name: str = 'zenithfocuslight/appforge:latest', docker_port: int = 6080)[source]

Bases: object

A class for building, testing, and fuzzing Android applications. This class provides functionality to compile Android apps from templates, run tests, and perform fuzzing operations either using local emulator or Docker.

docker_base_folder = WindowsPath('/AppDev-Bench/AppDev-Bench/runs')
docker_bench_folder = WindowsPath('/AppDev-Bench/AppDev-Bench')
template_folder = WindowsPath('compiler/templates')
task_num = 101
__init__(runs: str, base_folder: Path = WindowsPath('runs'), use_docker: bool = False, emulator_id: str | None = None, sdk_path: Path | None = None, bench_folder: Path | None = None, docker_name: str = 'zenithfocuslight/appforge:latest', docker_port: int = 6080)[source]

Initialize the AppForge instance.

Parameters:

  • runs (str) – Name identifier for the current run session. Will build a same name folder under base_folder for storing data.

  • base_folder (Path) – Base directory for storing run data. Defaults to ‘/runs’.

  • use_docker (bool) – Whether to use Docker for operations. Defaults to False.

  • emulator_id (Optional[str]) – ID of the emulator to use. Required if not using Docker.

  • sdk_path (Optional[Path]) – Path to Android SDK. Required if not using Docker.

  • bench_folder (Optional[Path]) – Path to benchmark folder. Required if not using Docker.

  • docker_name (str) – Docker image name to use. Defaults to ‘zenithfocuslight/appforge:latest’.

  • docker_port (int) – Port to expose from Docker container. Defaults to 6080.

clean_up()[source]

Clean up resources and stop Docker container if used.

description(task_id: int)[source]

Get description of a specific task.

Parameters:

task_id (int) – ID (0~100) of the task to describe.

Returns:

Dictionary containing task description with keys:

  • task: App name

  • features: Refined feature descriptions

  • gradle_version: Gradle version used

  • api_version: Android API version

  • device: Target device name

Return type:

dict

task_name(task_id: int)[source]
apk_folder(task_id)[source]
compile_log(task_id)[source]
test_log(task_id)[source]
fuzz_log(task_id)[source]
json_file(task_id)[source]
result_path(task_id)[source]
fuzz_result_path(task_id)[source]
direct_apk_path(task_id)[source]
docker_apk_folder(task_id)[source]
docker_json_file(task_id)[source]
docker_direct_apk_path(task_id)[source]
ensure_emulator()[source]

Ensure that the emulator is online and accessible.

compile_json_based_on_template(changed: dict[str, str], task_id: int)[source]

Apply changes from JSON on template and compile the application.

Parameters:

  • changed (dict[str, str]) – Dictionary containing changes to apply.

  • task_id (int) – ID of the task to compile.

Returns:

Compilation errors if any, empty string if successful.

Return type:

str

compile_folder(folder: Path, task_id: int)[source]

Copy files from target folder and compile the application.

Parameters:

  • folder (Path) – Source folder containing files to compile.

  • task_id (int) – ID of the task to compile.

Returns:

Compilation errors if any, empty string if successful.

Return type:

str

test(task_id: int)[source]

Run test cases on the specified task.

Parameters:

task_id (int) – ID of the task to test.

Returns:

Test results containing:
  • compile (int): 1 if compilation successful, 0 otherwise;

    if not successfully compiled, other value is set to be 0

  • test (float): Test pass rate between 0.0 and 1.0

  • all_pass (int): 1 if all tests passed, 0 otherwise

Return type:

dict

fuzz(task_id: int)[source]

Run 10-minute fuzzing on the specified task.

Parameters:

task_id (int) – ID of the task to fuzz.

Returns:

Fuzzing results containing:
  • compile (int): 1 if compilation successful, 0 otherwise;

    if not successfully compiled, other value is set to be 0

  • no_crash (int): 1 if no crashes detected, 0 otherwise

  • native (int): 1 if native crash detected, 0 otherwise

  • java (int): 1 if Java crash detected, 0 otherwise

  • anr (int): 1 if ANR detected, 0 otherwise

  • failtostart (int): 1 if app failed to start, 0 otherwise

Return type:

dict

evaluation_only_test(eval_list: list | None = None)[source]

Run test cases on specified tasks or all tasks.

Parameters:

eval_list (Optional[list]) – List of task IDs to evaluate. If None, evaluates all tasks.

Returns:

Aggregated test results for all evaluated tasks.

Return type:

dict

evaluation(eval_list: list | None = None)[source]

Run test cases and fuzzing on specified tasks or all tasks.

Parameters:

eval_list (Optional[list]) – List of task IDs to evaluate. If None, evaluates all tasks.

Returns:

Aggregated test and fuzzing results for all evaluated tasks. In addition, we calculate ‘crash_rate’ which stands for the crash rate on successfully compiled apks.

Return type:

dict

AppForge.extracts module

AppForge.extracts.extract_error(log: str, ignore_path_str: str | None = None)[source]

Extract compilation errors from build log output.

Parameters:

  • log (str) – The complete build log output as a string.

  • ignore_path_str (Optional[str]) – Path string to remove from error messages for cleaner output. If None, paths are preserved.

Returns:

Extracted error messages as a single string, or None if

no errors were found. Returns empty string if errors were detected but no specific error messages were captured.

Return type:

Optional[str]

AppForge.extracts.extract_fuzz(log: str)[source]

Extract fuzzing results from fuzzing log output.

This function analyzes the fuzzing log to detect various types of crashes and failures that occurred during the fuzzing process, including native crashes, Java crashes, ANRs (Application Not Responding), and startup failures.

Parameters:

log (str) – The complete fuzzing log output as a string.

Returns:

A dictionary containing fuzzing results with the following keys:

  • no_crash (int): 1 if no crashes detected in any cycle, 0 otherwise

  • native (int): 1 if native crash detected, 0 otherwise

  • java (int): 1 if Java crash detected, 0 otherwise

  • anr (int): 1 if ANR detected, 0 otherwise

  • failtostart (int): 1 if app failed to start, 0 otherwise

Return type:

dict

AppForge.extracts.extract_test(log: str)[source]

Extract test execution results from test log output.

This function parses the test log to determine compilation success and calculate test pass rate based on success/failure counts in the log.

Parameters:

log (str) – The complete test log output as a string.

Returns:

A dictionary containing test results with the following keys:
  • compile (int): 1 if compilation was successful (tests executed),

    0 if no tests were executed

  • test (float): Test pass rate as a float between 0.0 and 1.0,

    representing the ratio of passed tests

  • all_pass (int): 1 if all tests passed, 0 otherwise

Return type:

dict

AppForge.utils module

AppForge.utils.sumup_json(results: list[Dict])[source]

Calculate average values across multiple result dictionaries.

This function takes a dictionary of task results and computes the average value for each metric across all tasks. The input should be structured with task IDs as keys and dictionaries of metrics as values.

Parameters:

results (Dict[int, Dict]) – A dictionary where keys are task IDs and values are dictionaries containing metric names and their values.

Returns:

A dictionary with metric names as keys and their

average values across all tasks as values.

Return type:

Dict[str, float]

AppForge.utils.remove_directory(path: Path)[source]

Recursively remove a directory and all its contents.

This function safely deletes a directory by first removing all files and subdirectories within it, then removing the directory itself. It handles nested directory structures recursively.

Parameters:

path (Path) – The path to the directory to be removed.

AppForge.utils.compare_folder(folderA: Path, folderB: Path)[source]

Compare two folders and identify files that differ or are new.

This function recursively compares the contents of two folders and returns a dictionary of files that are either new in folderA or have different content compared to folderB. Only text files that can be read as UTF-8 are considered in the comparison.

Parameters:

  • folderA (Path) – The source folder to compare (typically newer version).

  • folderB (Path) – The target folder to compare against (typically baseline).

Returns:

A dictionary where keys are relative file paths (relative

to folderA) and values are the file contents from folderA. The dictionary includes files that exist in folderA but not in folderB, or files that have different content between the two folders.

Return type:

Dict[str, str]

Module contents