Generating Baseline Profiles in CI

By Michael Samuel Naeem · September 30, 2024 · 5 min read

Baseline Profiles are one of the highest-return, lowest-glamour performance features on Android: ship a small rules file that tells the runtime which code to compile ahead of time, and cold start, first frame, and initial scroll all get faster with zero changes to your actual code. I've seen double-digit-percent improvements in cold start from nothing but a good profile. The catch is that a profile is only as good as it is current, and generating one by hand is exactly the kind of chore that quietly stops happening. The fix is to make CI generate it, so it's always fresh and never depends on someone remembering.

What a Baseline Profile does, concretely

When Android installs your app, most code isn't compiled to native — it runs interpreted or JIT-compiled, warming up over the first few runs. A Baseline Profile is a list of the hot methods and classes for your critical journeys (startup, first scroll), and the platform uses it to compile those paths ahead of time at install. The result: the runtime skips the JIT warm-up on exactly the paths users hit first. No behavioral change, just less interpretation on the critical path.

You generate the profile by exercising those paths — a Macrobenchmark test that launches the app and scrolls the home screen — while the tooling records which code ran. That recording becomes the profile shipped in your app.

Why CI, not a laptop

Generating locally works once and rots immediately. Three problems:

1. It drifts. Code changes; the profile captured three months ago no longer matches the hot paths.
2. It's inconsistent. Different developers, different devices, different results — the profile depends on which machine happened to run it.
3. It gets skipped. Manual steps before a release are the first thing dropped under deadline pressure.

Putting generation in CI solves all three: it runs on a defined device, on a schedule you control, producing a reproducible artifact. This is the same reliability argument as any generated build artifact — build it in the pipeline, don't hand-craft it.

Gradle Managed Devices: the key enabler

The thing that makes CI generation clean is Gradle Managed Devices (GMD). Instead of requiring a connected phone or a manually-booted emulator, you declare a device in Gradle and the build spins it up, runs against it, and tears it down. No adb wrangling, no flaky "is a device attached" checks.

// app/build.gradle.kts
android {
    testOptions {
        managedDevices {
            localDevices {
                create("pixel6Api34") {
                    device = "Pixel 6"
                    apiLevel = 34
                    systemImageSource = "aosp"
                }
            }
        }
    }
}

baselineProfile {
    managedDevices += "pixel6Api34"
    useConnectedDevices = false
}

Now the profile generator targets a device that exists only as configuration. Any CI runner with the SDK can produce identical results — the device isn't a physical dependency, it's a declaration.

The generation task

With the Baseline Profile Gradle plugin and a benchmark module wired up, generation is a single task:

./gradlew :app:generateBaselineProfile

Under the hood this builds a non-minified benchmark variant, runs your profile-generator Macrobenchmark on the managed device (launch, scroll the key screens), collects the hot paths, and writes the profile into src/<variant>/generated/baselineProfiles/. The generator itself is just a Macrobenchmark that drives the critical journey:

@Test
fun generate() = baselineProfileRule.collect(packageName = "com.example.app") {
    startActivityAndWait()
    val list = device.findObject(By.res("feed_list"))
    list.fling(Direction.DOWN)
    device.waitForIdle()
}

The quality of the profile is exactly the quality of this journey — profile the paths users actually hit first. A generator that only launches and does nothing gives you a startup profile but nothing for that first scroll jank.

Wiring it into the pipeline

A pragmatic setup that keeps profiles fresh without slowing every PR:

• On release branches / before release builds: run generateBaselineProfile, then build the release with the fresh profile bundled. This guarantees every shipped build has a current profile.
• Optionally, scheduled (nightly/weekly): regenerate and open a PR if the profile changed, so drift is caught continuously rather than only at release.
• Don't run full generation on every PR — it's slower than a normal test run because it boots an emulator and runs benchmarks. Gate it to where it pays off.

A CI step is just the Gradle task plus emulator acceleration:

- name: Generate Baseline Profile
  run: ./gradlew :app:generateBaselineProfile
- name: Build release with profile
  run: ./gradlew :app:assembleRelease

Verify the win, don't assume it

A profile you never measure is faith-based performance. Pair generation with a startup Macrobenchmark that measures cold start with and without the profile, so you have a number proving it helps and a regression guard if a future change erodes it. The measurement discipline here is the same one behind ProfileInstaller and startup performance — the profile is only half the story; ProfileInstaller ensuring it's applied is the other half, and benchmarks confirm both are actually working.

What I'd take away

Baseline Profiles buy real startup and scroll performance for free at runtime, but only if the profile stays current — and manual generation always rots. Move generation into CI using Gradle Managed Devices so it runs on a declared, reproducible emulator with no physical device dependency. Run generateBaselineProfile on release branches (and optionally on a schedule) rather than every PR, make the generator exercise the journeys users actually hit first, and back it with a startup Macrobenchmark that proves the improvement. Automated this way, the profile is always fresh, the win is measured, and nobody has to remember a manual step under deadline pressure.

Resources

• Android — Baseline Profiles overview
• Create Baseline Profiles
• Gradle Managed Devices
• Macrobenchmark