Android CTS, VTS, and Play Protect as a BSP release gate

Android’s compatibility machinery exists because the platform is a contract. CTS asks whether the API surface and many behaviors still match what application authors were told. VTS asks whether the vendor interface still matches what the framework was built against. Play Protect is a different kind of gate: it is about whether Google will treat the image as a device that may carry Play, not about whether your demo activity launched.

On a phone-shaped reference board, people remember these suites. On a product that is not a phone — glasses, a panel, a living-room device — they still apply if you need the contract. Forgetting them until the week before a factory build is how a BSP program becomes a fire. This note is about putting them on the calendar as a release gate.

Name the contract you actually need

Not every Android image needs Play. Some products are AOSP-shaped and will never see a Play store. Say that in the first week. If you do need Play, Play Protect and the compatibility bar are not optional seasoning. If you do not, CTS and VTS may still be the cheapest way to know you have not broken the HAL contract you will need for the next framework drop.

Write the target Android version, the GMS or AOSP decision, and the hardware SKU the suites will run on. A pass on a development board is evidence about the development board. A pass on the product is evidence about the product. If they differ in memory, sensors, or a camera path, they are different devices for the purpose of the gate.

What each gate is for

CTS is large and uneven. Some failures are your HAL. Some are a permission you thought was a product feature. Some are a timeout on hardware that is slower than a phone. The useful habit is to classify: product waiver with a written reason, framework configuration you can change, vendor code you must fix, or a test that does not apply and must be documented as such. An unclassified fail is not a status. It is a debt.

VTS is the vendor-interface conversation. If you treat Treble-era boundaries as optional because “it boots,” you will pay when you want a framework update without a full vendor rewrite. Run VTS on the same cadence as CTS once the vendor interface is supposed to be stable. A VTS fail after you declared the interface frozen is a process fail as much as a code fail.

Play Protect is not a unit test you run in an afternoon and forget. It is a judgment about the image and the device identity. Plan the account, the build provenance, and the time. Do not schedule it as a Friday task after the factory has cut a date. If you cannot say who submits and who owns a rejection, you do not have a gate. You have a hope.

Put the suites on a train

A release train needs a freeze, a build identifier, a suite version, and a place the results live. Run a subset continuously if you can — camera, media, graphics, whatever your product actually is — and run the full CTS/VTS you committed to on the release candidate. Continuous subset is how you find the regression in the week it landed. Full suite is how you can say the candidate is the candidate.

Own the lab software, not a fantasy lab. You need the product hardware, power, and enough seats to finish the suite before the date. You need images that are the images, not a cousin with debug unlocked and a different kernel command line. If a test needs a harness, write the harness down. If a test needs a person in the room, write that down too. Hidden manual steps are how a “green” run becomes unreproducible.

Waivers are product documents

Some tests will never pass on a device that has no telephony, or a device whose camera is a presence sensor. A waiver is a sentence: which test, why it does not apply, who agreed, and whether a future SKU would change the answer. Store waivers next to the results. Do not store them in a chat.

A waiver that means “we will fix it after we ship” is not a waiver. It is a defect with a date. Treat it as one. Compatibility debt after a first factory build is more expensive than the same fail in bring-up, because the image now has a name in the field.

Unusual form factors

Glasses, panels, and other non-phone Android products fail in patterned ways: input, sensors, camera, power, display density, and assumptions about a telephony stack. The suites were written with a phone in the room. Your job is not to pretend the product is a phone. Your job is to make the vendor and framework contracts true for what the product is, and to document the rest.

The principal has taken an AR/VR smart-glasses BSP through CTS, VTS, and Play Protect. That program is anonymized on the Work pages. It is not an ASHNAY shipment. The lesson that belongs in public is the calendar: the suites were a release activity, not a certificate you buy at the end.

Handoff

When the image is “done,” another engineer should be able to name the build, reproduce the suite versions, read the fails and waivers, and know whether Play Protect was in scope. If that paragraph cannot be written, the BSP is not handed off. It is parked.

ASHNAY SOFTWARE will take Android BSP work as software on hardware you supply. We do not run a certification house, and we do not design the silicon. We will write the image work and run the suites you name on the target you provide.