An {@code AutofillService} is only bound to the Android System for autofill purposes if: *
The basic autofill process is defined by the workflow below: *
This workflow was designed to minimize the time the Android System is bound to the service; * for each call, it: binds to service, waits for the reply, and unbinds right away. Furthermore, * those calls are considered stateless: if the service needs to keep state between calls, it must * do its own state management (keeping in mind that the service's process might be killed by the * Android System when unbound; for example, if the device is running low in memory). * *
Typically, the * {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} will: *
For example, for a login screen with username and password views where the user only has one * account in the service, the response could be: * *
* new FillResponse.Builder()
* .addDataset(new Dataset.Builder()
* .setValue(id1, AutofillValue.forText("homer"), createPresentation("homer"))
* .setValue(id2, AutofillValue.forText("D'OH!"), createPresentation("password for homer"))
* .build())
* .build();
*
*
* But if the user had 2 accounts instead, the response could be: * *
* new FillResponse.Builder()
* .addDataset(new Dataset.Builder()
* .setValue(id1, AutofillValue.forText("homer"), createPresentation("homer"))
* .setValue(id2, AutofillValue.forText("D'OH!"), createPresentation("password for homer"))
* .build())
* .addDataset(new Dataset.Builder()
* .setValue(id1, AutofillValue.forText("flanders"), createPresentation("flanders"))
* .setValue(id2, AutofillValue.forText("OkelyDokelyDo"), createPresentation("password for flanders"))
* .build())
* .build();
*
*
* If the service does not find any autofillable view in the view structure, it should pass * {@code null} to {@link FillCallback#onSuccess(FillResponse)}; if the service encountered an error * processing the request, it should call {@link FillCallback#onFailure(CharSequence)}. For * performance reasons, it's paramount that the service calls either * {@link FillCallback#onSuccess(FillResponse)} or {@link FillCallback#onFailure(CharSequence)} for * each {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} received - if it * doesn't, the request will eventually time out and be discarded by the Android System. * * *
If the service is also interested on saving the data filled by the user, it must set a * {@link SaveInfo} object in the {@link FillResponse}. See {@link SaveInfo} for more details and * examples. * * *
The service can provide an extra degree of security by requiring the user to authenticate * before an app can be autofilled. The authentication is typically required in 2 scenarios: *
When using authentication, it is recommended to encrypt only the sensitive data and leave * labels unencrypted, so they can be used on presentation views. For example, if the user has a * home and a work address, the {@code Home} and {@code Work} labels should be stored unencrypted * (since they don't have any sensitive data) while the address data per se could be stored in an * encrypted storage. Then when the user chooses the {@code Home} dataset, the platform starts * the authentication flow, and the service can decrypt the sensitive data. * *
The authentication mechanism can also be used in scenarios where the service needs multiple * steps to determine the datasets that can fill a screen. For example, when autofilling a financial * app where the user has accounts for multiple banks, the workflow could be: * *
Another example of multiple-steps dataset selection is when the service stores the user * credentials in "vaults": the first response would contain fake datasets with the vault names, * and the subsequent response would contain the app credentials stored in that vault. * * *
The autofillable views in a screen should be grouped in logical groups called "partitions". * Typical partitions are: *
For security reasons, when a screen has more than one partition, it's paramount that the * contents of a dataset do not spawn multiple partitions, specially when one of the partitions * contains data that is not specific to the application being autofilled. For example, a dataset * should not contain fields for username, password, and credit card information. The reason for * this rule is that a malicious app could draft a view structure where the credit card fields * are not visible, so when the user selects a dataset from the username UI, the credit card info is * released to the application without the user knowledge. Similarly, it's recommended to always * protect a dataset that contains sensitive information by requiring dataset authentication * (see {@link Dataset.Builder#setAuthentication(android.content.IntentSender)}), and to include * info about the "primary" field of the partition in the custom presentation for "secondary" * fields—that would prevent a malicious app from getting the "primary" fields without the * user realizing they're being released (for example, a malicious app could have fields for a * credit card number, verification code, and expiration date crafted in a way that just the latter * is visible; by explicitly indicating the expiration date is related to a given credit card * number, the service would be providing a visual clue for the users to check what would be * released upon selecting that field). * *
When the service detects that a screen has multiple partitions, it should return a * {@link FillResponse} with just the datasets for the partition that originated the request (i.e., * the partition that has the {@link android.app.assist.AssistStructure.ViewNode} whose * {@link android.app.assist.AssistStructure.ViewNode#isFocused()} returns {@code true}); then if * the user selects a field from a different partition, the Android System will make another * {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} call for that partition, * and so on. * *
Notice that when the user autofill a partition with the data provided by the service and the * user did not change these fields, the autofilled value is sent back to the service in the * subsequent calls (and can be obtained by calling * {@link android.app.assist.AssistStructure.ViewNode#getAutofillValue()}). This is useful in the * cases where the service must create datasets for a partition based on the choice made in a * previous partition. For example, the 1st response for a screen that have credentials and address * partitions could be: * *
* new FillResponse.Builder()
* .addDataset(new Dataset.Builder() // partition 1 (credentials)
* .setValue(id1, AutofillValue.forText("homer"), createPresentation("homer"))
* .setValue(id2, AutofillValue.forText("D'OH!"), createPresentation("password for homer"))
* .build())
* .addDataset(new Dataset.Builder() // partition 1 (credentials)
* .setValue(id1, AutofillValue.forText("flanders"), createPresentation("flanders"))
* .setValue(id2, AutofillValue.forText("OkelyDokelyDo"), createPresentation("password for flanders"))
* .build())
* .setSaveInfo(new SaveInfo.Builder(SaveInfo.SAVE_DATA_TYPE_PASSWORD,
* new AutofillId[] { id1, id2 })
* .build())
* .build();
*
*
* Then if the user selected {@code flanders}, the service would get a new * {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} call, with the values of * the fields {@code id1} and {@code id2} prepopulated, so the service could then fetch the address * for the Flanders account and return the following {@link FillResponse} for the address partition: * *
* new FillResponse.Builder()
* .addDataset(new Dataset.Builder() // partition 2 (address)
* .setValue(id3, AutofillValue.forText("744 Evergreen Terrace"), createPresentation("744 Evergreen Terrace")) // street
* .setValue(id4, AutofillValue.forText("Springfield"), createPresentation("Springfield")) // city
* .build())
* .setSaveInfo(new SaveInfo.Builder(SaveInfo.SAVE_DATA_TYPE_PASSWORD | SaveInfo.SAVE_DATA_TYPE_ADDRESS,
* new AutofillId[] { id1, id2 }) // username and password
* .setOptionalIds(new AutofillId[] { id3, id4 }) // state and zipcode
* .build())
* .build();
*
*
* When the service returns multiple {@link FillResponse}, the last one overrides the previous; * that's why the {@link SaveInfo} in the 2nd request above has the info for both partitions. * * *
When autofilling app-specific data (like username and password), the service must verify * the authenticity of the request by obtaining all signing certificates of the app being * autofilled, and only fulfilling the request when they match the values that were * obtained when the data was first saved — such verification is necessary to avoid phishing * attempts by apps that were sideloaded in the device with the same package name of another app. * Here's an example on how to achieve that by hashing the signing certificates: * *
* private String getCertificatesHash(String packageName) throws Exception {
* PackageManager pm = mContext.getPackageManager();
* PackageInfo info = pm.getPackageInfo(packageName, PackageManager.GET_SIGNATURES);
* ArrayList hashes = new ArrayList<>(info.signatures.length);
* for (Signature sig : info.signatures) {
* byte[] cert = sig.toByteArray();
* MessageDigest md = MessageDigest.getInstance("SHA-256");
* md.update(cert);
* hashes.add(toHexString(md.digest()));
* }
* Collections.sort(hashes);
* StringBuilder hash = new StringBuilder();
* for (int i = 0; i < hashes.size(); i++) {
* hash.append(hashes.get(i));
* }
* return hash.toString();
* }
*
*
* If the service did not store the signing certificates data the first time the data was saved * — for example, because the data was created by a previous version of the app that did not * use the Autofill Framework — the service should warn the user that the authenticity of the * app cannot be confirmed (see an example on how to show such warning in the * Web security section below), and if the user agrees, * then the service could save the data from the signing ceriticates for future use. * * *
If the service find views that cannot be autofilled (for example, a text field representing * the response to a Captcha challenge), it should mark those views as ignored by * calling {@link FillResponse.Builder#setIgnoredIds(AutofillId...)} so the system does not trigger * a new {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} when these views are * focused. * * *
When handling autofill requests that represent web pages (typically * view structures whose root's {@link android.app.assist.AssistStructure.ViewNode#getClassName()} * is a {@link android.webkit.WebView}), the service should take the following steps to verify if * the structure can be autofilled with the data associated with the app requesting it: * *
Here's an example on how to get the canonical domain using * Guava: * *
* private static String getCanonicalDomain(String domain) {
* InternetDomainName idn = InternetDomainName.from(domain);
* while (idn != null && !idn.isTopPrivateDomain()) {
* idn = idn.parent();
* }
* return idn == null ? null : idn.toString();
* }
*
*
*
* If the association between the web domain and app package cannot be verified through the steps * above, but the service thinks that it is appropriate to fill persisted credentials that are * stored for the web domain, the service should warn the user about the potential data * leakage first, and ask for the user to confirm. For example, the service could: * *
This same procedure could also be used when the autofillable data is contained inside an * {@code IFRAME}, in which case the WebView generates a new autofill context when a node inside * the {@code IFRAME} is focused, with the root node containing the {@code IFRAME}'s {@code src} * attribute on {@link android.app.assist.AssistStructure.ViewNode#getWebDomain()}. A typical and * legitimate use case for this scenario is a financial app that allows the user * to login on different bank accounts. For example, a financial app {@code my_financial_app} could * use a WebView that loads contents from {@code banklogin.my_financial_app.com}, which contains an * {@code IFRAME} node whose {@code src} attribute is {@code login.some_bank.com}. When fulfilling * that request, the service could add an * {@link Dataset.Builder#setAuthentication(android.content.IntentSender) authenticated dataset} * whose presentation displays "Username for some_bank.com" and * "Password for some_bank.com". Then when the user taps one of these options, the service * shows the disclaimer dialog explaining that selecting that option would release the * {@code login.some_bank.com} credentials to the {@code my_financial_app}; if the user agrees, * then the service returns an unlocked dataset with the {@code some_bank.com} credentials. * *
Note: The autofill service could also add well-known browser apps into an allowlist and * skip the verifications above, as long as the service can verify the authenticity of the browser * app by checking its signing certificate. * * *
It's tricky to handle save for autofill in these situations, because the autofill service must * wait until the user enters both fields before the autofill save UI can be shown. But it can be * done by following the steps below: * *
For example, in an app that uses 2 steps for the username and password fields, the workflow * would be: *
* // On first fill request
* AutofillId usernameId = // parse from AssistStructure;
* Bundle clientState = new Bundle();
* clientState.putParcelable("usernameId", usernameId);
* fillCallback.onSuccess(
* new FillResponse.Builder()
* .setClientState(clientState)
* .setSaveInfo(new SaveInfo
* .Builder(SaveInfo.SAVE_DATA_TYPE_USERNAME, new AutofillId[] {usernameId})
* .build())
* .build());
*
* // On second fill request
* Bundle clientState = fillRequest.getClientState();
* AutofillId usernameId = clientState.getParcelable("usernameId");
* AutofillId passwordId = // parse from AssistStructure
* clientState.putParcelable("passwordId", passwordId);
* fillCallback.onSuccess(
* new FillResponse.Builder()
* .setClientState(clientState)
* .setSaveInfo(new SaveInfo
* .Builder(SaveInfo.SAVE_DATA_TYPE_USERNAME | SaveInfo.SAVE_DATA_TYPE_PASSWORD,
* new AutofillId[] {usernameId, passwordId})
* .setFlags(SaveInfo.FLAG_SAVE_ON_ALL_VIEWS_INVISIBLE)
* .build())
* .build());
*
* // On save request
* Bundle clientState = saveRequest.getClientState();
* AutofillId usernameId = clientState.getParcelable("usernameId");
* AutofillId passwordId = clientState.getParcelable("passwordId");
* List fillContexts = saveRequest.getFillContexts();
*
* FillContext usernameContext = fillContexts.get(0);
* ViewNode usernameNode = findNodeByAutofillId(usernameContext.getStructure(), usernameId);
* AutofillValue username = usernameNode.getAutofillValue().getTextValue().toString();
*
* FillContext passwordContext = fillContexts.get(1);
* ViewNode passwordNode = findNodeByAutofillId(passwordContext.getStructure(), passwordId);
* AutofillValue password = passwordNode.getAutofillValue().getTextValue().toString();
*
* save(username, password);
*
*
*
* The {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} method is called * without the user content. The Android system strips some properties of the * {@link android.app.assist.AssistStructure.ViewNode view nodes} passed to this call, but not all * of them. For example, the data provided in the {@link android.view.ViewStructure.HtmlInfo} * objects set by {@link android.webkit.WebView} is never stripped out. * *
Because this data could contain PII (Personally Identifiable Information, such as username or * email address), the service should only use it locally (i.e., in the app's process) for * heuristics purposes, but it should not be sent to external servers. * * *
The service can call {@link #getFillEventHistory()} to get metrics representing the user * actions, and then use these metrics to improve its heuristics. * *
Prior to Android {@link android.os.Build.VERSION_CODES#P}, the metrics covered just the * scenarios where the service knew how to autofill an activity, but Android * {@link android.os.Build.VERSION_CODES#P} introduced a new mechanism called field classification, * which allows the service to dynamically classify the meaning of fields based on the existing user * data known by the service. * *
Typically, field classification can be used to detect fields that can be autofilled with * user data that is not associated with a specific app—such as email and physical * address. Once the service identifies that a such field was manually filled by the user, the * service could use this signal to improve its heuristics on subsequent requests (for example, by * infering which resource ids are associated with known fields). * *
The field classification workflow involves 4 steps: * *
The field classification is an expensive operation and should be used carefully, otherwise it * can reach its rate limit and get blocked by the Android System. Ideally, it should be used just * in cases where the service could not determine how an activity can be autofilled, but it has a * strong suspicious that it could. For example, if an activity has four or more fields and one of * them is a list, chances are that these are address fields (like address, city, state, and * zip code). * * *
Apps that use standard Android widgets support autofill out-of-the-box and need to do * very little to improve their user experience (annotating autofillable views and providing * autofill hints). However, some apps (typically browsers) do their own rendering and the rendered * content may contain semantic structure that needs to be surfaced to the autofill framework. The * platform exposes APIs to achieve this, however it could take some time until these apps implement * autofill support. * *
To enable autofill for such apps the platform provides a compatibility mode in which the * platform would fall back to the accessibility APIs to generate the state reported to autofill * services and fill data. This mode needs to be explicitly requested for a given package up * to a specified max version code allowing clean migration path when the target app begins to * support autofill natively. Note that enabling compatibility may degrade performance for the * target package and should be used with caution. The platform supports creating an allowlist for * including which packages can be targeted in compatibility mode to ensure this mode is used only * when needed and as long as needed. * *
You can request compatibility mode for packages of interest in the meta-data resource * associated with your service. Below is a sample service declaration: * *
<service android:name=".MyAutofillService" * android:permission="android.permission.BIND_AUTOFILL_SERVICE"> * <intent-filter> * <action android:name="android.service.autofill.AutofillService" /> * </intent-filter> * <meta-data android:name="android.autofill" android:resource="@xml/autofillservice" /> * </service>* *
In the XML file you can specify one or more packages for which to enable compatibility * mode. Below is a sample meta-data declaration: * *
<autofill-service xmlns:android="http://schemas.android.com/apk/res/android"> * <compatibility-package android:name="foo.bar.baz" android:maxLongVersionCode="1000000000"/> * </autofill-service>* *
Notice that compatibility mode has limitations such as: *
<{@link
* android.R.styleable#AutofillService autofill-service}> tag.
* This is a a sample XML file configuring an AutoFillService:
* <autofill-service
* android:settingsActivity="foo.bar.SettingsActivity"
* . . .
* />
*/
public static final String SERVICE_META_DATA = "android.autofill";
/**
* Name of the {@link FillResponse} extra used to return a delayed fill response.
*
* Please see {@link FillRequest#getDelayedFillIntentSender()} on how to send a delayed * fill response to framework.
*/ public static final String EXTRA_FILL_RESPONSE = "android.service.autofill.extra.FILL_RESPONSE"; /** * Name of the {@link IResultReceiver} extra used to return the primary result of a request. * * @hide */ public static final String EXTRA_RESULT = "result"; /** * Name of the {@link IResultReceiver} extra used to return the error reason of a request. * * @hide */ public static final String EXTRA_ERROR = "error"; /** * Name of the key used to mark whether the fill response is for a webview. * * @hide */ public static final String WEBVIEW_REQUESTED_CREDENTIAL_KEY = "webview_requested_credential"; private final IAutoFillService mInterface = new IAutoFillService.Stub() { @Override public void onConnectedStateChanged(boolean connected) { mHandler.sendMessage(obtainMessage( connected ? AutofillService::onConnected : AutofillService::onDisconnected, AutofillService.this)); } @Override public void onFillRequest(FillRequest request, IFillCallback callback) { ICancellationSignal transport = CancellationSignal.createTransport(); try { callback.onCancellable(transport); } catch (RemoteException e) { e.rethrowFromSystemServer(); } mHandler.sendMessage(obtainMessage( AutofillService::onFillRequest, AutofillService.this, request, CancellationSignal.fromTransport(transport), new FillCallback(callback, request.getId()))); } @Override public void onConvertCredentialRequest( @NonNull ConvertCredentialRequest convertCredentialRequest, @NonNull IConvertCredentialCallback convertCredentialCallback) { mHandler.sendMessage(obtainMessage( AutofillService::onConvertCredentialRequest, AutofillService.this, convertCredentialRequest, new ConvertCredentialCallback(convertCredentialCallback))); } @Override public void onFillCredentialRequest(FillRequest request, IFillCallback callback, IBinder autofillClientCallback) { ICancellationSignal transport = CancellationSignal.createTransport(); try { callback.onCancellable(transport); } catch (RemoteException e) { e.rethrowFromSystemServer(); } mHandler.sendMessage(obtainMessage( AutofillService::onFillCredentialRequest, AutofillService.this, request, CancellationSignal.fromTransport(transport), new FillCallback(callback, request.getId()), autofillClientCallback)); } @Override public void onSaveRequest(SaveRequest request, ISaveCallback callback) { mHandler.sendMessage(obtainMessage( AutofillService::onSaveRequest, AutofillService.this, request, new SaveCallback(callback))); } @Override public void onSavedPasswordCountRequest(IResultReceiver receiver) { mHandler.sendMessage(obtainMessage( AutofillService::onSavedDatasetsInfoRequest, AutofillService.this, new SavedDatasetsInfoCallbackImpl(receiver, SavedDatasetsInfo.TYPE_PASSWORDS))); } }; private Handler mHandler; @CallSuper @Override public void onCreate() { super.onCreate(); mHandler = new Handler(Looper.getMainLooper(), null, true); BaseBundle.setShouldDefuse(true); } @Override public final IBinder onBind(Intent intent) { if (SERVICE_INTERFACE.equals(intent.getAction())) { return mInterface.asBinder(); } Log.w(TAG, "Tried to bind to wrong intent (should be " + SERVICE_INTERFACE + ": " + intent); return null; } /** * Called when the Android system connects to service. * *You should generally do initialization here rather than in {@link #onCreate}. */ public void onConnected() { } /** * Called by the Android system do decide if a screen can be autofilled by the service. * *
Service must call one of the {@link FillCallback} methods (like * {@link FillCallback#onSuccess(FillResponse)} * or {@link FillCallback#onFailure(CharSequence)}) * to notify the result of the request. * * @param request the {@link FillRequest request} to handle. * See {@link FillResponse} for examples of multiple-sections requests. * @param cancellationSignal signal for observing cancellation requests. The system will use * this to notify you that the fill result is no longer needed and you should stop * handling this fill request in order to save resources. * @param callback object used to notify the result of the request. */ public abstract void onFillRequest(@NonNull FillRequest request, @NonNull CancellationSignal cancellationSignal, @NonNull FillCallback callback); /** * Variant of onFillRequest for internal credential manager proxy autofill service only. * * @hide */ public void onFillCredentialRequest(@NonNull FillRequest request, @NonNull CancellationSignal cancellationSignal, @NonNull FillCallback callback, @NonNull IBinder autofillClientCallback) {} /** * Called by the Android system to convert a credential manager response to a dataset * * @param convertCredentialRequest the request that has the original credential manager response * @param convertCredentialCallback callback used to notify the result of the request. * * @hide */ public void onConvertCredentialRequest( @NonNull ConvertCredentialRequest convertCredentialRequest, @NonNull ConvertCredentialCallback convertCredentialCallback){} /** * Called when the user requests the service to save the contents of a screen. * *
If the service could not handle the request right away—for example, because it must * launch an activity asking the user to authenticate first or because the network is * down—the service could keep the {@link SaveRequest request} and reuse it later, * but the service must always call {@link SaveCallback#onSuccess()} or * {@link SaveCallback#onSuccess(android.content.IntentSender)} right away. * *
Note: To retrieve the actual value of fields input by the user, the service * should call * {@link android.app.assist.AssistStructure.ViewNode#getAutofillValue()}; if it calls * {@link android.app.assist.AssistStructure.ViewNode#getText()} or other methods, there is no * guarantee such method will return the most recent value of the field. * * @param request the {@link SaveRequest request} to handle. * See {@link FillResponse} for examples of multiple-sections requests. * @param callback object used to notify the result of the request. */ public abstract void onSaveRequest(@NonNull SaveRequest request, @NonNull SaveCallback callback); /** * Called from system settings to display information about the datasets the user saved to this * service. * *
There is no timeout for the request, but it's recommended to return the result within a * few seconds, or the user may navigate away from the activity that would display the result. * * @param callback callback for responding to the request */ public void onSavedDatasetsInfoRequest(@NonNull SavedDatasetsInfoCallback callback) { callback.onError(SavedDatasetsInfoCallback.ERROR_UNSUPPORTED); } /** * Called when the Android system disconnects from the service. * *
At this point this service may no longer be an active {@link AutofillService}. * It should not make calls on {@link AutofillManager} that requires the caller to be * the current service. */ public void onDisconnected() { } /** * Gets the events that happened after the last * {@link AutofillService#onFillRequest(FillRequest, android.os.CancellationSignal, FillCallback)} * call. * *
This method is typically used to keep track of previous user actions to optimize further * requests. For example, the service might return email addresses in alphabetical order by * default, but change that order based on the address the user picked on previous requests. * *
The history is not persisted over reboots, and it's cleared every time the service * replies to a {@link #onFillRequest(FillRequest, CancellationSignal, FillCallback)} by calling * {@link FillCallback#onSuccess(FillResponse)} or {@link FillCallback#onFailure(CharSequence)} * (if the service doesn't call any of these methods, the history will clear out after some * pre-defined time). Hence, the service should call {@link #getFillEventHistory()} before * finishing the {@link FillCallback}. * * @return The history or {@code null} if there are no events. * * @throws RuntimeException if the event history could not be retrieved. */ @Nullable public final FillEventHistory getFillEventHistory() { final AutofillManager afm = getSystemService(AutofillManager.class); if (afm == null) { return null; } else { return afm.getFillEventHistory(); } } }