java - Google即时的自定义命令

标签 java android google-now google-voice google-voice-search

我正在尝试让谷歌现在接受自定义命令,并在进行特定查询时向我的应用程序发送一个意图。
我使用Tasker和AutoVoice成功地做到了这一点,但我希望在不使用这些应用程序的情况下也能做到这一点。
我在文档中发现了这一点。我在哪里能处理没有完成我的任务的共同意图呢?
我也尝试过谷歌提供的语音交互API,这几乎是相同的事情,但这没有帮助。
这里有没有人在不使用其他应用程序(如指挥官、自动语音或任务管理器)的情况下实现了这一点?

最佳答案

Google现在不接受自定义命令。您详细介绍的应用程序使用“hack”来截获语音命令,对于根设备,则使用“hack”。
他们要么对他们采取行动,同时杀死谷歌,要么无视他们,允许谷歌像往常一样显示结果。
出于许多原因,这是一个坏主意:
谷歌会找到一种方法来阻止这种类型的互动,如果它成为一个共同的地方,因为他们显然不希望他们现在的服务受到负面影响。
它使用硬编码常量,与谷歌用来显示语音命令的视图类相关。这当然会随着每次发布而变化。
黑客攻击!
免责声明完成!自担风险使用……
您需要在AccessibilityService中注册一个Manifest

    <service
        android:name="com.something.MyAccessibilityService"
        android:enabled="true"
        android:label="@string/label"
        android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" >
        <intent-filter>
            <action android:name="android.accessibilityservice.AccessibilityService" />
        </intent-filter>

        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/accessibilityconfig" />
    </service>

并将配置文件添加到res/xml
<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeWindowContentChanged"
    android:accessibilityFeedbackType="feedbackGeneric"
    android:accessibilityFlags="flagIncludeNotImportantViews"
    android:canRetrieveWindowContent="true"
    android:description="@string/accessibility_description"
    android:notificationTimeout="100"
    android:settingsActivity="SettingsActivity"/>

您可以选择添加:
    android:packageNames="xxxxxx"

或通过添加其他事件类型扩展功能:
    android:accessibilityEventTypes="typeViewTextSelectionChanged|typeWindowContentChanged|typeNotificationStateChanged"

包括以下AccessibilityService示例类:
/*
 * Copyright (c) 2016 Ben Randall
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *   http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package com.your.package;

import android.accessibilityservice.AccessibilityService;
import android.support.annotation.NonNull;
import android.support.annotation.Nullable;
import android.util.Log;
import android.view.accessibility.AccessibilityEvent;
import android.view.accessibility.AccessibilityNodeInfo;


/**
 * @author benrandall76 AT gmail DOT com
 */

public class MyAccessibilityService extends AccessibilityService {

    private final boolean DEBUG = true;
    private final String CLS_NAME = MyAccessibilityService.class.getSimpleName();

    private static final String GOOGLE_VOICE_SEARCH_PACKAGE_NAME = "com.google.android.googlequicksearchbox";
    private static final String GOOGLE_VOICE_SEARCH_INTERIM_FIELD = "com.google.android.apps.gsa.searchplate.widget.StreamingTextView";
    private static final String GOOGLE_VOICE_SEARCH_FINAL_FIELD = "com.google.android.apps.gsa.searchplate.SearchPlate";

    private static final long COMMAND_UPDATE_DELAY = 1000L;

    private long previousCommandTime;
    private String previousCommand = null;

    private final boolean EXTRA_VERBOSE = false;

    @Override
    protected void onServiceConnected() {
        super.onServiceConnected();
        if (DEBUG) {
            Log.i(CLS_NAME, "onServiceConnected");
        }
    }

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {
        if (DEBUG) {
            Log.i(CLS_NAME, "onAccessibilityEvent");
        }

        if (event != null) {

            switch (event.getEventType()) {

                case AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED:
                    if (DEBUG) {
                        Log.i(CLS_NAME, "onAccessibilityEvent: checking for google");
                    }

                    if (event.getPackageName() != null && event.getPackageName().toString().matches(
                            GOOGLE_VOICE_SEARCH_PACKAGE_NAME)) {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "onAccessibilityEvent: checking for google: true");
                            Log.i(CLS_NAME, "onAccessibilityEvent: event.getPackageName: " + event.getPackageName());
                            Log.i(CLS_NAME, "onAccessibilityEvent: event.getClassName: " + event.getClassName());
                        }

                        final AccessibilityNodeInfo source = event.getSource();

                        if (source != null && source.getClassName() != null) {

                            if (source.getClassName().toString().matches(
                                    GOOGLE_VOICE_SEARCH_INTERIM_FIELD)) {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: className interim: true");
                                    Log.i(CLS_NAME, "onAccessibilityEvent: source.getClassName: " + source.getClassName());
                                }

                                if (source.getText() != null) {

                                    final String text = source.getText().toString();
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: interim text: " + text);
                                    }

                                    if (interimMatch(text)) {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: child: interim match: true");
                                        }

                                        if (commandDelaySufficient(event.getEventTime())) {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: true");
                                            }

                                            if (!commandPreviousMatches(text)) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: false");
                                                }

                                                previousCommandTime = event.getEventTime();
                                                previousCommand = text;

                                                killGoogle();

                                                if (DEBUG) {
                                                    Log.e(CLS_NAME, "onAccessibilityEvent: INTERIM PROCESSING: " + text);
                                                }

                                            } else {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: true");
                                                }
                                            }
                                        } else {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: false");
                                            }
                                        }
                                        break;
                                    } else {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: child: interim match: false");
                                        }
                                    }
                                } else {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: interim text: null");
                                    }
                                }
                            } else if (source.getClassName().toString().matches(
                                    GOOGLE_VOICE_SEARCH_FINAL_FIELD)) {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: className final: true");
                                    Log.i(CLS_NAME, "onAccessibilityEvent: source.getClassName: " + source.getClassName());
                                }

                                final int childCount = source.getChildCount();
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: childCount: " + childCount);
                                }

                                if (childCount > 0) {
                                    for (int i = 0; i < childCount; i++) {

                                        final String text = examineChild(source.getChild(i));

                                        if (text != null) {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: child text: " + text);
                                            }

                                            if (finalMatch(text)) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: child: final match: true");
                                                }

                                                if (commandDelaySufficient(event.getEventTime())) {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: true");
                                                    }

                                                    if (!commandPreviousMatches(text)) {
                                                        if (DEBUG) {
                                                            Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: false");
                                                        }

                                                        previousCommandTime = event.getEventTime();
                                                        previousCommand = text;

                                                        killGoogle();

                                                        if (DEBUG) {
                                                            Log.e(CLS_NAME, "onAccessibilityEvent: FINAL PROCESSING: " + text);
                                                        }

                                                    } else {
                                                        if (DEBUG) {
                                                            Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: true");
                                                        }
                                                    }
                                                } else {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: false");
                                                    }
                                                }
                                                break;
                                            } else {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: child: final match: false");
                                                }
                                            }
                                        } else {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: child text: null");
                                            }
                                        }
                                    }
                                }
                            } else {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: className: unwanted " + source.getClassName());
                                }

                                if (EXTRA_VERBOSE) {

                                    if (source.getText() != null) {

                                        final String text = source.getText().toString();
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: unwanted text: " + text);
                                        }
                                    } else {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: unwanted text: null");
                                        }
                                    }

                                    final int childCount = source.getChildCount();
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: unwanted childCount: " + childCount);
                                    }

                                    if (childCount > 0) {

                                        for (int i = 0; i < childCount; i++) {

                                            final String text = examineChild(source.getChild(i));

                                            if (text != null) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: unwanted child text: " + text);
                                                }
                                            }
                                        }
                                    }
                                }
                            }
                        } else {
                            if (DEBUG) {
                                Log.i(CLS_NAME, "onAccessibilityEvent: source null");
                            }
                        }
                    } else {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "onAccessibilityEvent: checking for google: false");
                        }
                    }
                    break;
                default:
                    if (DEBUG) {
                        Log.i(CLS_NAME, "onAccessibilityEvent: not interested in type");
                    }
                    break;
            }
        } else {
            if (DEBUG) {
                Log.i(CLS_NAME, "onAccessibilityEvent: event null");
            }
        }
    }

    /**
     * Check if the previous command was actioned within the {@link #COMMAND_UPDATE_DELAY}
     *
     * @param currentTime the time of the current {@link AccessibilityEvent}
     * @return true if the delay is sufficient to proceed, false otherwise
     */
    private boolean commandDelaySufficient(final long currentTime) {
        if (DEBUG) {
            Log.i(CLS_NAME, "commandDelaySufficient");
        }

        final long delay = (currentTime - COMMAND_UPDATE_DELAY);

        if (DEBUG) {
            Log.i(CLS_NAME, "commandDelaySufficient: delay: " + delay);
            Log.i(CLS_NAME, "commandDelaySufficient: previousCommandTime: " + previousCommandTime);
        }

        return delay > previousCommandTime;
    }

    /**
     * Check if the previous command/text matches the current text we are considering processing
     *
     * @param text the current text
     * @return true if the text matches the previous text we processed, false otherwise.
     */
    private boolean commandPreviousMatches(@NonNull final String text) {
        if (DEBUG) {
            Log.i(CLS_NAME, "commandPreviousMatches");
        }

        return previousCommand != null && previousCommand.matches(text);
    }

    /**
     * Check if the interim text matches a command we want to intercept
     *
     * @param text the intercepted text
     * @return true if the text matches a command false otherwise
     */
    private boolean interimMatch(@NonNull final String text) {
        if (DEBUG) {
            Log.i(CLS_NAME, "interimMatch");
        }
        return text.matches("do interim results work");
    }

    /**
     * Check if the final text matches a command we want to intercept
     *
     * @param text the intercepted text
     * @return true if the text matches a command false otherwise
     */
    private boolean finalMatch(@NonNull final String text) {
        if (DEBUG) {
            Log.i(CLS_NAME, "finalMatch");
        }

        return text.matches("do final results work");
    }

    /**
     * Recursively examine the {@link AccessibilityNodeInfo} object
     *
     * @param parent the {@link AccessibilityNodeInfo} parent object
     * @return the extracted text or null if no text was contained in the child objects
     */
    private String examineChild(@Nullable final AccessibilityNodeInfo parent) {
        if (DEBUG) {
            Log.i(CLS_NAME, "examineChild");
        }

        if (parent != null) {

            for (int i = 0; i < parent.getChildCount(); i++) {

                final AccessibilityNodeInfo nodeInfo = parent.getChild(i);

                if (nodeInfo != null) {
                    if (DEBUG) {
                        Log.i(CLS_NAME, "examineChild: nodeInfo: getClassName: " + nodeInfo.getClassName());
                    }

                    if (nodeInfo.getText() != null) {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: have text: returning: " + nodeInfo.getText().toString());
                        }
                        return nodeInfo.getText().toString();
                    } else {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: text: null: recurse");
                        }

                        final int childCount = nodeInfo.getChildCount();
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: childCount: " + childCount);
                        }

                        if (childCount > 0) {

                            final String text = examineChild(nodeInfo);

                            if (text != null) {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "examineChild: have recursive text: returning: " + text);
                                }
                                return text;
                            } else {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "examineChild: recursive text: null");
                                }
                            }
                        }
                    }
                } else {
                    if (DEBUG) {
                        Log.i(CLS_NAME, "examineChild: nodeInfo null");
                    }
                }
            }
        } else {
            if (DEBUG) {
                Log.i(CLS_NAME, "examineChild: parent null");
            }
        }

        return null;
    }

    /**
     * Kill or reset Google
     */
    private void killGoogle() {
        if (DEBUG) {
            Log.i(CLS_NAME, "killGoogle");
        }

        // TODO - Either kill the Google process or send an empty intent to clear current search process
    }

    @Override
    public void onInterrupt() {
        if (DEBUG) {
            Log.i(CLS_NAME, "onInterrupt");
        }
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
        if (DEBUG) {
            Log.i(CLS_NAME, "onDestroy");
        }
    }
}

我尽可能地使课程变得冗长,所以希望它更容易理解。
它执行以下操作:
检查事件类型是否为正确的类型
检查软件包是否是谷歌的“现在”软件包
检查硬编码类类型的节点信息
在将临时语音命令加载到视图中时检查该命令
在加载到视图中时检查最终语音命令
递归检查视图中的语音命令
检查事件之间的时差
检查语音命令是否与先前检测到的相同
要测试:
启用Android辅助功能设置中的Service
您的应用程序可能需要重新启动,服务才能正确注册。
启动谷歌语音识别并说“做临时结果工作”
立即退出Google
开始识别并说“做最后的结果工作”
上面将演示从两个硬编码视图中提取的文本/命令。如果现在不重新启动Google,该命令仍会被检测为临时命令。
使用提取的语音命令,您需要执行自己的语言匹配,以确定这是否是您感兴趣的命令。如果是,你需要阻止谷歌说出或显示结果。这是通过现在杀死Google或发送一个空的语音搜索意图来实现的,其中包含了一些应该clear/reset task的标志。
这样做的话,您将处于一种竞争状态,因此您的语言处理需要相当智能,或者相当基本……
希望有帮助。
编辑:
对于那些要求“立即杀死”Google的人,您要么需要拥有杀死进程的权限,要么发送一个空的(“”)搜索意图来清除当前搜索:
public static final String PACKAGE_NAME_GOOGLE_NOW = "com.google.android.googlequicksearchbox";
public static final String ACTIVITY_GOOGLE_NOW_SEARCH = ".SearchActivity";

/**
 * Launch Google Now with a specific search term to resolve
 *
 * @param ctx        the application context
 * @param searchTerm the search term to resolve
 * @return true if the search term was handled correctly, false otherwise
 */
public static boolean googleNow(@NonNull final Context ctx, @NonNull final String searchTerm) {
    if (DEBUG) {
        Log.i(CLS_NAME, "googleNow");
    }

    final Intent intent = new Intent(Intent.ACTION_WEB_SEARCH);
    intent.setComponent(new ComponentName(PACKAGE_NAME_GOOGLE_NOW,
            PACKAGE_NAME_GOOGLE_NOW + ACTIVITY_GOOGLE_NOW_SEARCH));

    intent.putExtra(SearchManager.QUERY, searchTerm);
    intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TOP
            | Intent.FLAG_ACTIVITY_RESET_TASK_IF_NEEDED);

    try {
        ctx.startActivity(intent);
        return true;
    } catch (final ActivityNotFoundException e) {
        if (DEBUG) {
            Log.e(CLS_NAME, "googleNow: ActivityNotFoundException");
            e.printStackTrace();
        }
    } catch (final Exception e) {
        if (DEBUG) {
            Log.e(CLS_NAME, "googleNow: Exception");
            e.printStackTrace();
        }
    }

    return false;

}

本文翻译自 https://stackoverflow.com/questions/38482626/

网站遵循 CC BY-SA 4.0 协议,转载或引用请注明出处。


相关文章:

java - 在Spring 5中删除jdbc.support.nativejdbc的替代品

java - 如何在Criteria API(JPA)中获得单个列的不同值

java - 如何使用Lucene StoredFieldsFormat?

android - 如何禁用Google Assist API(现在点击)

android - 如何使用自定义操作在Android中扩展Google即时语音命令

java - 如何使用Java在网络打印机上打印?

java - android开发 - 注册和取消注册onClick,无法解析方法

android - Android在android源代码中更改iptable规则

android - Android - 打开电子邮件应用程序?

java - 开发类似于“Google now”的应用程序 - 是否可以在没有键输入的情况下使用语音识别?