概述

TvtsAssistantTest是google最近推出的关于TV的TVTS中的一个测试模块，用来测试系统健壮性（system health ）的，主要是通过katniss（google 语音助手）查找天气、启动youtube等相关测试本文基于android-tvts_r1.03版本进行说明；

测试内容

环境准备与测试方法

环境准备：

1.需要连接外网
2.需要将系统语言切换为English（US）
注意：Assistant的后台服务器会根据不同系统语言返回不一样的搜索结果，Assistant会展示不同的界面布局，非English（US）语言下会因找不到对应控件ID而导致测试失败

测试方法：

进入tools目录，执行tvts-tradefed，会进入测试控制台；
测试TvtsAssistantTest模块：run tvts-rc –m TvtsAssistantTest
测试单用例：run tvts-rc –mTvtsAssistantTest –t testcase_name

测试代码流程

配置文件：

通过编辑器打开android-tvts_r1.03\android-tvts\testcases下的TvtsAssistantTestCases.config文件可以看到对应的测试代码在TvtsAssistantTestCases.jar中

测试代码：

通过反编译TvtsAssistantTestCases.jar，在TvtsAssistantTest中可以看到

  @Testpublic void testNormalDeviceUsage() {this.mMetricAggregators.add(new KatnissNormalUseSearchToQueryKpi());this.mMetricAggregators.add(new KatnissNormalUseQueryToResultKpi());runAssistantTest("ShowInformationTest", 20);runAssistantTest("OpenYouTubeTest", 20);runAssistantTest("WeatherTest", 20);}@Testpublic void testHeavyDeviceUsage() {this.mMetricAggregators.add(new KatnissHeavyUseSearchToQueryKpi());this.mMetricAggregators.add(new KatnissHeavyUseQueryToResultKpi());runAssistantTest("HeavyDeviceUsageTest", 20);}

其中的runAssistantTest方法在tvts-assistant-common.jar中的TvtsBaseAssistantTest类中，具体方法如下

 public void runAssistantTest(String testName, int numRuns) {Throwable ex;this.mCurrentTestName = testName;if (getNumTestRuns() != null && getNumTestRuns().intValue() > 0) {numRuns = getNumTestRuns().intValue();}CLog.logAndDisplay(LogLevel.INFO, "Test: %s", new Object[]{testName});CLog.logAndDisplay(LogLevel.INFO, "%s\t%s\t%s", new Object[]{SEARCH_TO_QUERY, QUERY_TO_RESULT, QUERY_TO_RESULT_NO_NET});for (int i = 1; i <= numRuns; i++) {try {this.mSystemHealthMetricParser = getSystemHealthMetricParser();preConditionChecks();preIterationActions();File metricsFile = getDevice().pullFile(METRICS_PATH);if (metricsFile != null) {runBashCmd(String.format("rm -f %s", new Object[]{metricsFile.getAbsolutePath()}));}getDevice().executeShellCommand(String.format("truncate -s 0 %s", new Object[]{METRICS_PATH}));runTestAndProcess(testName); //此方法中会安装testName对应的apkpostIterationActions();} catch (AssertionError e) {ex = e;try {CLog.logAndDisplay(LogLevel.ERROR, "Run %d/%d encountered exception. See logs for details", new Object[]{Integer.valueOf(i), Integer.valueOf(numRuns)});CLog.e(ex.getMessage());} finally {postIterationActions();}} catch (DeviceNotAvailableException e2) {ex = e2;CLog.logAndDisplay(LogLevel.ERROR, "Run %d/%d encountered exception. See logs for details", new Object[]{Integer.valueOf(i), Integer.valueOf(numRuns)});CLog.e(ex.getMessage());} catch (FileNotFoundException e3) {ex = e3;CLog.logAndDisplay(LogLevel.ERROR, "Run %d/%d encountered exception. See logs for details", new Object[]{Integer.valueOf(i), Integer.valueOf(numRuns)});CLog.e(ex.getMessage());}}}

在runAssistantTest方法中会安装传入testName对应的testName.apk，并运行对应的apk进行测试，根据传入的numRuns确定循环测试的次数，即各测试单项将运行测试20次；
以ShowInformationTest.apk为例进行流程说明：
通过反编译ShowInformationTest.apk找到对应方法，

package com.google.android.apps.tvsearch.systemhealth;import androidx.test.ext.junit.runners.AndroidJUnit4;
import com.google.android.apps.tvsearch.testing.systemhealth.SystemHealthTestCase;
import org.junit.Test;
import org.junit.runner.RunWith;@RunWith(AndroidJUnit4.class)
public class ShowInformationTest extends SystemHealthTestCase {@Testpublic void testDetailedShowInformation() throws Exception {openApp("audio/searchShowInformation.pcm");awaitKnowledgePanelWithTitle("The Big Bang Theory");}
}

   /* access modifiers changed from: protected */public void openApp(String packageName, String audioFile) throws Exception {String valueOf = String.valueOf(PATH);String valueOf2 = String.valueOf(audioFile);sendAudioFileBroadcast(valueOf2.length() != 0 ? valueOf.concat(valueOf2) : new String(valueOf));Thread.sleep(1000);setOpaTvIntent();awaitAppOpen(packageName);}/* access modifiers changed from: protected */public void openApp(String audioFile) throws Exception {openApp("com.google.android.katniss", audioFile);}

在 openApp(“audio/searchShowInformation.pcm”)方法中会启动com.google.android.katniss，并通过广播启动播放audio/searchShowInformation.pcm音频文件（对应音频文件在testcase文件中，配置文件中有提到 <0ption name=“push” value=“searchShowInformation.pcm-/storage/emulated/0/googletest/test_runfiles/google3/javatests/com/google/android/apps/tvsearch/systemhealth/audio/searchShowInformation.pcm” />,然后通过awaitKnowledgePanelWithTitle(“The Big Bang Theory”)方法在界面上搜索The Big Bang Theory内容，其中的判断方法如下

  public boolean titleMatchesExpectedString(String expectedTitle) throws UiObjectNotFoundException {return expectedTitle.equals(getByResourceId("com.google.android.katniss:id/entity_row_header_title").getText());}

titleMatchesExpectedString方法中通过"com.google.android.katniss:id/entity_row_header_title"控件中的字符串与"The Big Bang Theory"进行比对，awaitKnowledgePanelWithTitle方法中会循环调用titleMatchesExpectedString，找到了就返回测试耗时，若超时（设定时间为10s）则返回WaitTimeoutException报错，当测试完20次后会进行下一项测试，待测试完成后会先统计20次的成功率，若20次的成功率低于0.9则会报如下错误
[FAILED] Success Rate Requirement Source: Katniss Normal Use ShowInformationTest-search-to-query, Minimum Success Rate: 90.000000%, Actual Success Rate: 0.000000%
，当各测试项的成功率皆满足要求时，再取每项的20次的搜索耗时的平均值，当都小于3.5s时测试通过，否则报耗时大于要求的3500而测试fail；

常见错误现象及分析

1.adb 未连接错误

06-15 16:29:10 E/DeviceMonitor: Adb connection Error:EOF
06-15 16:29:10 E/DeviceMonitor: Unable to open connection to: localhost/127.0.0.1:5037, due to: java.net.ConnectException: 拒绝连接
06-15 16:29:10 E/DeviceMonitor: Connection attempts: 1
06-15 16:29:10 E/TvtsBaseAssistantTest: Run 4/20 encountered exception. See logs for details
06-15 16:29:10 E/TvtsBaseAssistantTest: Could not find device 192.168.1.111:5555
06-15 16:29:13 W/NativeDevice: AdbCommandRejectedException (device '192.168.1.111:5555' not found) when attempting shell dumpsys package com.android.tradefed.utils.wifi on device 192.168.1.111:5555

分析及解决方法：
log中显示无法找到设备device，检查pc与设备间的adb连接是否正常

2.网络未连接

06-15 16:58:16 E/TvtsBaseAssistantTest: Run 17/20 encountered exception. See logs for details
06-15 16:58:16 E/TvtsBaseAssistantTest: Device is not connected to network. Invalid metrics will be recorded.
06-15 17:00:39 I/TvtsBaseAssistantTest: -1.0               -1.0

分析及解决方法：该测试需要网络，检查网络是否正常，一般都是用的外网

3.未找到指定界面出现超时异常

06-15 16:57:53 E/TvtsBaseAssistantTest: Error in sideloaded apk validation. Skipping metrics gathering.
06-15 16:57:53 I/TvtsBaseAssistantTest: -1.0               -1.0
06-15 16:57:53 E/TvtsBaseAssistantTest: Run 16/20 encountered exception. See logs for details
06-15 16:57:53 E/TvtsBaseAssistantTest: OpenYouTubeTest failed:
com.google.android.apps.tvsearch.systemhealth.OpenYouTubeTest:
Error in testOpenYouTube(com.google.android.apps.tvsearch.systemhealth.OpenYouTubeTest):
com.google.android.apps.gsa.testing.ui.support.retrymanager.WaitTimeoutException: Retry timed out with error: Action: App is open: com.google.android.youtube.tv. Expected return value: true. Unexpected return value: false.at com.google.android.apps.gsa.testing.ui.support.retrymanager.Wait.determineExceptionToBeThrown(Wait.java:456)at com.google.android.apps.gsa.testing.ui.support.retrymanager.Wait.get(Wait.java:337)at com.google.android.apps.gsa.testing.ui.support.retrymanager.Wait.untilTrue(Wait.java:348)at com.google.android.apps.gsa.testing.ui.testcase.OpaTvTestCase.awaitAppOpen(OpaTvTestCase.java:212)at com.google.android.apps.tvsearch.systemhealth.OpenYouTubeTest.testOpenYouTube(OpenYouTubeTest.java:14)at java.lang.reflect.Method.invoke(Native Method)

分析及解决方法：通过log可以看出是com.google.android.youtube.tv未成功启动，先查看测试时指定界面YouTube是否存在，若测试时正常启动了，再跟踪代码查找判断方法，若是查找控件中的内容，则借助AS查看YouTube界面上的指定控件是否存在，内容是否符合，若youtube界面上无指定控件或控件中内容不符，则切换语言或重启试试，若仍不行，考虑向google提case；当然不排除中间由于网络不稳定断网或者设备与pc间断开链接的影响，这种情况可以通过之后打印出的log很容易发现；

4.耗时超时

junit.framework.AssertionFailedError: search-to-query 4302.000000 is above maximum 3500.000000

分析及解决方法：这种可能比较麻烦，先尝试重启机器、保证网络良好的情况下重跑用例看是否通过，若不能通过，说明设备性能达不到要求，查看cpu占用率，通过裁剪掉一些非必要的apk、kill掉一些系统中非必要的常驻进程和提高cpu最高频率等方法来提高设备性能；

其他类型异常
主要根据log信息，查看报错原因，反编译测试用例，结合代码具体分析原因；

TvtsAssistantTest相关推荐

Google XTS TV认证测试经验总结
一.简介销往海外的Android电视,若想使用Google的应用程序和服务,如YouTube.Gmail.Google Play Store等,必须通过此认证,获得Google授权. 目前XTS认证 ...

TvtsAssistantTest