The method and apparatus of test macro and execution test assignment
Technical field
The present invention relates to automatic test technology field, particularly relate to the method and apparatus of a kind of test macro and execution test assignment.
Background technology
Along with the increase day by day of mobile terminal product, the performance of mobile terminal and function enjoy user to pay close attention to, and therefore, the test job of mobile terminal is particularly important for guarantee mobile terminal quality.
A kind of structure of automatization test system as shown in Figure 1, comprises mobile terminal 1, testing server 2 and testing apparatus 3.Testing server 2 performs test assignment, and carry out various test by testing apparatus 3 pairs of mobile terminals 1, described test assignment comprises multiple test case.
But, in the implementation of test assignment, when there is test errors, owing to not having effective measures to tackle, test process will interrupt or stop, and even test macro can be caused to paralyse when performing test assignment and frequently makeing mistakes, therefore, the less stable of existing automatization test system.
On the other hand, when testing appearance and extremely causing systemic breakdown, need restarting systems and re-execute test assignment, if frequent starting system, system performance also can be affected, and not only testing efficiency reduces, and testing cost also can increase.
Summary of the invention
What the embodiment of the present invention solved is the problem that the poor stability of existing Auto-Test System and testing efficiency are low.
For solving the problems of the technologies described above, the embodiment of the present invention provides a kind of method performing test assignment, and described method is applied in test macro, and described test macro comprises testing server and testing apparatus, described testing server performs test assignment, is tested mobile terminal by testing apparatus; Described test assignment comprises at least one test case; The method of described execution test assignment comprises:
When detecting that test is abnormal, terminate the test case performed; Described test is abnormal comprises that testing server is abnormal, testing apparatus exception and mobile terminal abnormal at least one;
All detect that when the test case of predetermined number performs test is abnormal, preserves the implementation progress of described test assignment, continue to perform described test assignment after restarting at least one equipment in testing server and testing apparatus.
Optionally, the described test case when predetermined number all occurs when performing that test is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting at least one equipment in testing server and testing apparatus and comprise following at least one:
All detect that when the test case of continuous first predetermined amount performs testing server is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting described testing server;
All detect that when the test case of continuous second predetermined amount performs testing apparatus is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting described testing apparatus;
All detect that when the test case of continuous 3rd predetermined amount performs mobile terminal is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting described testing server;
All detect that when the test case of the 4th predetermined amount performs testing server is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting described testing server;
All detect that when the test case of the 5th predetermined amount performs testing apparatus is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting described testing apparatus;
All detect that when the test case of the 6th predetermined amount performs mobile terminal is abnormal, preserve the implementation progress of described test assignment, continue to perform described test assignment after restarting described testing server;
All detect that when the test case of the 7th predetermined amount performs test is abnormal, preserves the implementation progress of described test assignment, continue to perform described test assignment after restarting testing server and testing apparatus.
Optionally, the method for described execution test assignment also comprises: before execution test assignment, check standardization and the rationality of described test assignment.
Optionally, the method of described execution test assignment also comprises: repeat described test assignment until perform number of times and reach test cases all in preset times or described test assignment and all test and pass through, wherein, only perform when repeating test assignment at every turn and test unsanctioned test case when previous execution test assignment.
Optionally, the method for described execution test assignment also comprises: reported by E-mail mode feedback test.
Optionally, described testing server is abnormal comprises following at least one situation: test interface mistake, test process conflict, test process are closed by mistake and testing server crashes or power-off.
Optionally, described testing apparatus is abnormal comprises: testing apparatus crashes or power-off.
Optionally, described mobile terminal is abnormal comprises following at least one situation: it is abnormal that terminal loses port phenomenon, coprocessor exception and application processor.
For solving the problems of the technologies described above, the embodiment of the present invention also provides a kind of device performing test assignment, is applied in test macro, and described test macro comprises testing server and testing apparatus, described testing server performs test assignment, is tested mobile terminal by testing apparatus; Described test assignment comprises at least one test case; The device of described execution test assignment comprises:
Abnormality detecting unit, is suitable for the execute phase in test assignment, detects and whether occurs that test is abnormal, described test is abnormal comprises that testing server is abnormal, testing apparatus exception and mobile terminal abnormal at least one;
Finishing control unit, is suitable for, when described abnormality detecting unit detects that test is abnormal, controlling the test case that described testing server terminates performing;
Preserve control module, be suitable for the described abnormality detecting unit when the test case of predetermined number performs and all detect that test is abnormal, control the implementation progress that described testing server preserves described test assignment;
Restart control module, be suitable for controlling the described abnormality detecting unit when the test case of predetermined number performs and all detect that test is abnormal, control after described testing server preserves the implementation progress of described test assignment at described preservation control module, at least one equipment controlled in described testing server and testing apparatus restarts;
After restarting, described testing server continues to perform described test assignment.
Optionally, the device performing test assignment also comprises: previewing unit, is suitable for, before execution test assignment, checking standardization and the rationality of described test assignment.
Optionally, the device of described execution test assignment also comprises: Repetitive controller unit, be suitable for controlling described testing server and repeat described test assignment until perform number of times and reach test cases all in preset times or described test assignment and all test and pass through, wherein, only perform when repeating test assignment at every turn and test unsanctioned test case when previous execution test assignment.
Optionally, the device of described execution test assignment also comprises: feedback unit, is suitable for by email feedback test report.
The embodiment of the present invention also provides a kind of test macro, comprises the device of testing server, testing apparatus and described execution test assignment.
Compared with prior art, the method and apparatus of the execution test assignment of the embodiment of the present invention at least has following beneficial effect:
Detect test in real time abnormal, with the execution of tracking and testing task, the test case performed can be terminated in time when there being test abnormal, to stop the test case that may cause test errors in time, reducing the possibility of the test macro paralysis caused because testing exception thus, improve the stability of test macro.All detect that when there being the test case of predetermined number to perform test is abnormal, continue to perform described test assignment according to testing progress after restarting systems, avoid the impact of frequent starting system on system performance, also ensure that the continuity that test assignment performs, thus improve testing efficiency, reduce testing cost.
Repeat test assignment, again perform the unsanctioned test case of test and can reduce the impact of non-determined factor on implementation of test cases, thus improve test percent of pass.
Before execution test assignment, incorrect test assignment can be fed back to tester and system manager by the standardization of checkout task and rationality in time, avoids because irrational test assignment waste test resource.
By email feedback test report, in real time test progress can be fed back to tester or system manager, make tester or system manager can find test errors as early as possible, retrofit testing scheme and test macro.Further, tester or system manager do not need control and measuring system always yet, can understand testing progress and test result at any time by read emails.
Accompanying drawing explanation
Fig. 1 is a kind of structural representation of test macro;
Fig. 2 is the schematic flow sheet of the method for the execution test assignment of the embodiment of the present invention;
Fig. 3 is the structural representation of the device of the execution test assignment of method shown in corresponding diagram 2;
Fig. 4 is the schematic flow sheet of the method for the execution test assignment of the embodiment of the present invention;
Fig. 5 is the structural representation of the device of the execution test assignment of method shown in corresponding diagram 4.
Embodiment
The method of the execution test assignment of the embodiment of the present invention shown in Fig. 2 can be applied in test macro, test macro such as shown in Fig. 1 comprises testing server and testing apparatus, described testing server performs test assignment, tested mobile terminal by testing apparatus, described test assignment comprises at least one test case.Described testing apparatus can be corresponding with network formats, described network formats comprises 2G network, (as GSM), 3G network (as TD-SCDMA, WCDMA, CDMA2000), 4G network (as LTE) etc., in general, the corresponding a kind of network formats of each testing apparatus (also referred to as tester), described testing server is suitable for being connected with testing apparatus, carry out various test by testing apparatus to tested mobile terminal, testing apparatus can be communicated to connect with testing server by netting twine or Connection Card (as GPIB Connection Card).
Test under adopting different testing apparatuss to realize different network formats respectively, the test that multimode is relevant, the communication pattern can supported according to different tested mobile terminal, configures different testing apparatuss and carries out combined test.Described communication pattern is the combination of network formats and single mode, bimodulus or multimode; such as, GSM single mode, TD-SCDMA single mode, WCDMA single mode, TD-LTE single mode, FDD-LTE single mode, GSM/TD-SCDMA bimodulus, GSM/WCDMA bimodulus, GSM/TD-SCDMA/WCDMA tri-mould, GSM/TD-SCDMA/TD-LTE tri-mould, GSM/TD-SCDMA/FDD-LTE tri-mould, GSM/WCDMA/TD-LTE tri-mould, GSM/WCDMA/FDD-LTE tri-mould, GSM/TD-SCDMA/TD-LTE/WCDMA/FDD-LTE five mould etc.
As shown in Figure 2, testing server starts to perform test assignment.During concrete enforcement, testing server can comprise performance element, and after testing server gets test assignment, described performance element starts to perform test assignment, to be tested mobile terminal by testing apparatus.Usually, described performance element can according to the order implementation of test cases successively of the test case listed by test case list, and described test case list lists the test case that will perform when performing test assignment.
When testing server implementation of test cases, perform step S21, detect whether test exception, if then perform step S22; Then perform step S23 if not.Particularly, perform in the process of each test case at described performance element, whether real-time detection occurs that test is abnormal, described test is abnormal comprises that testing server is abnormal, testing apparatus exception and mobile terminal abnormal at least one.Described testing server can comprise extremely: the closedown by mistake of test interface mistake, test process conflict, test process, testing server deadlock or power-off etc.Described testing apparatus can comprise extremely: testing apparatus deadlock, power-off or improper closedown etc.Described mobile terminal can comprise extremely: terminal loses port phenomenon, coprocessor exception or application processor exception etc.Some prior aries can be adopted to detect in real time testing server and testing apparatus, to determine whether to occur that test is abnormal.
Step S22, terminates the test case performed, and then performs step S24.Particularly, when detecting that test is abnormal, terminating the test case performed, can also the test case terminating to perform be labelled further, for identifying this testing case not by (fail).In embodiments of the present invention, abnormal for different tests, can by adding different marks to distinguish.When an execution test case, detect testing server and testing apparatus in real time, with the execution of tracking and testing task, the test case performed can be terminated in time when there being test abnormal, to stop the test case that may cause test errors in time, reduce the possibility of the test macro paralysis caused because testing exception thus.
Step S23, judges that whether current test case is complete, if then perform step S26, then performs step S21 if not, also, in the implementation of a test case, detects test in real time abnormal.When a test case is complete, namely the judged result of step S23 is yes, records this test case for test is by (pass).
Step S24, judges whether that the test case of predetermined number all detects that test is abnormal when performing, if then perform step S25, then performs step S26 if not.
In embodiments of the present invention, judge whether described in that the test case of predetermined number all detects when performing that test judges any one or more situation following abnormal can comprising:
(1) all detect that testing server is abnormal when whether the test case of continuous first predetermined amount performs;
(2) all detect that testing apparatus is abnormal when whether the test case of continuous second predetermined amount performs;
(3) all detect that mobile terminal is abnormal when whether the test case of continuous 3rd predetermined amount performs;
(4) all detect that testing server is abnormal when whether the test case of the 4th predetermined amount performs;
(5) all detect that testing apparatus is abnormal when whether the test case of the 5th predetermined amount performs;
(6) all detect that mobile terminal is abnormal when whether the test case of the 6th predetermined amount performs;
(7) all detect that test is abnormal when whether the test case of the 7th predetermined amount performs.
The numerical value of each predetermined amount can correspondingly be arranged according to the actual requirements, does not limit at this.Judge whether that the test case of predetermined number all detects when performing that test can be judge whether that the test case of the some performed continuously all occurs same class test extremely when performing, as the situation cited by aforementioned (1) to (3) extremely.Judge whether that the test case of predetermined number all detects when performing that test also can be judge whether that the quantity occurring the test case that same class test is abnormal when performing reaches certain quantity extremely, no matter whether these test cases are perform continuously, as the situation cited by aforementioned (4) to (6).Judging whether that the test case of predetermined number all detects when performing that test is abnormal can not also the classification of distinguishing tests exception, as long as occur that when performing the quantity of the test case that test is abnormal reaches certain quantity, as the situation cited by aforementioned (7).
Step S25, preserves the implementation progress of described test assignment, restarts at least one equipment in testing server and testing apparatus, then performs step S27.
Particularly, if the judged result of step S24 is yes, then preserve the implementation progress of described test assignment, such as can preserve the list of unenforced test case, then restart at least one equipment in test macro, can be restart testing server and testing apparatus, in embodiments of the present invention, be the different situations judged for step S24, correspondingly restart and produce abnormal equipment, as: testing server as described in then can restarting for situation (1), (4); Then described testing apparatus can be restarted for situation (2), (5), usually testing apparatus can be restarted by controlling testing server to testing apparatus transmission order; Described testing server and testing apparatus can be restarted for situation (7); And for situation (3), (6), owing to being mobile terminal exception, the phenomenon detected normally testing server normally can not identify mobile terminal or can not carry out proper communication with mobile terminal, and therefore taked measure also can be restart described testing server.
Step S26, judges whether unenforced test case in addition, if then perform step S27, then current test assignment test terminates if not.Particularly, the test case whether having execution in the test case list of current test assignment or not can be judged.
Step S27, continues to perform another test case, then performs step S21, in the implementation of current test case, detects test in real time abnormal.During concrete enforcement, can, according to the order of the test case in current test case list, continue to perform next test case.After testing server or testing apparatus restart, owing to saving the list of unenforced test case, then continue the test case performed in list to be equivalent to continue to perform test assignment according to restarting front test assignment progress of preserving, by preserving testing progress, after again, recover test assignment according to testing progress can ensure the continuity that test assignment performs, improve testing efficiency.
Corresponding to the method for the execution test assignment shown in Fig. 2, the device of the execution test assignment of the embodiment of the present invention as shown in Figure 3, comprises abnormality detecting unit 31, finishing control unit 32, preserves control module 33 and restart control module 34.Described device can be arranged in the testing server of test macro, also can be the autonomous device for controlling testing server in test macro.
Abnormality detecting unit 31 is suitable for the execute phase in test assignment, detects and whether occurs that test is abnormal.During concrete enforcement, abnormality detecting unit 31 is suitable in the implementation of each test case, detects test abnormal, described test is abnormal comprises that testing server is abnormal, testing apparatus exception and mobile terminal abnormal at least one.
Finishing control unit 32 is suitable for, when abnormality detecting unit 31 detects that test is abnormal, controlling the test case that described testing server terminates performing.
Preservation control module 33 is suitable for the abnormality detecting unit 31 when the test case of predetermined number performs and all detects that test is abnormal, controls the implementation progress that described testing server preserves described test assignment.
Restart control module 34 to be suitable for the abnormality detecting unit 31 when the test case of predetermined number performs and all to detect that test is abnormal, control after described testing server preserves the implementation progress of described test assignment at preservation control module 33, at least one equipment controlled in described testing server and testing apparatus restarts.After restarting, described testing server continues to perform described test assignment.
The method of the execution test assignment of the embodiment of the present invention shown in Fig. 2 shows a test assignment implementation, in a test assignment implementation, each test case performs once, complete test case is recorded as the test case of testing and passing through, and is recorded as the unsanctioned test case of test in the process of implementation because testing the abnormal test case be terminated.Further, in order to reduce the impact of non-determined factor on implementation of test cases, repeatedly can repeat test assignment until perform number of times and reach test cases all in preset times or described test assignment and all test and pass through, when repeating test assignment at every turn, only can perform and test unsanctioned test case when previous execution test assignment.
First the method for the execution test assignment of the embodiment of the present invention as shown in Figure 4 performs step S41, and the execution number of times arranging test assignment is 1, then performs step S42.
Step S42, performs test assignment.The process that first time performs test assignment as shown in Figure 2, then can perform step S43.
Step S43, performs number of times and adds 1, judges whether reach preset times, if so, namely testing time equals preset times, then perform step S44 if performing number of times; If not, namely testing time is less than preset times, then perform step S45.Described preset times be greater than 1 integer, can arrange according to the actual requirements, consider that the test duration is unsuitable long, testing time also should not arrange too large, generally can be set to 3 ~ 5 times.
Step S44, performs test assignment.The process performing test assignment can reference diagram 2, with step S42 slightly unlike, when this performs test assignment, only perform and test unsanctioned test case when previous execution test assignment.Step S44 performs test assignment for the last time, and whether all the test of the test case no matter in test assignment is passed through, and then terminates the execution flow process of described test assignment after this is complete.
Step S45, performs test assignment.The process performing test assignment can reference diagram 2, with step S42 slightly unlike, when this performs test assignment, only perform and test unsanctioned test case when previous execution test assignment.
Step S46, judges whether that all test cases are all tested and passes through, if then terminate, then performs step S43 if not.
Corresponding to the method for the execution test assignment shown in Fig. 4, the device of the execution test assignment of the embodiment of the present invention as shown in Figure 5, comprises abnormality detecting unit 31, finishing control unit 32, preserves control module 33, restarts control module 34 and Repetitive controller unit 35.Abnormality detecting unit 31, finishing control unit 32, preserve control module 33 and restart control module 34 same as shown in Figure 2.Repetitive controller unit 35 is suitable for controlling described testing server and repeats described test assignment until perform number of times and reach test cases all in preset times or described test assignment and all test and pass through, when repeating test assignment at every turn, only can perform and test unsanctioned test case when previous execution test assignment.According to the testing result of abnormality detecting unit 31, Repetitive controller unit 35 can judge whether that all test cases are all tested passes through.
In possibility, the method for the execution test assignment of the embodiment of the present invention can also comprise: before execution test assignment, check standardization and the rationality of described test assignment.Accordingly, the device of the execution test assignment of the embodiment of the present invention can also comprise previewing unit, is suitable for, before execution test assignment, checking standardization and the rationality of described test assignment.
Described test assignment can be set up by tester and input, and also automatically can set up test assignment by test macro according to test case.Such as, tester according to testing requirement, can fill in and submits to test single in inputting interface.Test single test single-character given name that mainly comprises to claim (Test_ID), test the title of the test case (case) of needs checking, software version path, platform information, test prioritization and whether inform the progress etc. of testing single corresponding test assignment by timed sending mail.Test_ID can give prominence to the characteristic information of test, the name of such as tester, test purpose etc.Testing server singly sets up test case list corresponding to test assignment according to the test that tester fills in.
Before test assignment performs, can the content of test list corresponding to checkout task whether correct, such as, check that whether the software version path of filling in is correct, detect the platform filled in whether to exist, when a station server carrying multi-section mobile terminal simultaneously, whether the port that detect corresponding terminal exists, whether communicate normal.The incorrect test of content is single can be return, and Wrong localization, by mail sending to system manager and tester, be convenient to tester and change test list in time.So just single for incorrect for content test very first time can be fed back to tester and system manager, avoid because irrational test assignment waste test resource.
In possibility, the method for the execution test assignment of the embodiment of the present invention can also comprise: reported by E-mail mode feedback test.Accordingly, the device of the execution test assignment of the embodiment of the present invention can also comprise: feedback unit, is suitable for being reported by E-mail mode feedback test.
During concrete enforcement, performance element is after acquisition test assignment, and configuration testing environment, sends corresponding AT order and make mobile terminal enter downloading mode, automatically downloads the software version under the software version path that test list is filled in.After performance element detects the mark that software version has been downloaded, software version will be closed and download interface, enter test interface, perform test.
Performance element is tested each case, each case generates a file extent and is called tracking (trace) file of .txt and corresponding daily record (log, such as, dsplog, armlog etc.), the generation of log and preservation are optional, if produce log information, log preserves on the server, to save storage space after being through compression.
Performance element produces at test process test (Test) file that a file extent is called .txt, and the test result (pass, fail) of each case of real time record and each case complete the test duration of test.Test file is preserved on the server after overcompression, to save storage space.
Further, performance element is after each execution test assignment, the case number of all right statistical test pass and the case number of test fail, calculate the percent of pass (pass ratio) of case, and statistics and result of calculation be recorded in test file.
Described test report (Test Report) can comprise at least one in the implementation progress and test file etc. of trace file, daily record, test assignment.Such as when implementation of test cases, in real time the trace file of test case and daily record can be fed back to tester or system manager; In the implementation of test assignment, in real time the trace file of test case and the implementation progress of test assignment can be fed back to tester or system manager; After test assignment execution terminates, test file can be fed back to tester or system manager.
By Email, test report is sent to tester or system manager, can when testing abnormal, notify tester or system manager in time, make tester or system manager can find test errors as early as possible, retrofit testing scheme and test macro.Further, tester or system manager do not need control and measuring system always yet, can understand testing progress and test result at any time by read emails.
It will be appreciated by those skilled in the art that, the all or part of of above-described embodiment is that the hardware that can carry out instruction relevant by program has come, described program can be stored in computer-readable recording medium, and described storage medium can be ROM, RAM, magnetic disc, CD etc.
Although the present invention discloses as above, the present invention is not defined in this.Any those skilled in the art, without departing from the spirit and scope of the present invention, all can make various changes or modifications, and therefore protection scope of the present invention should be as the criterion with claim limited range.