Integrating REST APIs into CI / CD pipelines #
This article describes how to use the apptest.ai with CI(Continuous Integration) tools. You can use apptest.ai with the test creation, status view, and result view APIs.
1. [POST] Run New Testset #
This Run New Testset allows you to create a TestSet and run. Testset is a group of unit Tests.
Some options for the new test run follow the configuration information stored in the specific project.
[Request]
POST /openapi/v2/testset
Host : https://api.apptest.ai
Authorization : Basic {user_id}:{access_key}
Request Body Multipart Form Data #1
Key | Type | Description | Required |
---|---|---|---|
app_file | File | Target APP File to Test. Automatic test only. |
Yes |
Request Body Multipart Form Data #2
Key | Type | Description | Required | ||
---|---|---|---|---|---|
pid | Positive Number | – | Unique number of Project ex) 509 |
Yes | |
testset_name | String | Max 100 Characters | Name of Testset ex) “Testset name Example#1” |
Yes | |
os_type | String | ANDROID | iOS | OS type of test target ex) “ANDROID” |
Yes | |
scenario_id | Number | 0 | Positive | If Automated test, 0 (default). Otherwise, a unique number of Scenario. ex) 509 |
No | |
source_type | String | file | app_id | Currently scenario test only Type of Test Source (App file | App ID) Type of test source set in App Repository of the project |
Yes | |
time_limit | Positive Number | Min : 5 Max : 30 | Test time limit (Minutes) If the value is empty, it follows the time-limit saved in the project settings. ex) 5 |
No | |
use_vo | Boolean | Default: false | Automated test only Whether AT&T Video Optimizer(ARO) is used (true | false) ex) true |
No | |
callback | String | Max 250 characters | Callback URL to be called after test completion. ex) ‘https://127.0.0.1/callback/url/example’ |
No | |
credentials | login_id | String | Max 150 Characters | Account information(ID) of the test target app to be used to test the app (Test credentials info – Login ID) ex) ’test@gmail.com’, ‘01012345678’, ’testaccount’ |
No |
login_pw | String | Max 150 Characters | Account information(Password) of the test target app to be used to test the app (Test credentials info – Login PW) ex) ‘password1234!’ |
No |
[ Request Example #1 – Automated Test]
curl --request POST \\ --user {user_id}:{access_key} \\ -F \'app_file=@/path/of/your/app_file\' \\ -F \'data={\\ \"pid\": 509, \\ \"testset_name\": \"Testset Name Example #1\", \\ \"os_type\": \"ANDROID\", \\ \"time_limit\": 15, \\ \"use_vo\": false, \\ \"credentials\": { \\ \"login_id\": \"test@gmail.com\", \\ \"login_pw\": \"password1234!\" \\ } \\ }\' https://api.apptest.ai/openapi/v2/testset
[ Request Example #2 – Scenario Test]
curl --request POST \\ --user {user_id}:{access_key} \\ -F \'data={\\ \"pid\": 509, \\ \"testset_name\": \"Testset Name Example #1\", \\ \"os_type\": \"ANDROID\", \\ \"scenario_id\": 1204, \\ \"source_type\": \"file\", \\ \"time_limit\": 15, \\ }\' https://api.apptest.ai/openapi/v2/testset
[ Response ]
Key | Type | Description |
---|---|---|
test_count | Positive Number | Number of Unit tests run on the Testset ex) 3 |
testset_id | Positive Number | Unique number of the Testset ex) 251929 |
[ Response Example ]
{ \"data\": { \"test_count\": 1, \"testset_id\": 251929 }, \"result_code\": 0, \"result_msg\": \"\", \"reason\": \"\" }
[ Error Code ]
result_code | result_msg | reason |
---|---|---|
4000 | Missing Request Parameter | The required parameter was not found in the request parameters : {{ PARAMETER KEY }} |
4030 | App File Analysis Failed | iOS plistlib Parsing Error : {{ PARSING ERROR MSG }} |
4031 | Test creation failed : There was a problem pre-processing your app file. The IPA file must be a development version that is signed by your development certificate. | |
4032 | Test creation failed. Invalid app file. There was a problem pre-processing your app file. | |
4040 | Android Manifest Parsing Error : {{ PARSING ERROR MSG }} | |
4041 | No launchable activity found. (android.intent.category.LAUNCHER) | |
4050 | Invalid Request Parameter | File extension not supported. ( Support : ipa, zip, app, apk, xapk, apks ) |
4051 | The required parameter was not found in the request parameters : (app_file) | |
4052 | Invalid request parameter : (project_id: {{ PROJECT_ID }}) | |
4053 | Invalid request parameter : (testset_id: {{ TESTSET_ID }}) | |
4055 | Device is not compatible with this app : ( APP os type : {{ APP_OS_TYPE }} ) | |
4080 | Invalid Request Parameter | Device info does not exist in preset project data. |
4081 | Available devices don’t exist in preset project data. | |
5001 | Run New Testset Error | Global Exception Error – {{ ERROR_MSG }} |
6001 | App File Upload Error | App file upload failed. |
8000 | System Maintenance | The service is currently under system maintenance. |
2. [ GET ] Testset Status #
Check the progress of the testset
[ Request ]
GET /openapi/v2/testset/{testset_id}
Host : https://api.apptest.ai
Authorization : Basic {user_id}:{access_key}
Request URL Parameter
Key | Type | Description | Required |
---|---|---|---|
testset_id | Positive number | Unique number of Testset | Yes |
[ Request Example ]
curl --request GET \\ --user {user_id}:{access_key} \\ https://api.apptest.ai/openapi/v2/testset/55716
[ Response ]
Response Body Data Type : JSON
Key | Type | Description | |
---|---|---|---|
testset_status | String | Testset Status ( Complete, Running ) | |
testset_status_detail | JSON | Count of the result for each test | |
total_test_cnt | Positive number | Total test counts | |
error_cnt | Positive number | Error detected test counts | |
fail_cnt | Positive number | Failed test counts | |
initializing_cnt | Positive number | Initializing test counts | |
pass_cnt | Positive number | Passed test counts | |
running_cnt | Positive number | Running test counts | |
stop_cnt | Positive number | Stopped test counts | |
response_time | Datetime | Response time (timezone : UTC) |
[Response Example ]
{ \"data\": { \"testset_status\": \"Complete\", \"testset_status_detail\": { \"total_test_cnt\": 3, \"error_cnt\": 1, \"fail_cnt\": 1, \"initializing_cnt\": 0, \"running_cnt\": 1, \"stop_cnt\": 0, \"pass_cnt\": 0 }, \"response_time\": \"Mon, 25 May 2020 06:40:24 GMT\" }, \"result_code\": 0, \"result_msg\": \"\", \"reason\": \"\" }
[ Error Code ]
result_code | result_msg | reason |
---|---|---|
5002 | Get Testset Status Error | Global Exception Error – {ERROR_MSG} |
8000 | System Maintenance | The Service is currently under system maintenance. |
3. [ GET ] Testset Result #
This API can be called only when the test is completed. Returns the result data of the testset.
[ Request ]
GET /openapi/v2/testset/{testset_id}/result
Host : https://api.apptest.ai
Authorization : Basic {user_id}:{access_key}
Request URL Parameter
Key | Type | Description | Required |
---|---|---|---|
testset_id | Positive number | Unique number of Testset | Yes |
Request Query Parameter
Key | Type | Description | Required |
---|---|---|---|
data_type | String | Test result data type (all | xml | bare_xml | json | html) Default : all ex) https://api.apptest.ai/openapi/v2/testset/{testset_id}/result?data_type=all | No |
curl --request GET \\ --user {user_id}:{access_key} \\ https://api.apptest.ai/openapi/v2/testset/55716/result
curl --request GET \\ --user {user_id}:{access_key} \\ https://api.apptest.ai/openapi/v2/testset/55716/result?data_type=xml[ Response ] Response Body Data Type : JSON
Key | Type | Description |
---|---|---|
complete | Boolean | Whether the test is running or completed ( true | false ) |
result_xml | Boolean | Result data in XML format in JUnit format |
result_bare_xml | Boolean | Result data in XML format in JUnit format Exclude “?xml” and “testsuites” tag Use to run multiple tests at once |
result_html | Boolean | Result data in HTML format |
result_json | Boolean | Result data in JSON format in JUnit format |
Key | Type | Description |
---|---|---|
testsuite | name | apptest.ai’s project name |
testcase | name | Name of the device under test (Unit Test) |
time | Time the test was performed | |
system-out | Test result link of apptest.ai | |
error | message | Test results link where errors were found |
<?xml version=\"1.0\" encoding=\"UTF-8\"?> <testsuites name=\"TestBot Test\"> <testsuite name=\"{PROJECT NAME}.TestBot\"> <testcase name=\"{DEVICE NAME}\" time=\"{TEST TIME SECONDS}\"> <system-out>{RESULT PAGE LINK URL}</system-out> </testcase> <testcase name=\"{DEVICE NAME}\" time=\"{TEST TIME SECONDS}\"> <error message=\"{RESULT PAGE LINK URL}\" /> </testcase> </testsuite> </testsuites>[ Error Code ]
result_code | result_msg | reason |
---|---|---|
5003 | Get Testset Result Data Error | Global Exception Error – {ERROR_MSG} |
6002 | Get Testset Result Data Failed | Test is running yet. Please request when the test is complete. |
6003 | Data Does Not Exist | Test data does not exist. |
8000 | System Maintenance | The Service is currently under system maintenance. |
4. [ GET ] List of Scenario IDs #
This is an API to get the scenario information to be used when running the scenario test.
[ Request ]
GET /openapi/v2/project/{project_id}/scenarios
Host : https://api.apptest.ai
Authorization : Basic {user_id}:{access_key}
Request URL Parameter
Key | Type | Description | Required |
---|---|---|---|
project_id | Positive number | Unique number of project | Yes |
Request Query Parameter
Key | Type | Description | Required |
---|---|---|---|
os_type | String | OS type selected when registering the scenario ( ANDROID | iOS ) ex) https://api.apptest.ai/openapi/v2/project/{project_id}/scenarios?os_type=ANDROID |
No |
q | String | A query string to search in the scenario name ex) https://api.apptest.ai/openapi/v2/project/{project_id}/scenarios?q=loginhttps://api.apptest.ai/openapi/v2/project/{project_id}/scenarios?q=login |
No |
[ Request Example ]
curl --request GET \\ --user {user_id}:{access_key} \\ https://api.apptest.ai/openapi/v2/project/2019/scenarios?os_type=ANDROID&q=login_fail
[ Response ]
Response Body Data Type : JSON
Key | Type | Description |
---|---|---|
scenarios | Array | list of Scenario ID\’s |
response_time | Datetime | Response time (timezone: UTC) |
[ Response Example ]
{ \"data\": { \"scenarios\": [1204, 1206], \"response_time\": \"Mon, 25 May 2020 06:40:24 GMT\" }, \"result_code\": 0, \"result_msg\": 0, \"reason\": \"\" }
[ Error Code ]
result_code | result_msg | reason |
---|---|---|
2001 | Get Project Data Fail | Project data does not exist |
4051 | Invalid request parameter | Invalid request parameter : (project id) |
5005 | Get Scenario List Error | Global Exception Error – {ERROR_MSG} |
8000 | System Maintenance | The Service is currently under system maintenance. |
Appendix. API Result Codes #
result_code | result_msg | reason |
---|---|---|
4000 | Missing Request Parameter | The required parameter was not found in the request parameters : ({PARAMETER KEY}) |
4030 | App File Analysis Failed | iOS plistlib Parsing Error : {PARSING ERROR MSG} |
4031 | App File Analysis Failed | Test creation failed : There was a problem pre-processing your app file. The IPA file must be a development version that is signed by your development certificate. |
4032 | App File Analysis Failed | Test creation failed : Invalid app file. There was a problem pre-processing your app file. |
4040 | App File Analysis Failed | Android Manifest Parsing Error : {PARSING ERROR MSG} |
4041 | App File Analysis Failed | No launchable activity found. (android.intent.category.LAUNCHER) |
4050 | Invalid Request Parameter | File extension not supported. (support : ipa, zip, app, apk, xapk, apks) |
4051 | Invalid Request Parameter | The required parameter was not found in the request parameters: (app_file) |
4052 | Invalid Request Parameter | Invalid request parameter : (project_id : {PROJECT_ID}) |
4053 | Invalid Request Parameter | Invalid request parameter : (testset_id : {TESTSET_ID}) |
4055 | Invalid Request Parameter | Device is not compatible with this app : (App os type : {APP_OS_TYPE}) |
4080 | Invalid Request Parameter | Device info does not exist in preset project data. |
4081 | Invalid Request Parameter | Available devices don’t exist in preset project data. |
5001 | Run New Testset Error | Global Exception Error – {ERROR_MSG} |
5002 | Get Testset Status Error | Global Exception Error – {ERROR_MSG} |
5003 | Get Testset Result Data Error | Global Exception Error – {ERROR_MSG} |
6001 | App File Upload Error | App file upload failed. |
6002 | Get Testset Result Data Failed | Test is running yet. Please request when the test is complete. |
6003 | Data Does Not Exist | Test data does not exist. |
8000 | System Maintenance | The Service is currently under system maintenance. |