Take full reuse of Python's existing powerful libraries: Requests, unittest and Locust. And achieve the goal of API automation test, production environment monitoring, and API performance test, with a concise and elegant manner.
- Inherit all powerful features of
Requests, just have fun to handle HTTP in human way. - Define testcases in YAML or JSON format in concise and elegant manner.
- Supports
function/variable/extract/validatemechanisms to create full test scenarios. - Testcases can be run in diverse ways, with single testset, multiple testsets, or entire project folder.
- Test report is concise and clear, with detailed log records. See
PyUnitReport. - Perfect combination with Jenkins, running continuous integration test and production environment monitoring. Send mail notification with
jenkins-mail-py. - With reuse of
Locust, you can run performance test without extra work. - It is extensible to facilitate the implementation of web platform with
Flaskframework.
Background Introduction (中文版) | Feature Descriptions (中文版)
$ pip install git+https://github.com/debugtalk/ApiTestEngine.git#egg=ApiTestEngine --process-dependency-linksTo upgrade all specified packages to the newest available version, you should add the -U option.
$ pip install -U git+https://github.com/debugtalk/ApiTestEngine.git#egg=ApiTestEngine --process-dependency-linksIf there is a problem with the installation or upgrade, you can check the FAQ.
To ensure the installation or upgrade is successful, you can execute command ate -V to see if you can get the correct version number.
$ ate -V
jenkins-mail-py version: 0.2.5
ApiTestEngine version: 0.4.0
Execute the command ate -h to view command help.
$ ate -h
usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME]
[--failfast]
[testset_paths [testset_paths ...]]
Api Test Engine.
positional arguments:
testset_paths testset file path
optional arguments:
-h, --help show this help message and exit
-V, --version show version
--log-level LOG_LEVEL
Specify logging level, default is INFO.
--report-name REPORT_NAME
Specify report name, default is generated time.
--failfast Stop the test run on the first error or failure.
If you want to use ApiTestEngine with Jenkins, you may need to send mail notification, and jenkins-mail-py will be of great help.
To install mail helper, run this command in your terminal:
$ pip install -U git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py
With jenkins-mail-py installed, you can see more optional arguments.
$ ate -h
usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME]
[--failfast] [--mailgun-api-id MAILGUN_API_ID]
[--mailgun-api-key MAILGUN_API_KEY] [--email-sender EMAIL_SENDER]
[--email-recepients EMAIL_RECEPIENTS] [--mail-subject MAIL_SUBJECT]
[--mail-content MAIL_CONTENT] [--jenkins-job-name JENKINS_JOB_NAME]
[--jenkins-job-url JENKINS_JOB_URL]
[--jenkins-build-number JENKINS_BUILD_NUMBER]
[testset_paths [testset_paths ...]]
Api Test Engine.
positional arguments:
testset_paths testset file path
optional arguments:
-h, --help show this help message and exit
-V, --version show version
--log-level LOG_LEVEL
Specify logging level, default is INFO.
--report-name REPORT_NAME
Specify report name, default is generated time.
--failfast Stop the test run on the first error or failure.
--mailgun-api-id MAILGUN_API_ID
Specify mailgun api id.
--mailgun-api-key MAILGUN_API_KEY
Specify mailgun api key.
--email-sender EMAIL_SENDER
Specify email sender.
--email-recepients EMAIL_RECEPIENTS
Specify email recepients.
--mail-subject MAIL_SUBJECT
Specify email subject.
--mail-content MAIL_CONTENT
Specify email content.
--jenkins-job-name JENKINS_JOB_NAME
Specify jenkins job name.
--jenkins-job-url JENKINS_JOB_URL
Specify jenkins job url.
--jenkins-build-number JENKINS_BUILD_NUMBER
Specify jenkins build number.
It is recommended to write testcases in YAML format.
And here is testset example of typical scenario: get token at the beginning, and each subsequent requests should take the token in the headers.
- config:
name: "create user testsets."
import_module_functions:
- tests.data.custom_functions
variable_binds:
- user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)}
- os_platform: 'ios'
- app_version: '2.8.6'
request:
base_url: http://127.0.0.1:5000
headers:
Content-Type: application/json
device_sn: $device_sn
- test:
name: get token
request:
url: /api/get-token
method: POST
headers:
user_agent: $user_agent
device_sn: $device_sn
os_platform: $os_platform
app_version: $app_version
json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds:
- token: content.token
validators:
- {"check": "status_code", "comparator": "eq", "expected": 200}
- {"check": "content.token", "comparator": "len_eq", "expected": 16}
- test:
name: create user which does not exist
request:
url: /api/users/1000
method: POST
headers:
token: $token
json:
name: "user1"
password: "123456"
validators:
- {"check": "status_code", "comparator": "eq", "expected": 201}
- {"check": "content.success", "comparator": "eq", "expected": true}For detailed regulations of writing testcases, you can read the QuickStart documents.
ApiTestEngine can run testcases in diverse ways.
You can run single testset by specifying testset file path.
$ ate filepath/testcase.yml
You can also run several testsets by specifying multiple testset file paths.
$ ate filepath1/testcase1.yml filepath2/testcase2.yml
If you want to run testsets of a whole project, you can achieve this goal by specifying the project folder path.
$ ate testcases_folder_path
When you do continuous integration test or production environment monitoring with Jenkins, you may need to send test result notification. For instance, you can send email with mailgun service as below.
$ ate filepath/testcase.yml --report-name ${BUILD_NUMBER} \
--mailgun-api-id samples.mailgun.org \
--mailgun-api-key key-3ax6xnjp29jd6fds4gc373sgvjxteol0 \
--email-sender excited@samples.mailgun.org \
--email-recepients ${MAIL_RECEPIENTS} \
--jenkins-job-name ${JOB_NAME} \
--jenkins-job-url ${JOB_URL} \
--jenkins-build-number ${BUILD_NUMBER}
With reuse of Locust, you can run performance test without extra work.
$ ate-locust -V
Locust 0.8a2For full usage, you can run ate-locust -h to see help, and you will find that it is the same with locust -h.
The only difference is the -f argument. If you specify -f with a Python locustfile, it will be the same as locust, while if you specify -f with a YAML/JSON testcase file, it will convert to Python locustfile first and then pass to locust.
$ ate-locust -f examples/first-testcase.yml
[2017-08-18 17:20:43,915] Leos-MacBook-Air.local/INFO/locust.main: Starting web monitor at *:8089
[2017-08-18 17:20:43,918] Leos-MacBook-Air.local/INFO/locust.main: Starting Locust 0.8a2In this case, you can reuse all features of Locust.
Enjoy!
Python 2.7, 3.3, 3.4, 3.5 and 3.6.
ApiTestEngine has been tested on macOS, Linux and Windows platforms.
To develop or debug ApiTestEngine, you can install relevant requirements and use main-ate.py or main-locust.py as entrances.
$ pip install -r requirements_dev.txt
$ python main-ate -h
$ python main-locust -h