Getting Started
Pre-requisites
Make sure below pre-requisites are installed in your system.
python 3.7
virtualenv
Installation
It is recommended to use a python virtual environment to install the necessary dependencies and execute cephci.
Setup a Python virtualenv
python3 -m venv <path/to/venv>
source <path/to/venv>/bin/activate
Install the latest version
pip install --upgrade pip
Clone CephCI repository.
git clone https://github.com/red-hat-storage/cephci
cd cephci
Install cephci’s dependencies:
pip install -r requirements.txt
Initial Setup
Configure cephci.yaml
cephci.yaml file is used to allow configuration around a number of things within cephci. The template can be found at the top level of the repository, cephci.yaml.template. The required keys are in the template. Values are placeholders and should be replaced by legitimate values. Values for report portal or polarion are only required if you plan on posting to that particular service.
# cp cephci.yaml.template ~/.cephci.yaml
# cat cephci.yaml.template
report-portal:
endpoint: http://hostname:port
project: project_name
token: user_token
polarion:
endpoint: polarion_url
username: polarion_user_name
password: polarion_password
cdn_credentials:
username: user_name
password: user_password
stage_credentials:
username: user_name
password: user_password
email:
address: <email-address>
CentralCI Authentication
CentralCI auth files are kept in the osp directory.
The osp-cred-ci-2.yaml file has OpenStack credentials details to create/destroy resources. For local cephci runs, you will want to replace the username/password with your own OpenStack credentials.
globals:
openstack-credentials:
username: '<user_name>'
password: '<user_password>'
auth-url: 'https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000'
auth-version: '3.x_password'
tenant-name: 'ceph-ci'
service-region: 'regionOne'
domain: 'redhat.com'
tenant-domain-id: '62cf1b5ec006489db99e2b0ebfb55fx'
Execution
run.py is the main script for ceph-ci. You can view the full usage details by passing in the –help argument.
python run.py --help
Required Arguments
There are a few arguments that are required for cephci execution
- --rhbuild
RH Ceph build version ex., 4.2
- --platform
The OS platform deployed. Example: rhel-8
- --osp-cred
OpenStack credential file
- --global-conf
cluster configuration file
- --inventory
OS inventory file
- --suite
Test suite file
Useful Arguments
Some non-required arguments that we end up using a lot
- --log-level
set the log level that is output to stdout.
–store
store the deployed cluster information in a pickle file under cephci/rerun directory.
python run.py --osp-cred <cred_file> \
--global-conf <conf_file> \
--suite <suite_file> \
--inventory <inventory_file> \
--rhbuild <build_version> \
--platform rhel-<Major Version> \
--store
–reuse <file>
Reuse a deployed cluster details for running additional tests. The pickle must have been generated by cephci.
python run.py --osp-cred <cred_file> \
--global-conf <conf_file> \
--suite <suite_file> \
--inventory <inventory_file> \
--rhbuild <build_version> \
--reuse <rerun/issue122-1622530751458>
Examples
Ceph ansible install suite
python run.py --rhbuild 3.3 \
--global-conf conf/luminous/ansible/sanity-ansible-lvm.yaml \
--osp-cred osp/osp-cred-ci-2.yaml \
--inventory conf/inventory/rhel-7.8-server-x86_64.yaml \
--suite suites/luminous/ansible/sanity_ceph_ansible_lvm.yaml \
--platform rhel-7 \
--log-level info
Upgrade suite
python run.py --rhbuild 3.3 \
--global-conf conf/luminous/upgrades/upgrade.yaml \
--osp-cred osp/osp-cred-ci-2.yaml \
--inventory conf/inventory/rhel-7.8-server-x86_64-released.yaml \
--suite suites/luminous/upgrades/upgrades.yaml \
--platform rhel-7 \
--build tier-0 \
--log-level info
Containerized upgrade suite
python run.py --rhbuild 3.2 \
--global-conf conf/luminous/upgrades/upgrade.yaml \
--osp-cred osp/osp-cred-ci-2.yaml \
--inventory conf/inventory/rhel-7.8-server-x86_64-released.yaml \
--suite suites/luminous/upgrades/upgrades_containerized.yaml \
--log-level info \
--build tier-0 \
--insecure-registry \
--skip-version-compare
Using IBM-C Infrastructure as a Service platform
python run.py --rhbuild 5.1 \
--cloud ibmc \
--global-conf conf/pacific/rgw/tier-0_rgw.yaml \
--inventory ibm-vpc-rhel-8-latest.yaml \
--suite suites/pacific/rgw/tier-0_rgw.yaml \
--build latest \
--osp-cred osp/osp-cred-ci-2.yaml \
--log-level debug \
--platform rhel-8
Using User Provided Infrastructure (UPI)
python run.py --rhbuild 5.1 \
--cloud baremetal \
--cluster-conf <absolute-path-of-cluster> \
--suite suites/pacific/rgw/tier-0_rgw.yaml \
--build latest \
--log-level debug \
--platform rhel-8
Sample cluster-conf
$ cat cluster-conf.yaml
---
# Scale lab environment to be used for Ceph workload testing
globals:
- # List of clusters part of the test env.
ceph-cluster: # Ceph storage cluster deployment
name: "ceph" # String: The name of the cluster
networks: # network information
public: # Ceph public network
- "16.128.103.0/24" # CIDR notation is only supported
- "16.128.104.0/24" # Required when deploying in stretched mode.
- "16.128.105.0/24" # Arbitrary public network
cluster: # Ceph cluster network to be used by OSD
- "192.168.103.0/24" # CIDR notation is only support
- "192.168.104.0/24" # Required when deploying in stretched mode.
nodes:
- # list of nodes part of this cluster.
hostname: "ndoe1.test.local" # FQDN of the server
location: "site-01" # Optional - datacenter or rack meta details.
ip: # Public IPv4 address of the host
root_password: # Provide the root password (Optional)
root_private_key: # Private SSH key to be used for remote login
# It is optional if root_password is provided
role: # List of services to be deployed
- _admin
- installer
- mon
- mgr
volumes: # List of volumes to be used.
- /dev/sdb
- /dev/sdc
- /dev/sdd
Results
In order to post results properly or receive results emails you must first configure your ~/.cephci.yaml file. Please see the Initial Setup section of the readme if you haven’t done that.
Polarion
Results are posted to polarion if the –post-results argument is passed to run.py. When this argument is used, any tests that have a polarion-id configured in the suite will have it’s result posted to polarion.
Report Portal
Results are posted to report portal if the –report-portal argument is passed to run.py.
Email
A result email is automatically sent to the address configured in your ~/.cephci.yaml file. In addition to personally configured emails, if the –post-results or –report-portal arguments are passed to run.py an email will also be sent to cephci@redhat.com.
Cleanup
Ceph-CI also has the ability to manually clean up cluster nodes if anything was left behind during a test run. All you need to provide is your osp credentials and the instances name for the cluster.
For example, this command will delete all volumes and nodes that have the substring ceph-kdreyer in their names
python run.py --osp-cred <cred_file> --log-level info --cleanup ceph-kdreyer
Warning
Don’t use subset naming for custom instances name. Example: –instances-name vp and –instances-name vpoliset at same time.