Writing and Running STS Tests

This guide explains how to write test cases using the STS framework and run them with tmt and Testing Farm.

Writing a Test Case

STS test cases follow pytest conventions with additional storage-specific fixtures and utilities. Here's a complete example (see also tests/iscsi/parameters/ for a real test):

1. Create the Test File

Create a Python test file (e.g., example-test.py) in your test directory, using sts-libs fixtures and utilities:

# Copyright: Contributors to the sts project
# SPDX-License-Identifier: GPL-3.0-or-later

from __future__ import annotations

import logging
from typing import TYPE_CHECKING

import pytest

from sts.fio.fio import FIO
from sts.utils.cmdline import run

if TYPE_CHECKING:
    from sts.loop import LoopDevice

logger = logging.getLogger(__name__)


@pytest.mark.parametrize('loop_devices', [1], indirect=True)
def test_example(loop_devices: list[LoopDevice]) -> None:
    """Test example using sts-libs features.

    This test:
    1. Uses loop_devices fixture to get a test device
    2. Verifies device exists using run()
    3. Runs I/O test using FIO
    """
    device = str(loop_devices[0].path)
    logger.info(f'Starting example test with device {device}')

    # Verify device exists
    result = run(f'lsblk {device}')
    assert result.succeeded, 'lsblk command failed'
    assert device.rsplit('/', maxsplit=1)[-1] in result.stdout

    # Run I/O test
    fio = FIO(filename=device)
    assert fio.run()

Here is another example — an LVM test that creates a logical volume on loop-backed storage and verifies basic operations:

from __future__ import annotations

from typing import TYPE_CHECKING

import pytest

from sts.lvm.logical_volume import LogicalVolume
from sts.utils.files import mkfs

if TYPE_CHECKING:
    from sts.lvm.volume_group import VolumeGroup


@pytest.mark.parametrize('loop_devices', [1], indirect=True)
class TestLvmBasic:
    """Basic LVM operations on loop-backed storage."""

    def test_create_and_format_lv(self, setup_loopdev_vg: VolumeGroup) -> None:
        """Create an LV, format it, and verify the report."""
        lv = LogicalVolume(name='testlv', vg=setup_loopdev_vg.name)
        lv.create(size='100M').assert_ok()
        lv.refresh_report()

        assert lv.report is not None
        assert lv.report.lv_name == 'testlv'

        assert mkfs(lv.path, 'ext4')

        lv.remove().assert_ok()

2. Create the FMF Metadata File

Create a main.fmf file alongside the test to let tmt discover it:

summary: My storage test example
description: |
  Detailed description of what the test validates.
  Include test setup, test steps, expected result and cleanup.
framework: shell
test: pytest ./example-test.py
component:
  - kernel
tag:
  - my-example
  - storage
tier: 1
duration: 10m
check:
  - how: avc
    result: xfail
  - how: dmesg
  - how: coredump

3. Create a tmt Test Plan

Create a plan file (e.g., example-plan.fmf) under plans/ that selects which tests to run and how to provision the test environment:

summary: My storage test plan
discover:
  - how: fmf
    filter:
      - tag:my-example
provision:
  how: virtual
  image: fedora
execute:
  how: tmt

Alternative Plan Configurations

For container testing:

provision:
  how: container
  image: centos:stream10

For specific hardware requirements (Beaker):

provision:
  how: beaker
  image: RHEL-9%
  hardware:
    disk:
      - size: '>= 10 GB'
    memory: '>= 4 GB'

Running Tests Locally

Using tmt (Recommended)

# Run a specific test plan
TMT_SHOW_TRACEBACK=full tmt run plan --name plans/iscsi/tier0 -dddvvv

# Login on failure for debugging
tmt run login --when fail --when error plans --name /plans/iscsi/tier0

Full documentation: tmt.readthedocs.io | Matrix: #tmt:fedora.im

Using Pytest Directly

SSH into the SUT, then:

dnf install -y python3-pip
pip install sts-libs

pytest example-test.py -v
pytest -o log_cli_level='debug' -vrA example-test.py

Running Tests with Testing Farm

# Basic submission
testing-farm request \
  --context distro=rhel-9 \
  --compose RHEL-9.7.0-Nightly \
  --git-url https://gitlab.com/rh-kernel-stqe/sts \
  --plan /plans/iscsi/tier0 \
  --arch x86_64,aarch64

# From your branch
testing-farm request \
  --compose Fedora-Rawhide \
  --git-url https://gitlab.com/your-username/sts \
  --git-ref your-feature-branch \
  --plan /plans/your-test-plan

Common Fixture Patterns

from __future__ import annotations

from typing import TYPE_CHECKING

import pytest

if TYPE_CHECKING:
    from sts.loop import LoopDevice
    from sts.lvm.volume_group import VolumeGroup


# Loop devices — parametrize count via indirect
@pytest.mark.parametrize('loop_devices', [2], indirect=True)
def test_with_loop_devices(loop_devices: list[LoopDevice]) -> None:
    device1, device2 = loop_devices
    # device1.path, device1.name, etc.


# Loop devices with custom size
@pytest.mark.parametrize('loop_devices', [{'count': 1, 'size_mb': 4096}], indirect=True)
def test_with_large_loop(loop_devices: list[LoopDevice]) -> None: ...


# LVM — setup_loopdev_vg creates a VG from loop devices
@pytest.mark.parametrize('loop_devices', [1], indirect=True)
def test_with_lvm(setup_loopdev_vg: VolumeGroup) -> None:
    vg = setup_loopdev_vg
    # vg.name, vg.create_lv(), etc.

Good Practices

• Descriptive test names: test functions should clearly indicate what they test
• Use fixtures: leverage sts-libs fixtures for storage setup/teardown
• Set realistic durations in main.fmf — tests that time out are hard to debug
• Start local: test with tmt run before submitting to Testing-Farm
• Use verbose mode: TMT_SHOW_TRACEBACK=full and -dddvvv for debugging

Troubleshooting

# Lint FMF metadata
tmt lint

# Verbose run with traceback
TMT_SHOW_TRACEBACK=full tmt run ... -dddvvv

# Dry run — show what would be executed
tmt run --dry

# Show discovered tests / plans
tmt tests ls
tmt plans ls

# Clean workdirs, guests and images
tmt clean