Automation Detection

TestIntel can detect which inventory tests have corresponding automation code in a repository. This links test management (Zephyr/Xray/Jira) to test automation (pytest, Playwright, Jest, etc.) without requiring manual tagging.

How it works

Repo scan fetches test files from a GitHub repository

Test extraction parses test names from the code (function names, scenario names, etc.)

Fuzzy matching compares extracted names against inventory test names

Update sets automation_status: "Automated" and adds a repo external key for matched tests

Tests with no repo match remain unchanged (typically "Not Planned" or "Manual").

Supported frameworks

Language	Frameworks	File patterns	Extraction
Python	pytest, unittest	`test_.py`, `_test.py`	`def test_*` functions
JavaScript/TypeScript	Jest, Mocha, Playwright	`.spec.ts`, `.test.js`	`it('...')`, `test('...')`
Java	JUnit	`*Test.java`	`@Test` annotated methods
Go	testing	`*_test.go`	`func Test*` functions
C#	xUnit, NUnit	`Test.cs`, `Tests.cs`	`[Test]`, `[Fact]`, `[Theory]` methods
Gherkin	Cucumber, SpecFlow	`*.feature`	`Scenario:` lines

Usage

API


# Scan the default project's repo
curl -X POST "http://localhost:8000/sync/repo/scan" \
  -H "x-api-key: your-key"

# Scan a specific repo
curl -X POST "http://localhost:8000/sync/repo/scan?repo=owner/repo&branch=main" \
  -H "x-api-key: your-key"

# Adjust match threshold (0.0-1.0, default 0.65)
curl -X POST "http://localhost:8000/sync/repo/scan?threshold=0.7" \
  -H "x-api-key: your-key"

Response


{
  "scanned": 42,
  "tests_found": 18,
  "matched": 12,
  "updated": 12,
  "matches": [
    {
      "repo_test": "Login valid",
      "repo_path": "tests/test_auth.py",
      "inventory_id": "TC-0001",
      "inventory_name": "Test1: Login with valid credentials",
      "similarity": 78
    }
  ]
}

Other automation signals

Results import

When test results (JUnit XML) are imported via upload, webhook, or S3 watcher, any inventory test that matches a result is automatically set to automation_status: "Automated". If a test produced a result, it was executed by automation.

Manual tagging

Tests can also be manually set to Automated/Manual via the API:


curl -X PUT "http://localhost:8000/inventory/TC-0001" \
  -H "x-api-key: your-key" \
  -H "Content-Type: application/json" \
  -d '{"automation_status": "Automated"}'

Matching details

Python test names are humanized before matching: test_login_valid_credentials becomes "Login valid credentials" for comparison against inventory names like "Login with valid credentials"
Threshold controls minimum similarity (0.65 default). Lower = more matches but more false positives
External key is stored as repo: "path/to/test.py::test_name" for traceability
Tests already marked as "Automated" are not downgraded if they don't match a repo test

Prerequisites

code_context.provider: github in config.yaml
GITHUB_TOKEN environment variable or saved in Settings > API Keys & Tokens
A project with a configured repo, or pass ?repo=owner/repo explicitly