Testing

Chef runs tests in a real browser via Playwright. Two types of tests are supported: unit tests (Mocha + Chai) and E2E tests (Playwright Test API). E2E tests can be written for a single extension or at the module level — for scenarios that span several extensions.

Setup

Initialize the test environment:

chef init tests

This creates two files in the project root:

File	Description
playwright.config.ts	Playwright config for running unit and E2E tests in browser
.env.test	Credentials for automatic authentication during tests

Fill in your local Bitrix installation credentials:

BASE_URL=http://localhost
LOGIN=admin
PASSWORD=your_password

Variable	Description
BASE_URL	URL of your local Bitrix installation
LOGIN	Test user login
PASSWORD	Test user password
MOCHA_WRAPPER	Path or full URL of the unit-test runner page. Defaults to /dev/ui/cli/mocha-wrapper.php

Set MOCHA_WRAPPER when your installation serves the unit-test runner page at a different address — for example, when main/dev/public is mounted under another prefix:

MOCHA_WRAPPER=/internal/dev/ui/cli/mocha-wrapper.php

WARNING

Do not commit .env.test to version control — it contains sensitive credentials.

Install Playwright browsers:

npx playwright install

IDE Types

mocha, chai and their types are included in Chef and used when running chef test. For IDE autocompletion, install the types locally:

npm install --save-dev @types/mocha @types/chai @playwright/test

Running Tests

# All tests for an extension
chef test vendor.my-extension

# Unit tests only
chef test unit vendor.my-extension

# E2E tests only
chef test e2e vendor.my-extension

# Module-level scenario tests (several extensions)
chef test module crm

# Specific file
chef test unit vendor.my-extension ./utils.test.ts

# Tests matching pattern
chef test vendor.* --grep "should render"

# List tests without running them
chef test vendor.my-extension --list

# Watch mode — rerun on changes
chef test vendor.my-extension -w

Live output

During a run chef shows a single per-browser status bar — one look from start to finish. While an engine warms up you see its stage; once tests start, a counter and the test currently running:

○ Chromium starting  ·  ○ Firefox  ·  ○ WebKit · 0.7s
○ Chromium 12/132  ·  ○ Firefox preparing  ·  ○ WebKit 8/132 · 5.3s
✓ Chromium 132/132  ·  ○ Firefox 90/132  ·  ○ WebKit building · 12.1s

Results are grouped by describe block: the suite path is a heading printed once, tests indented beneath it. A test that ran in several browsers is one line tagged with every engine (◌ — still running):

   ui.notification > Position
     ✓ has TOP_LEFT     [Chromium ✓ · Firefox ✓ · WebKit ◌]
     ✗ has BOTTOM_RIGHT  [Chromium ✗ · Firefox ✓ · WebKit ✓]

Tests that passed only after a retry (Playwright's retry) are flagged as flaky:

     ✓ opens the dialog  (passed on attempt 2)   [Chromium ✓]

The final summary counts flaky tests separately — after the total, because they are already counted in passed and are not a fourth category:

   Extensions  1 passed (1)
   Tests       130 passed (130) 2 flaky

   Flaky tests: failed at first, passed on a retry
     ↻ ui.dialog › Sharing > opens the dialog
     ↻ ui.dialog › Sharing > closes on Escape

Each flaky test is named and tagged with the extension it came from — a bare 2 flaky would give you no way to find them.

A green run with retries is weak evidence

On the first (failed) attempt Playwright writes a missing reference screenshot, and the retry then compares against a baseline this very run produced. To take a baseline deliberately: run with --ignore-snapshots first — that proves the assertions pass — and only then --update-snapshots.

Mismatch with the selected test count

The printed categories must add up to the number of tests the runner selected. If a test reported no result at all, chef prints Mismatch and fails the run — a summary that does not cover the whole set proves nothing, and exiting 0 on it is not an option:

   Tests       6 passed (6)
   Mismatch    5 unreported — selected tests with no result

The count is in test runs, not unique tests: one test in three engines is three runs. So the deduplication in the output (4 passed for 12 runs) is not a mismatch by itself.

Debugging

# Open browser with DevTools
chef test vendor.my-extension --debug

# With visible browser window
chef test vendor.my-extension --headed

# In a specific browser
chef test vendor.my-extension --project chromium

In --debug mode, source maps are enabled and DevTools are opened — you can set breakpoints directly in your TypeScript source code.

Listing without running

--list enumerates the tests that would run, without running them — handy for seeing what a suite contains or picking a pattern for --grep:

chef test vendor.my-extension --list

Tests are grouped by describe block, with deferred ones (skip/fixme) marked. A Summary at the end breaks the counts down per kind — unit and e2e separately (a module has e2e only) — with the number of tests and how many run / are skipped:

  Summary
  Unit  132 tests · 132 runnable
  E2E   25 tests · 25 runnable

Works for extensions (unit + e2e), modules (chef test module) and every reporter (--reporter default|json|teamcity). It can't be combined with --watch.

Playwright arguments

chef has its own options and deliberately does not mirror the whole Playwright CLI. Anything chef does not claim as its own goes to the runner verbatim:

# Update reference snapshots
chef test e2e vendor.my-extension --update-snapshots

# Run a test several times in a row (hunting flakes)
chef test e2e vendor.my-extension --repeat-each=3 --workers=1

# Collect a trace to investigate a failing test
chef test e2e vendor.my-extension --trace=on

The common ones are listed in chef test e2e --help, but that list is not exhaustive: every Playwright option works, including one added in a fresh runner release — no chef update needed.

This applies to e2e tests — unit tests have no separate runner process to pass options to, and chef says so explicitly.

chef keeps the options that drive chef itself: --watch, --path, --console, --list, --reporter, --cdp-port, plus --headed, --debug, --grep and --project, which chef translates into runner arguments of its own. Everything else belongs to Playwright.

A typo in an option name is caught by the runner, usually with a suggestion:

error: unknown option '--headles'
(Did you mean --headed?)

Forwarded arguments are appended last, so on a conflict yours win over what chef set.

Calling Playwright directly

If you need to run Playwright by hand, bypassing chef, use the project binary:

./node_modules/.bin/playwright test <spec>

The binary shipped with chef won't do: @playwright/test then loads twice — once from chef's tree, once from the project's — and the run fails with:

Playwright Test did not expect test.describe() to be called here
You have two different versions of @playwright/test

chef test itself never hits this trap — it always starts the runner from the project root. If the Playwright versions in the project and in chef have drifted apart, an e2e run prints a warning showing both versions and the reminder to use the project binary.

Bulk Runs

chef test without arguments or with a glob pattern (im.v2.**) walks through every matching extension. Extensions without tests are skipped silently — only those with tests or problems show up in the output.

Browser console output is hidden by default to keep bulk reports clean. Add --console if you need it:

chef test im.v2.** --console

Task Statuses

Chef shows the failure reason inline next to each extension:

Status	Meaning
✓ Unit tests	All tests passed
✗ Unit tests (3 failed)	Some tests failed, the number is how many
✗ Unit tests (build failed)	The test bundle did not compile (Rollup)
✗ Unit tests (crashed before any tests ran)	Crashed before the first it — usually a setup error
⚠ Unit tests (no tests collected)	Files exist but Mocha found no it (empty describe, .skip)
— Unit tests (no test files)	No *.test.{ts,js} in the test/unit/ (or test/) directory
— E2E tests (no test files)	Same for e2e

An extension or module with no tests is marked skipped (not passed) — counted separately in the summary.

Final Summary

After the run chef prints an aggregated report:

• Failed Tests (N) — all failed tests with stack traces and code frames, grouped by extension. For e2e, paths to Playwright artifacts (screenshot, video, trace) are printed alongside, grouped by browser — ready to open in your editor.
• Errors (N) — build errors and runtime crashes, one line per cause.
• Issues — list of extensions with error/warning counts.
• Extensions / Tests / Time — totals: how many extensions and tests passed, failed or were skipped, how long it took. The Tests line flags flaky tests (passed after a retry) as a separate count. For chef test module the line is labeled Modules.

Tips

Test Isolation

Each test should be independent. Use beforeEach/afterEach for setup and cleanup:

describe('TodoList', () => {
  let list: TodoList;

  beforeEach(() => {
    list = new TodoList();
  });

  afterEach(() => {
    list.destroy();
  });

  it('should add item', () => {
    list.add('Buy milk');
    assert.equal(list.getCount(), 1);
  });

  it('should start empty', () => {
    assert.equal(list.getCount(), 0);
  });
});

Test Organization

Group tests by functionality:

describe('UserService', () => {
  describe('create', () => {
    it('should create user with valid data', () => { /* ... */ });
    it('should throw on duplicate email', () => { /* ... */ });
  });

  describe('update', () => {
    it('should update user name', () => { /* ... */ });
    it('should not allow empty name', () => { /* ... */ });
  });

  describe('delete', () => {
    it('should soft delete user', () => { /* ... */ });
  });
});