Skip to content

test: propose focused component testing conventions - #3346

Open
nperez0111 wants to merge 6 commits into
a11y-testingfrom
testing/component-test-examples
Open

nperez0111 wants to merge 6 commits into
a11y-testingfrom
testing/component-test-examples

Conversation

@nperez0111

@nperez0111 nperez0111 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

A concrete proposal for how we write tests

This is how I would want tests to be written for BlockNote. I'd like to use this PR as a discussion starting point, rather than treating it as an already agreed standard.

The main idea is: test behavior beside its owner, at the lowest reasonable rung of the test ladder that can reliably prove it works. Use integrations to verify boundaries, not repeat dependencies' tests.

A bold button's tests explain its behavior and its integration with the editor. The toolbar's tests explain composition and traversal, assuming its children work. Rendering the component directly inside an editor gives it real context without bringing along incidental parent UI.

Conventions developed in the review

  • Use the right environment. Logic that doesn't need rendering stays in faster Node tests. Clicking, keyboard navigation, focus, and rendered component behavior need a real browser. Browser tests always run in Docker.
  • Use real internal dependencies. A real editor lets us check the result rather than only a mocked callback. Observation-only spies are fine; mock external dependencies when necessary.
  • Distinguish scope from machinery. Reading editor state to decide what a component displays is a unit contract. Clicking the component and asserting the resulting editor mutation exercises an integration. Neither needs a separate e2e directory just because it runs in a browser.
  • Keep applicable coverage together. One component-owned browser file, with groups ordered unit, integration, keyboard, a11y. Omit irrelevant groups, not meaningful coverage. ARIA and axe belong together under a11y; accessibility should not be a separate file that is easy to forget.
  • Cover user-important paths, not every branch. Parent tests don't repeat child contracts. Each control owns its activation and focus-ring checks; the toolbar owns traversal of its default controls and leaving the toolbar.
  • Make the test readable. Arrange, act, assert; repeated act/assert pairs are fine when clearer. Use explicit cases or it.for according to readability, not minimum line count.
  • Assert the relevant contract. Direct assertions (toEqual, or partial assertions when appropriate) express deliberate expected values and shapes. Prefer inline snapshots for small serialized documents/trees/results because they are reviewable beside the test. External snapshots suit larger output that would obscure the test. Size is guidance, not a hard cutoff.
  • Wait for observable readiness. Use retrying assertions or expect.poll, not arbitrary sleeps. Investigate local retries rather than using them as a substitute for readiness.
  • Treat accessibility as behavior. Check keyboard access, visible focus, exposed names/states, and axe results, including open menus and popovers. DOM accessibility trees do not replace native screen-reader testing.
  • Keep known defects explicit. Assert precise known axe violations rather than marking a whole scan as expected to fail. A narrow, documented it.fails can represent a known non-axe defect without masking working behavior.
  • Use screenshots deliberately. Prefer document state, DOM, or accessibility assertions for behavior. A few subsystem visual baselines can be useful, but before/after screenshots are not the default proof that a click worked.

The short decision guide and golden examples now live in the testing skill, with a component-testing reference. The new lint rules are opt-in for the eight toolbar example suites, not retroactive requirements on unrelated legacy tests.

Shared setup and helper boundaries

Tests receive helpers directly from Vitest's context: setupEditor, selectText, page, pressKey, focusUntil, readAriaSnapshot, readFocusOutline, assertFocusOutline, and runAxe.

  • setupEditor preserves custom-schema types and cleans up automatically. In the browser it mounts a StrictMode view, waits for fonts, and supports props, children, outside, and rerender. Skins and their styles load lazily, so core-only tests don't load Mantine CSS; Mantine, shadcn, and Ariakit are supported, with one skin per file.
  • The Node fixture uses the same setupEditor name and editor options, mounting the core editor in jsdom without React. For headless logic, use BlockNoteEditor.create directly. Both helpers return { editor, rootNode }.
  • Helpers that only need the DOM stay in tests/src/commands/browser; genuine automation operations stay in node. A small transport layer avoids import cycles. Native screen-reader commands retain their separate runner.
  • Commands accept locators where relevant. runAxe returns the full results so each test owns its violation assertions; its fixture annotates incomplete/manual-review findings. Helper signatures derive from implementations rather than a duplicated command interface.

Coverage in this PR

  • Text styles, alignment, nesting, colors, link creation, and block-type selection: relevant behavior, editor integrations, keyboard/focus, and accessibility.
  • Formatting toolbar: composition, selection preservation, complete forward/backward traversal in parent/child/sibling contexts, ARIA structure, and axe.
  • Desktop controller: extension-driven visibility, dismissal, focus/selection preservation, and portals.
  • Existing Heading and editor-view consumers adapted to the shared helper APIs.
  • Native screen-reader test decoupled from importing a browser test module.

There are no production behavior changes. Known defects remain visible: missing checked-state semantics in the Mantine color/block-type adapters, block-type editability reactivity, the link URL input's missing focus outline, and the precise known menu axe violations. The color menu's scrollability violation is checked when reported, since it depends on scrolling/focus state; unexpected violations still fail.

This is not a project-wide test migration or complete coverage of everything on a11y-testing. The examples establish meaningful component coverage and a reusable approach without requiring unrelated legacy suites to change.

Verification

  • Combined Docker run of all eight toolbar component suites plus the Heading and React editor-view consumers, across Chromium, Firefox, and WebKit, with --retry=0: 332 passed, 12 known expected failures, 25 existing skips; no unexpected failures.
  • All 18 core browser suites on Chromium, including the color regression called out in review: 185 passed, 1 existing skip, no retries.
  • Repository-wide lint: 0 errors, 10 pre-existing React warnings. Formatting passes for all changed files; unrelated pre-existing repository formatting failures are outside this PR.
  • Custom lint-rule tests: 13 passed.
  • Node setupEditor mounting, document updates, custom schemas, and cleanup were smoke-tested in jsdom.
  • Native screen-reader tests were not run; their existing skipped announcement case remains skipped.

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
blocknote Ready Ready Preview Nov 17, 2026 7:33pm UTC
blocknote-website Ready Ready Preview Nov 17, 2026 7:33pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 025c7765-b756-4364-93a7-3374839e9fdc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pkg-pr-new

pkg-pr-new Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@blocknote/ariakit

npm i https://pkg.pr.new/@blocknote/ariakit@3346

@blocknote/code-block

npm i https://pkg.pr.new/@blocknote/code-block@3346

@blocknote/core

npm i https://pkg.pr.new/@blocknote/core@3346

@blocknote/diagram-block

npm i https://pkg.pr.new/@blocknote/diagram-block@3346

@blocknote/mantine

npm i https://pkg.pr.new/@blocknote/mantine@3346

@blocknote/math-block

npm i https://pkg.pr.new/@blocknote/math-block@3346

@blocknote/react

npm i https://pkg.pr.new/@blocknote/react@3346

@blocknote/server-util

npm i https://pkg.pr.new/@blocknote/server-util@3346

@blocknote/shadcn

npm i https://pkg.pr.new/@blocknote/shadcn@3346

@blocknote/xl-ai

npm i https://pkg.pr.new/@blocknote/xl-ai@3346

@blocknote/xl-docx-exporter

npm i https://pkg.pr.new/@blocknote/xl-docx-exporter@3346

@blocknote/xl-email-exporter

npm i https://pkg.pr.new/@blocknote/xl-email-exporter@3346

@blocknote/xl-multi-column

npm i https://pkg.pr.new/@blocknote/xl-multi-column@3346

@blocknote/xl-odt-exporter

npm i https://pkg.pr.new/@blocknote/xl-odt-exporter@3346

@blocknote/xl-pdf-exporter

npm i https://pkg.pr.new/@blocknote/xl-pdf-exporter@3346

@blocknote/xl-typst-exporter

npm i https://pkg.pr.new/@blocknote/xl-typst-exporter@3346

commit: b014b3d

Comment thread tests/src/fixtures/editor.tsx Resolved
Comment thread tests/src/fixtures/editor.tsx Resolved

@matthewlipski matthewlipski left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, agree with the change to split the formatting toolbar tests further, it's cleaner this way.

Also like the renderEditor pattern, definitely more convenient for rendering individual items. I'm curious to see how it'll work for other components - I think for the side menu it would also be useful and not require changing the side menu buttons at all, though for the slash menu or file panel it may not be as clear cut. We'll cross that bridge when we get to it though.

Still have some comments r.e. organization of individual tests and test types. It seems like some components under DefaultButtons are also missing aria/axe tests, though maybe they're just not covered by this slice.


await button.click();

expect(editor.getBlock("format")?.content).toEqual([

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this not raise the issue again of UI tests having to know implementation details? I suppose it's pretty trivial for text styles, but it feels like this goes against the splitting of UI interaction and editor functionality. Like this is basically an e2e test imo.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this is an integration test. it is testing the integration of the button & the editor state. This acts as an e2e test but does not have to be in a separate test suite folder, instead it can be co-located, and try only a single trivial case rather than a number of situations

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, so in the case of the formatting toolbar, is there anything we'd wanna test e2e?

@nperez0111

Copy link
Copy Markdown
Contributor Author

Also like the renderEditor pattern, definitely more convenient for rendering individual items. I'm curious to see how it'll work for other components - I think for the side menu it would also be useful and not require changing the side menu buttons at all, though for the slash menu or file panel it may not be as clear cut. We'll cross that bridge when we get to it though.

Yep, there still may be some more setup for other component types, but at least this layer is settled.

Still have some comments r.e. organization of individual tests and test types. It seems like some components under DefaultButtons are also missing aria/axe tests, though maybe they're just not covered by this slice.

Yep, I commented on that here: #3346 (comment) I agree they may as well have those additional tests

@YousefED YousefED left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can scope tests much more narrowly than we have previously by rendering these buttons directly as children of the editor.

Nice find! seems to work well

Accessibility is part of whether the component works. Can someone reach it with a keyboard, see where focus is, and understand its current state? Axe catches another useful set of problems, but it can't answer all of those questions.

This seems like an argument to not have an a11y group, which I agree with. But it mostly feels like sth it figured out when you iterated on the testing design, as a reason to remove a11y groups. Not sure it should remain as a "principle" (assuming the principles will end up in a skill file or similar.

The examples are deliberately small. We want to demonstrate useful ways to test different behaviors, not repeat every possible combination or make each file contain every kind of test.
...
Shared setup takes care of creating and cleaning up the editor, leaving each test mostly as “set up this situation, do this, check the result.” Snapshots help when the whole output matters; direct assertions are clearer when checking one specific thing.
Browser tests run in Docker so everyone gets a consistent environment. Native screen-reader tests stay separate because checking actual spoken announcements is different from inspecting accessibility information in a browser.

same

Overall looks great (I'll do one more close look tomorrow), happy we're exploring this. Besides rolling this out further, I assume an important next step would be skill files / linting rules etc?

@YousefED YousefED left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did one more review with claude. Full report is here, but I also extracted the (imo) most relevant bits to PR comments; https://claude.ai/artifact/RMQ7tWsPr3NQU9aWae3R9u#284ea76f-727d.m294n8x5tb3.15233

Comment thread tests/src/utils/focus.ts Resolved
Comment thread tests/src/utils/commands.ts Resolved
Comment thread tests/src/fixtures/editor.tsx Resolved
Comment thread tests/src/fixtures/editor.tsx Resolved
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.2

QR code for preview link

🚀 View preview at
https://TypeCellOS.github.io/BlockNote/pr-preview/pr-3346/

Built to branch gh-pages at 2026-11-17 19:38 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

This branch was successfully deployed

2 active deployments
Preview – blocknote-website — 334515f2 Deployed Nov 17, 2026 by vercel[bot]
Preview – blocknote — 334515f2 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants