Repository navigation
test: propose focused component testing conventions - #3346
nperez0111 wants to merge 6 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
@blocknote/ariakit
@blocknote/code-block
@blocknote/core
@blocknote/diagram-block
@blocknote/mantine
@blocknote/math-block
@blocknote/react
@blocknote/server-util
@blocknote/shadcn
@blocknote/xl-ai
@blocknote/xl-docx-exporter
@blocknote/xl-email-exporter
@blocknote/xl-multi-column
@blocknote/xl-odt-exporter
@blocknote/xl-pdf-exporter
@blocknote/xl-typst-exporter
commit: |
matthewlipski
left a comment
There was a problem hiding this comment.
Overall looks good, agree with the change to split the formatting toolbar tests further, it's cleaner this way.
Also like the renderEditor pattern, definitely more convenient for rendering individual items. I'm curious to see how it'll work for other components - I think for the side menu it would also be useful and not require changing the side menu buttons at all, though for the slash menu or file panel it may not be as clear cut. We'll cross that bridge when we get to it though.
Still have some comments r.e. organization of individual tests and test types. It seems like some components under DefaultButtons are also missing aria/axe tests, though maybe they're just not covered by this slice.
|
|
||
| await button.click(); | ||
|
|
||
| expect(editor.getBlock("format")?.content).toEqual([ |
There was a problem hiding this comment.
Does this not raise the issue again of UI tests having to know implementation details? I suppose it's pretty trivial for text styles, but it feels like this goes against the splitting of UI interaction and editor functionality. Like this is basically an e2e test imo.
There was a problem hiding this comment.
Yep, this is an integration test. it is testing the integration of the button & the editor state. This acts as an e2e test but does not have to be in a separate test suite folder, instead it can be co-located, and try only a single trivial case rather than a number of situations
There was a problem hiding this comment.
I see, so in the case of the formatting toolbar, is there anything we'd wanna test e2e?
Yep, there still may be some more setup for other component types, but at least this layer is settled.
Yep, I commented on that here: #3346 (comment) I agree they may as well have those additional tests |
YousefED
left a comment
There was a problem hiding this comment.
We can scope tests much more narrowly than we have previously by rendering these buttons directly as children of the editor.
Nice find! seems to work well
Accessibility is part of whether the component works. Can someone reach it with a keyboard, see where focus is, and understand its current state? Axe catches another useful set of problems, but it can't answer all of those questions.
This seems like an argument to not have an a11y group, which I agree with. But it mostly feels like sth it figured out when you iterated on the testing design, as a reason to remove a11y groups. Not sure it should remain as a "principle" (assuming the principles will end up in a skill file or similar.
The examples are deliberately small. We want to demonstrate useful ways to test different behaviors, not repeat every possible combination or make each file contain every kind of test.
...
Shared setup takes care of creating and cleaning up the editor, leaving each test mostly as “set up this situation, do this, check the result.” Snapshots help when the whole output matters; direct assertions are clearer when checking one specific thing.
Browser tests run in Docker so everyone gets a consistent environment. Native screen-reader tests stay separate because checking actual spoken announcements is different from inspecting accessibility information in a browser.
same
Overall looks great (I'll do one more close look tomorrow), happy we're exploring this. Besides rolling this out further, I assume an important next step would be skill files / linting rules etc?
YousefED
left a comment
There was a problem hiding this comment.
did one more review with claude. Full report is here, but I also extracted the (imo) most relevant bits to PR comments; https://claude.ai/artifact/RMQ7tWsPr3NQU9aWae3R9u#284ea76f-727d.m294n8x5tb3.15233
|
A concrete proposal for how we write tests
This is how I would want tests to be written for BlockNote. I'd like to use this PR as a discussion starting point, rather than treating it as an already agreed standard.
The main idea is: test behavior beside its owner, at the lowest reasonable rung of the test ladder that can reliably prove it works. Use integrations to verify boundaries, not repeat dependencies' tests.
A bold button's tests explain its behavior and its integration with the editor. The toolbar's tests explain composition and traversal, assuming its children work. Rendering the component directly inside an editor gives it real context without bringing along incidental parent UI.
Conventions developed in the review
unit,integration,keyboard,a11y. Omit irrelevant groups, not meaningful coverage. ARIA and axe belong together undera11y; accessibility should not be a separate file that is easy to forget.it.foraccording to readability, not minimum line count.toEqual, or partial assertions when appropriate) express deliberate expected values and shapes. Prefer inline snapshots for small serialized documents/trees/results because they are reviewable beside the test. External snapshots suit larger output that would obscure the test. Size is guidance, not a hard cutoff.expect.poll, not arbitrary sleeps. Investigate local retries rather than using them as a substitute for readiness.it.failscan represent a known non-axe defect without masking working behavior.The short decision guide and golden examples now live in the testing skill, with a component-testing reference. The new lint rules are opt-in for the eight toolbar example suites, not retroactive requirements on unrelated legacy tests.
Shared setup and helper boundaries
Tests receive helpers directly from Vitest's context:
setupEditor,selectText,page,pressKey,focusUntil,readAriaSnapshot,readFocusOutline,assertFocusOutline, andrunAxe.setupEditorpreserves custom-schema types and cleans up automatically. In the browser it mounts a StrictMode view, waits for fonts, and supportsprops,children,outside, andrerender. Skins and their styles load lazily, so core-only tests don't load Mantine CSS; Mantine, shadcn, and Ariakit are supported, with one skin per file.setupEditorname and editor options, mounting the core editor in jsdom without React. For headless logic, useBlockNoteEditor.createdirectly. Both helpers return{ editor, rootNode }.tests/src/commands/browser; genuine automation operations stay innode. A small transport layer avoids import cycles. Native screen-reader commands retain their separate runner.runAxereturns the full results so each test owns its violation assertions; its fixture annotates incomplete/manual-review findings. Helper signatures derive from implementations rather than a duplicated command interface.Coverage in this PR
There are no production behavior changes. Known defects remain visible: missing checked-state semantics in the Mantine color/block-type adapters, block-type editability reactivity, the link URL input's missing focus outline, and the precise known menu axe violations. The color menu's scrollability violation is checked when reported, since it depends on scrolling/focus state; unexpected violations still fail.
This is not a project-wide test migration or complete coverage of everything on
a11y-testing. The examples establish meaningful component coverage and a reusable approach without requiring unrelated legacy suites to change.Verification
--retry=0: 332 passed, 12 known expected failures, 25 existing skips; no unexpected failures.setupEditormounting, document updates, custom schemas, and cleanup were smoke-tested in jsdom.