Skip to content

Tags: explosion/spaCy

Tags

release-v3.8.19

Toggle release-v3.8.19's commit message
Set version to v3.8.19

release-v3.8.18

Toggle release-v3.8.18's commit message
Set version to v3.8.18

release-v3.8.17

Toggle release-v3.8.17's commit message
Drop pip-based before-build step from cibuildwheel config

cibuildwheel 4 builds in a uv venv without pip, so the step failed with
'pip: command not found'. Builds are isolated, so installing
requirements.txt into the host venv isn't needed, and a fresh checkout
has no generated files to clean.

release-v3.8.16

Toggle release-v3.8.16's commit message
Remove publish_pypi workflow

Its trusted-publisher configuration no longer exists on PyPI, so it fails
on every release tag. Publishing is handled by a separate release process.

release-v3.8.15

Toggle release-v3.8.15's commit message
Install click with previous spaCy in upgrade test

spacy <=3.8.14 imports click without declaring it (#13971), and modern
typer no longer depends on click, so the pre-upgrade install fails on
import without it.

release-v3.8.14

Toggle release-v3.8.14's commit message
Set version to 3.8.14

release-v3.8.13

Toggle release-v3.8.13's commit message
Set version to 3.8.13

release-v3.8.12

Toggle release-v3.8.12's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fully migrate to Pydantic v2 (#13940)

Use confection v1.3 and Thinc v8.3.13, which implement custom validation logic in place of Pydantic, allowing us to properly adopt Pydantic v2 and provide full Python 3.14 support.

Our dependency tree used Pydantic v1 in unusual ways, and relied on behaviours that Pydantic v2 reformed. In the time since Pydantic v2 was released there were a few attempts to migrate over to it, but the task has been complicated by the fact that the confection library has a fairly tangled implementation and I had reduced availability for open-source work in 2024 and 2025.

Specifically, our library confection provides the extensible configuration system we use in spaCy and Thinc. The config system allows you to refer to values that will be supplied by arbitrary functions, that e.g. define some neural network model or its sublayers. The functionality in confection is complicated because we aggressively prioritised user experience in the specification, even if it required increased implementation complexity.

Confection's original implementation built a dynamic Pydantic v1 schema for function-supplied values ("promises"). We validate the schema before calling any promises, and then validate the schema again after calling all the promises and substituting in their values. The variable-interpolation system adds further difficulties to the implementation, and we have to do it all subclassing the Python built-in configparser, which ties us to implementation choices I'd do differently if I had a clean slate.

Here's one summary of Pydantic v1-specific behaviours that the migration to v2 particularly difficult for us. This particular summary was produced during a session with Claude Code Opus 4.6, so nuances of it might be wrong. The full history of attempts at doing this spans over different refactors separated by a few months at a time, so I don't have a full record of all the things that I struggled with. It's possible some details of this summary are incorrect though.

The core problem we kept hitting: Pydantic v2 compiles validation schemas upfront and has much stricter immutability. The whole session has been a series of workarounds for this:

```
 1. Schema mutation — v1 let you mutate __fields__ in place; v2 needs model_rebuild() which loses forward ref namespaces, or create_model subclasses which don't propagate to parent schemas.
 2. model_dump vs dict — v2 converts dataclasses to dicts, breaking resolved objects. Needed a custom _model_to_dict helper.
 3. model_construct drops extras — v2 silently drops fields with extra="forbid", needed manual workarounds.
 4. Strict coercion — v2 coerces ndarray to List[Floats1d] via iteration, needed strict=True.
 5. Forward refs — Every schema with TYPE_CHECKING imports needs model_rebuild() with the right namespace, and that breaks when confection re-rebuilds later.
In order to adjust for behavioural differences like this, I'd refactored confection to build the different versions of the schema in multiple passes, instead of building all the representations together as we'd been doing. However this refactor itself had problems, further complicating the migration.
```

~I've now bitten the bullet and rolled back the refactor I'd been attempting of confection, and instead replaced the Pydantic validation with custom logic. This allows Confection to remove Pydantic as a dependency entirely.~ Update: Actually I went back and got the refactor working. All much nicer now.

I've taken some lengths to explain this because migrating off a dependency after breaking changes can be a sensitive topic. I want to stress that the changes Pydantic made from v1 to v2 are very good, and I greatly appreciate them as a user of FastAPI in our services. It would be very bad for the ecosystem if Pydantic pinned themselves to exactly matching the behaviours they had in v1 just to avoid breaking support for the sort of thing we'd been doing. Instead users who were relying on those behaviours like us should just find some way to adapt --- either vendor the v1 version we need, or change our behaviours, or implement an alternative. I would have liked to do this sooner but we've ultimately gone with the third option.

release-v3.8.11

Toggle release-v3.8.11's commit message
Skip Python 3.10 on Windows ARM

release-v3.8.10

Toggle release-v3.8.10's commit message
Windows arm needs to be disabled at the ci level, so remove this skip…

… selector