Add hatch/hatchling packaging support with configurable default build tool

[Copilot]

The stubber packaging pipeline was locked to Poetry with no way to adopt modern PEP 517 build backends. This PR introduces a pluggable build-tool architecture supporting both Poetry (default, backward-compatible) and Hatchling, with the default driven by a single config entry rather than hardcoded in multiple places.

New PackageType enum & HatchBuilder

• PackageType.POETRY / PackageType.HATCH in publish/enums.py
• HatchBuilder(Builder) in stubpackage.py — mirrors PoetryBuilder but drives hatch build / hatch publish, writes [tool.hatch.build.targets.wheel] include instead of listing individual .pyi files in [tool.poetry] packages, and reads from the new pyproject_hatch.toml template

StubPackage dispatch

StubPackage retains PoetryBuilder inheritance for backward compat but now accepts package_type and overrides create_update_pyproject_toml(), update_pyproject_stubs(), build_distribution(), and publish_distribution() to route to the correct builder:

# existing code unchanged
StubPackage("micropython-esp32-stubs", port="esp32", version="1.24.1")

# opt into hatchling
StubPackage("micropython-esp32-stubs", port="esp32", version="1.24.1",
            package_type=PackageType.HATCH)

Config-driven default

The default build tool lives solely in StubberConfig.package_type — there is no hardcoded PackageType.POETRY anywhere else:

# pyproject.toml — opt the whole project into hatch
[tool.micropython-stubber]
package-type = "hatch"

CONFIG.package_type (default PackageType.POETRY) is used as the default for StubPackage.__init__, create_package, get_package, build_multiple, and publish_multiple. show-config now prints it.

CLI

Both stubber build and stubber publish gain --package-type [poetry|hatch] (default sourced from config):

stubber build --package-type hatch --port rp2 --version 1.24.1
stubber publish --package-type hatch --port rp2 --version 1.24.1

Templates

New publish/template/pyproject_hatch.toml for hatchling packages — [project] + [build-system] with hatchling, [tool.hatch.build.targets.wheel] include populated at build time via glob-friendly patterns instead of individual file listings.

Database storage of build backend

The packages database now stores package_type alongside every published package record:

• New DB schema includes package_type TEXT DEFAULT 'poetry'
• _migrate_add_package_type() runs on every get_database() call, adding the column to pre-existing databases transparently (old rows default to 'poetry')
• from_dict() restores the correct backend when loading a package from the DB; for legacy rows that predate the column, _infer_package_type() auto-detects the backend by reading [build-system].build-backend from the on-disk pyproject.toml

This prevents a correctness bug where reloading a hatch-built package under the Poetry default would attempt to access [tool.poetry] keys in a hatchling-format pyproject.toml.

Build backend comparison

For stub-only packages, Hatchling is recommended for new packages:

	Poetry	Hatchling	Flit	Setuptools
PEP 617/621 compliant	Partial	✅ Full	✅ Full	✅ Full
Glob includes for .pyi files	❌ Each file listed individually	✅ **/*.pyi patterns	❌ Module-centric	✅ via MANIFEST
Stub-only packages	Works but verbose	✅ Natural fit	❌ Requires importable module	✅ Extra config needed
Integrated publish	✅	✅	✅	❌ needs twine

Poetry remains the default so all existing published packages are unaffected.

Original prompt

create a plan to start using hatch in stead of poetry to do the packaging of new stubs.
in order to maintain backwards compatible the poetry package format should also be support in order to be able to rebuild/ update existing packages

the 'stubber build' command should have an option to indicate the package type , initialy default to poetry.

additional research is needed if hatch, or another build tool is best suited for packaging stub-only packages.
please compare and recommend the best solution.
the architecture should support multiple package formats from here forward.

don't forget the templates and substitutions needed.

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

[Josverl]

@ copilot, do not hardcode the default package type in different places. Put it in the config

[Josverl]

@copilot is the a need or benefit to store the build backend of the package on the database?

Also you have not done or at least reported on the fit of different package backendd for this as use.
Please do that noey

[Copilot]

@ copilot, do not hardcode the default package type in different places. Put it in the config

Done in commit 55d840c. PackageType.POETRY is now the default only in StubberConfig.package_type in config.py. All function signatures (StubPackage.__init__, create_package, get_package, build_multiple, publish_multiple) and CLI option defaults reference CONFIG.package_type — users can override globally with package-type = "hatch" in [tool.micropython-stubber].

@copilot is the a need or benefit to store the build backend of the package on the database?

Also you have not done or at least reported on the fit of different package backendd for this as use....

Database storage (commit 0b4c224)

Yes, there is a concrete need. from_dict() immediately calls create_update_pyproject_toml() when reconstructing a package from a DB row. Without the stored backend, a hatch-built package reloaded under the Poetry default would try to access [tool.poetry] keys in a hatchling pyproject.toml — causing either a KeyError or silent format corruption.

Changes made:

• package_type TEXT DEFAULT 'poetry' added to the DB schema
• _migrate_add_package_type() runs on every get_database() call, adding the column to existing databases without breaking them (old rows default to 'poetry')
• to_dict() includes package_type; the INSERT in publish_distribution() stores it
• from_dict() reads it back; for legacy rows that predate the column, _infer_package_type() auto-detects the backend from [build-system].build-backend in the on-disk pyproject.toml

---

Build backend comparison for stub-only packages

	Poetry	Hatchling	Flit	Setuptools
PEP 517/621 compliant	Partial ([tool.poetry] alongside [project])	✅ Full	✅ Full	✅ Full
Glob patterns for includes	❌ Each .pyi file must be listed individually in packages = [...]	✅ include = ["**/*.pyi"]	❌ Module-centric	✅ via find_packages/MANIFEST
Stub-only packages	Works but verbose	✅ Natural fit	❌ Requires importable module	✅ Works with extra config
sdist + wheel in one command	✅ poetry build	✅ hatch build	✅ flit build	✅ python -m build
Upload without separate tool	✅ poetry publish	✅ hatch publish	✅ flit publish	❌ needs twine
Maintenance / PyPA alignment	Maintained; non-standard table	Actively developed; PyPA recommended	Maintained	Maintained; legacy toolchain

Recommendation: Hatchling is the best fit going forward.

Key reasons for stub-only packages specifically:

1. Glob includes — instead of listing 100+ .pyi files individually (current Poetry approach), hatchling accepts include = ["*.pyi", "**/*.pyi"] in [tool.hatch.build.targets.wheel]
2. Standard [project] table — the existing template already used PEP 621 [project]; the [build-system] line was the only mismatch (poetry-core backend against a standard project table)
3. No module requirement — flit requires an importable top-level module, making it unsuitable for distributed stub packages
4. Simpler publish workflow — hatch publish works without configuring a separate publisher like twine

Poetry remains the backward-compatible default so existing published packages are unaffected. New packages or projects that opt in via package-type = "hatch" in pyproject.toml get the cleaner hatchling build.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/repositories/15337142/tags
  • Triggering command: /home/REDACTED/work/micropython-stubber/micropython-stubber/.venv/bin/pytest /home/REDACTED/work/micropython-stubber/micropython-stubber/.venv/bin/pytest tests/publish/test_hatch_builder.py tests/publish/test_package.py -v --tb=short --no-cov (http block)
  • Triggering command: /home/REDACTED/work/micropython-stubber/micropython-stubber/.venv/bin/pytest /home/REDACTED/work/micropython-stubber/micropython-stubber/.venv/bin/pytest tests/publish/test_hatch_builder.py -v --tb=short --no-cov (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Josverl]

@copilot
please rebase this on branch: main

for the folders mip/** and src/stubber/board ignore any changes in this branch , and prefer the changes in the main branch

[Copilot]

@copilot
please rebase this on branch: main

...

Done in commit 46da93e. Rebased onto main by cherry-picking only the 3 hatch-specific commits (onto the latest main tip 8203167). The mip/** and src/stubber/board files are now taken entirely from main. The only files that differ from main in this branch are the hatch packaging additions (src/stubber/publish/, src/stubber/commands/, src/stubber/utils/config.py, and the new test/template files).

[codecov]

Codecov Report

❌ Patch coverage is 47.05882% with 81 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.76%. Comparing base (db70f07) to head (27827ce).

Files with missing lines	Patch %	Lines
src/stubber/publish/stubpackage.py	36.36%	74 Missing and 3 partials ⚠️
src/stubber/publish/publish.py	50.00%	2 Missing ⚠️
src/stubber/commands/build_cmd.py	66.66%	1 Missing ⚠️
src/stubber/publish/package.py	66.66%	1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (47.05%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #722      +/-   ##
==========================================
- Coverage   72.36%   69.76%   -2.60%     
==========================================
  Files          71       71              
  Lines        8399     8540     +141     
  Branches     1541     1555      +14     
==========================================
- Hits         6078     5958     -120     
- Misses       1823     2102     +279     
+ Partials      498      480      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Josverl]

@copilot
|There are CI Erros.|

|Please investigate , and fix ( similar database test issues have been seen and fixed before) |also make sure to run a full test locally before comitting.|

Error:|
=================================== FAILURES ===================================
_____________________________ test_publish_package _____________________________

mocker = <pytest_mock.plugin.MockerFixture object at 0x10faf3ba0>
tmp_path = PosixPath('/private/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/pytest-of-runner/pytest-0/test_publish_package0')
pytestconfig = <_pytest.config.Config object at 0x1039e9e80>
fake_package = micropython-fake-stubs==1.19.1
test_db_conn = <sqlite3.Connection object at 0x110b0d990>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x1109de9c0>

@pytest.mark.integration
def test_publish_package(
    mocker: MockerFixture,
    tmp_path: Path,
    pytestconfig: pytest.Config,
    fake_package: StubPackage,
    test_db_conn: sqlite3.Connection,
    monkeypatch: pytest.MonkeyPatch,
):
    # Set Poetry to not create virtual environments during build
    monkeypatch.setenv("POETRY_VIRTUALENVS_CREATE", "false")
    pkg = fake_package

    m_publish: MagicMock = mocker.patch("stubber.publish.package.StubPackage.poetry_publish", autospec=True, return_value=True)

    # allow the publishing logic to run during test
    pkg._publish = True  # type: ignore

    # FIXME : dependency to access to test.pypi.org

  result = pkg.publish_distribution_ifchanged(production=False, force=False, db_conn=test_db_conn)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/publish/test_publish.py:143:

---

src/stubber/publish/stubpackage.py:1342: in publish_distribution_ifchanged
return self.publish_distribution(dry_run, production, db_conn)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

---

self = micropython-fake-stubs==1.19.1, dry_run = False, production = False
db_conn = <sqlite3.Connection object at 0x110b0d990>

def publish_distribution(
    self,
    dry_run: bool,
    production: bool,
    db_conn: sqlite3.Connection,
) -> bool:
    """
    Publishes the package to PyPi or Test-PyPi.

    Args:
        dry_run (bool): If True, performs a dry run without actually publishing.
        production (bool): If True, publishes to PyPi. If False, publishes to Test-PyPi.
        db: The database object to save the package state.

    Returns:
        bool: True if the publish was successful, False otherwise.
    """
    self.update_hashes()  # resets is_changed to False
    if not dry_run:
        if self.package_type == PackageType.HATCH:
            pub_ok = HatchBuilder.hatch_publish(self, production=production)  # type: ignore[arg-type]
        else:
            pub_ok = self.poetry_publish(production=production)
    else:
        log.warning(f"{self.package_name}: Dry run, not publishing to {'' if production else 'Test-'}PyPi")
        pub_ok = True
    if not pub_ok:
        log.warning(f"{self.package_name}: Publish failed for {self.pkg_version}")
        self.status["error"] = "Publish failed"
        return False
    self.status["result"] = "Published to PyPi" if production else "Published to Test-PyPi"
    self.update_hashes()
    if dry_run:
        log.warning(f"{self.package_name}: Dry run, not saving to database")
    else:
        cursor = db_conn.cursor()

        d = self.to_dict()

      cursor.execute(

            """
            INSERT INTO packages (name, description, mpy_version, pkg_version, publish, stub_sources, path, hash, stub_hash, port, board, variant, package_type)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
            """,
            (
                d["name"],
                d["description"],
                d["mpy_version"],
                d["pkg_version"],
                d["publish"],
                d["stub_sources"],
                d["path"],
                d["hash"],
                d["stub_hash"],
                d["port"],
                d["board"],
                d["variant"],
                d["package_type"],
            ),
        )

E sqlite3.OperationalError: table packages has no column named package_type

src/stubber/publish/stubpackage.py:1391: OperationalError
=============================== warnings summary ===============================
tests/createstubs/createstubs_all_test.py: 100 warnings
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/mock/mock.py:494: DeprecationWarning: 'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead
if iscoroutinefunction(getattr(spec, attr, None)):

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0e4d0>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0e980>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0d7b0>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0e7a0>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0ea70>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0d6c0>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0ec50>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/rst/test_rst.py::test_rst_all
/Users/runner/work/micropython-stubber/micropython-stubber/.venv/lib/python3.14/site-packages/pyflakes/checker.py:694: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x110b0ed40>
def in_annotation(func):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform darwin, python 3.14.4-final-0 _______________

Coverage XML written to file results/coverage.xml
=========================== short test summary info ============================
FAILED tests/publish/test_publish.py::test_publish_package - sqlite3.OperationalError: table packages has no column named package_type
= 1 failed, 795 passed, 41 skipped, 6 xfailed, 14 xpassed, 108 warnings in 249.28s (0:04:09) =

[Copilot]

@copilot
|There are CI Erros.|

...

Fixed in commit a99c23b. The failure was sqlite3.OperationalError: table packages has no column named package_type in test_publish_package. The test_db_conn fixture was creating a raw SQLite connection that bypassed get_database(), so _migrate_add_package_type() (which adds the new column) was never called on the test database. Added _migrate_add_package_type(db_conn) to the fixture after the connection is created — matching exactly how get_database() handles it.