Skip to content

net.mbedtls: enable MBEDTLS_THREADING_C on Windows and macOS#27437

Merged
JalonSolov merged 3 commits into
vlang:masterfrom
quaesitor-scientiam:mbedtls-threading-windows-macos
Jun 15, 2026
Merged

net.mbedtls: enable MBEDTLS_THREADING_C on Windows and macOS#27437
JalonSolov merged 3 commits into
vlang:masterfrom
quaesitor-scientiam:mbedtls-threading-windows-macos

Conversation

@quaesitor-scientiam

@quaesitor-scientiam quaesitor-scientiam commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Enables mbedtls' internal locking (MBEDTLS_THREADING_C) on Windows and macOS;
previously it was on only for Linux/FreeBSD/OpenBSD.

Why

V wired up only mbedtls' pthread mutex backend, gated to __linux__ || __FreeBSD__ || __OpenBSD__, and #undef'd threading everywhere else. Windows
(MSVC) has no pthread.h, and macOS — which does have pthreads — was simply
not in the gate. With threading off, mbedtls does not mutex-protect its shared
state (the ctr_drbg RNG, RSA key blinding, library globals), so sharing an
mbedtls config/RNG across threads is unsafe on those platforms. That blocks, for
example, parallelizing TLS server handshakes across workers (the deferred
item #3 from #27433).

What

  • macOS: add __APPLE__ to the pthread gate — it has pthreads, so
    MBEDTLS_THREADING_PTHREAD works as-is.
  • Windows: enable MBEDTLS_THREADING_ALT and supply the four mutex callbacks
    over a Win32 CRITICAL_SECTION:
    • thirdparty/mbedtls/include/mbedtls/threading_alt.h — the
      mbedtls_threading_mutex_t type.
    • vlib/net/mbedtls/mbedtls_threading.h — the init/free/lock/unlock
      callbacks + a v_mbedtls_threading_setup() that calls
      mbedtls_threading_set_alt(). A no-op on every other platform.
    • Installed once from the new net.mbedtls module init(), before any TLS use.
  • Folded into thirdparty/mbedtls/mbedtls.patch so it survives the next mbedtls
    upgrade.

Builder: auto-invalidate thirdparty objects on header changes

The threading flip lives in include/mbedtls/mbedtls_config.h, but V's thirdparty
object cache only invalidated against the source .c and headers in the source
directory
(thirdparty/mbedtls/library/) — so an existing checkout silently
reused objects built with the old struct layout. The MSVC path was worse: it
reused a cached .obj whenever it merely existed, with no mtime check at all
(the root of the long-standing "-cc msvc: delete library/*.obj by hand"
workaround).

Per the discussion below (thanks @GGRei @JalonSolov), this is fixed generically:
both the C and MSVC paths now share one thirdparty_deps_mtime() that folds the
newest .h/.hpp mtime under the whole thirdparty module root (include/
included) into the cache key, memoized per module root. This is a general builder
improvement (it applies to every thirdparty module, not just mbedtls) — called
out here so it gets distinct review attention even though it's folded into this
PR. It is its own commit.

Verification

Platform / compiler Result
Windows (tcc) builds + links with THREADING_ALT; net.http TLS handshakes work (the mutex callbacks are exercised on every TLS op); server_tls_test.v unchanged from master (only the pre-existing net.shutdown idle-interrupt timeouts)
Windows (msvc) builds + links with THREADING_ALT; TLS handshake (new_ssl_conn) runs
Windows (gcc, mingw) builds + TLS handshake runs
Windows (clang, mingw) builds + TLS handshake runs
Linux unaffected (gate unchanged; shim/init no-op); server_tls_test.v passes
macOS (Apple Silicon, M-series) builds + server_tls_test.v passes

Builder cache invalidation (both object paths): editing
include/mbedtls/mbedtls_config.h rebuilds the 112 mbedtls objects and an
unchanged tree rebuilds nothing — verified on tcc (C path) and msvc (the
.obj path; confirmed via the per-object trace_thirdparty_obj_files signal,
since MSVC compiles through @response.rsp files). gcc/clang share the tcc C path.

Note for reviewers / local builds

Changing the threading config alters mbedtls struct layouts, so existing local
checkouts must rebuild the mbedtls objects. With the builder fix above this is now
automatic — pulling this branch updates mbedtls_config.h's mtime, which
invalidates the cached objects on the next build. v wipe-cache is no longer
required; it remains a safe manual fallback. Fresh clones and CI are unaffected.

This is a prerequisite for safe parallel TLS handshakes on Windows/macOS (#27433
item #3); it fixes no current bug on its own (today's single-context-per-thread
usage is already safe) but unblocks that work.

Closes #27436.

🤖 Generated with Claude Code

mbedtls' internal locking (around the shared RNG, RSA key blinding and
the library globals) was enabled only on Linux/FreeBSD/OpenBSD and
disabled on Windows and macOS, because V only wired up mbedtls' pthread
mutex backend and Windows (MSVC) has no pthreads. This blocks any design
that shares an mbedtls config/RNG across threads (e.g. parallel TLS
server handshakes).

- macOS: it has pthreads, so add it to the gate (it was simply omitted).
- Windows: enable MBEDTLS_THREADING_ALT and supply the four mutex
  callbacks over a Win32 CRITICAL_SECTION (vlib/net/mbedtls/
  mbedtls_threading.h + thirdparty threading_alt.h), installed once via
  mbedtls_threading_set_alt() from the net.mbedtls module init() before
  any TLS use. The setup is a no-op on non-Windows builds.

Folded into mbedtls.patch so it survives the next mbedtls upgrade.

Note: changing the threading config alters mbedtls struct layouts, so an
existing local build must rebuild the mbedtls objects (touch the sources
or delete the cached .o); fresh/CI builds are unaffected.

Verified on Windows: net.http TLS handshakes work with threading enabled
(the mutex callbacks are exercised on every TLS operation); the only
server_tls_test failures are the pre-existing net.shutdown idle-interrupt
ones, unchanged from master. macOS to be verified on Apple Silicon.

Closes vlang#27436

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@JalonSolov

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8853eed2cc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread thirdparty/mbedtls/include/mbedtls/mbedtls_config.h
@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Re: the Codex P2 about stale cached mbedtls objects — it's a real issue and I'd like the team's call on how to address it.

Root cause. Flipping the threading macros lives in include/mbedtls/mbedtls_config.h, but V's third-party object cache only invalidates on the source .c and sibling headers in the source dir (thirdparty/mbedtls/library/) — see latest_thirdparty_header_mtime / build_thirdparty_obj_file in vlib/v/builder/cc.v. So in an existing checkout that already built net.mbedtls, the cached threading.o (and the rest, whose struct layout changes with MBEDTLS_THREADING_C) are reused with the old no-threading layout. On Windows the new init() then links against a mbedtls_threading_set_alt the cached object never defined; on macOS the intended locking is silently absent. Fresh clones and CI are unaffected (objects are built from scratch with the new config); only pre-existing local builds are.

The PR currently documents the manual workaround (v wipe-cache, then rebuild). Codex (correctly) wants it automatic. Three ways to do that, each with trade-offs — which do you prefer?

  1. Builder fix (cc.v). Teach latest_thirdparty_header_mtime to also scan the module's sibling include/ tree (e.g. thirdparty/mbedtls/include), so config-header edits invalidate the cached objects.

    • ✅ Fixes the root cause generically and permanently (helps any future config change, and any third-party module that keeps headers under include/).
    • ⚠️ It's a compiler-build change: affects cache invalidation for all third-party objects, adds some stat overhead, and arguably belongs in its own PR rather than bundled with an mbedtls config change.
  2. Self-contained mbedtls touch. Modify a header under thirdparty/mbedtls/library/ as part of this PR so the existing source-dir scan invalidates all mbedtls objects on checkout (git sets the changed file's mtime to "now").

    • ✅ No compiler change; self-contained; forces the rebuild for everyone pulling this branch.
    • ⚠️ Indirect/non-obvious (the bump lives in a different dir than the actual change), relies on git updating mtimes, and is a one-off — it doesn't close the underlying cache gap, so the next config change would need the same trick.
  3. Document only (status quo) + follow-up. Keep the v wipe-cache note for existing checkouts (fresh/CI already fine), and do the builder auto-invalidation (option 1) as a separate follow-up PR.

    • ✅ Keeps this PR focused on the mbedtls config; no build-system change mixed in.
    • ⚠️ Contributors with an existing build hit a confusing link error until they wipe the cache.

I'm happy to implement whichever you pick (1 and 2 are both ready to go). My weak preference is 1 as the correct long-term fix, possibly split into its own PR per option 3 so this one stays scoped — but it's your call.

@GGRei

GGRei commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

For me, option 1 seems like the best long-term fix, but I think it should cover both current thirdparty object paths.

This is just my personal opinion:

For the V1 builder, fixing only the regular C path would not be enough. The MSVC path also has its own cached .obj reuse so both paths should share the same dependency/invalidation decision, even if the compile commands stay separate.

For this PR, the scope can stay focused: invalidate mbedTLS objects from the source file, sibling headers and the relevant mbedTLS include/config headers, especially thirdparty/mbedtls/include/mbedtls/mbedtls_config.h.
That fixes the stale-object problem directly instead of relying on v wipe-cache.

I would also like to add another important point in my opinion, version 2 of the V compiler.
From my point of view, it is important to prepare the transition as well. V2 already has CleanC/module/self-build caches, but I do not see a V1-equivalent persistent cache for thirdparty/external objects yet.

If V2 eventually owns that path later, the cache key should include source/header/config dependencies plus the target/build context. ( Still, this should be validated with Medvednikov first ).

Of course, not necessarily for this PR, but it could make the future V1 -> V2 migration smoother.

I hope this opinion is useful.

@JalonSolov

Copy link
Copy Markdown
Collaborator

I agree with @GGRei that option 1 would be best.

Objects should always be rebuilt from header only changes, as that type of change can affect the .c files, too.

@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Thanks @GGRei @JalonSolov — going with option 1 in this PR, scoped as you described, and covering both thirdparty object paths through a single shared decision so they can't drift.

Confirming the plan after reading the builder:

  1. Shared helper. A single thirdparty_deps_mtime(source_file) returning max(source mtime, newest header mtime under the thirdparty module root), called by both build_thirdparty_obj_file (C path) and build_thirdparty_obj_file_with_msvc (MSVC path). Same dependency set, one place to maintain.

  2. Scan the module root, not just the source dir. Today latest_thirdparty_header_mtime only scans the source file's own directory (thirdparty/mbedtls/library/), so edits to thirdparty/mbedtls/include/mbedtls/mbedtls_config.h — exactly what this PR does — never invalidate the cache. The helper will walk the module root (include/ included) for .h/.hpp, which is generic (no mbedtls-specific paths) and config-change-proof.

  3. The MSVC path needs a real check, not just a header scan. Right now it reuses the .obj if it merely exists (if os.exists(obj_path) { return }) — no mtime comparison against source or headers at all. That's the root of the long-standing "delete library/*.obj by hand" workaround on -cc msvc. The shared helper adds the full obj_mtime < deps_mtime comparison there too.

  4. Memoized per module root. mbedtls compiles many objects; recomputing a recursive header max-mtime per object would be O(objects×headers). I'll cache the result keyed by module root so it's computed once.

V2 cache work is out of scope here per @GGRei — agreed; that's for a separate discussion with Medvednikov.

Since this turns into a general builder change (cache invalidation for all thirdparty objects, not just mbedtls), I'll call it out as a distinct improvement in the PR description so it gets the right review attention even though it's folded in here. Implementing now.

Cached thirdparty objects were only invalidated against the source `.c`
and its sibling headers in the source directory (e.g.
thirdparty/mbedtls/library/). A change to a config header elsewhere in
the module — notably thirdparty/mbedtls/include/mbedtls/mbedtls_config.h,
which alters mbedtls struct layouts — left stale objects cached, so an
existing checkout silently reused the old layout (link errors on MSVC,
missing locking on macOS) until `v wipe-cache` was run by hand.

Worse, the MSVC path reused a cached `.obj` whenever it merely existed,
with no mtime comparison against the source or any header at all — the
root of the long-standing "-cc msvc: delete library/*.obj by hand"
workaround.

Both paths now share one decision via thirdparty_deps_mtime(), which
folds the newest `.h`/`.hpp` mtime under the whole thirdparty module root
(include/ included) into the cache key. The recursive header scan is
memoized per module root, so it runs once per module rather than once per
object. Source-less bundled objects keep their reuse-if-present behavior.

Verified (tcc): editing include/mbedtls/mbedtls_config.h now rebuilds the
mbedtls objects, an unchanged tree rebuilds nothing, and the build
re-stabilizes afterwards.

Co-Authored-By: WOZCODE <contact@withwoz.com>
@quaesitor-scientiam

Copy link
Copy Markdown
Contributor Author

Done — pushed in b96251d (its own commit).

Both paths now go through a shared thirdparty_deps_mtime() that folds the newest .h/.hpp mtime under the whole thirdparty module root (so include/mbedtls/mbedtls_config.h counts), memoized per root. The MSVC path additionally gained the real obj_mtime >= deps_mtime comparison it never had (source-less bundled .objs keep reuse-if-present).

Verified on tcc: editing include/mbedtls/mbedtls_config.h rebuilds the 112 mbedtls objects; an unchanged tree rebuilds nothing and re-stabilizes; vlib/net/mbedtls + vlib/net/ssl tests and compiler_errors_test.v pass. Updated the PR description to call this out as a distinct, general builder change and to note that v wipe-cache is no longer required (the cache self-invalidates). V2 cache work left out of scope as agreed.

@GGRei

GGRei commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Looks good to me. :)

This is only my opinion so it is up to you, one small regression test would make this cache fix safer long term. A fake thirdparty/<name> tree with a source under library/ and a config/header under include/, where touching the include header makes the cached object stale. If possible, it would be nice for the test to exercise the shared dependency decision used by both the regular C path and the MSVC object reuse path, without requiring a real MSVC toolchain.

If I were you and when you are done, chain @codex review until Codex validates with a 👍 on the PR opening post. It can help attract attention for the final review / merge on your PRs.

Good job!

@GGRei

GGRei commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Sorry about that, I was only giving an opinion and accidentally triggered Codex. ^^

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: b96251dcc7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…/ tree

Regression guard for the cache-invalidation fix: thirdparty objects must
rebuild when any header in the module changes, including config headers
under include/ (e.g. mbedtls_config.h), not just siblings of the .c.

- test_thirdparty_module_root_spans_library_and_include: the pure path
  helper resolves a library/ source to the whole thirdparty/<mod> root,
  normalizes separators, and falls back to the source dir off-tree.
- test_thirdparty_deps_mtime_includes_module_include_headers: a temp
  thirdparty/<mod>/{library,include} tree where an include/ header is
  newer than the .c; thirdparty_deps_mtime must reflect it. Verified to
  fail against the old source-dir-only scan. This helper is shared by the
  C and MSVC object paths, so it guards both.

Co-Authored-By: WOZCODE <contact@withwoz.com>
@JalonSolov

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 81ef524afa

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@JalonSolov JalonSolov merged commit 2085cbb into vlang:master Jun 15, 2026
86 of 93 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

net.mbedtls: enable MBEDTLS_THREADING_C on Windows and macOS (currently Linux/BSD only)

3 participants