CLI, so many bugfixes, block structured systems, mhom exposed, user homotopy, two-level parallelism...#238
Open
ofloveandhate wants to merge 274 commits into
Open
CLI, so many bugfixes, block structured systems, mhom exposed, user homotopy, two-level parallelism...#238ofloveandhate wants to merge 274 commits into
ofloveandhate wants to merge 274 commits into
Conversation
…espace Lifts sin, cos, tan, asin, acos, atan, exp, log, and sqrt into the bertini package so users can access them with a single import line (`from bertini import *`). Adds sqrt as a free-function wrapper around the Sqrt class, which was previously only callable as a constructor. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…namespace AbstractNode, AbstractSymbol, AbstractNamedSymbol, AbstractNumber, Handle, and AbstractStartSystem are Boost.Python wrappers for C++ abstract base classes. Python users cannot construct or subclass them; they existed in the public namespace only because the subpackage __init__.py files used `__all__ = dir(<cpp_module>)`, which blindly re-exported everything. Each affected __init__.py now dels the abstract names after the star import and builds __all__ with an explicit exclusion set. The C++ submodule references for symbol, root, and start_system are also overridden with their Python wrapper modules via importlib.import_module (a plain `from . import` is short-circuited when the name is already defined by the star import). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three CI failures introduced by 2efdb4c: 1. Windows lld-link: CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS does not export private static data members of classes, so ExplicitRKPredictor Butcher-tableau constants (defined in the data-only explicit_predictors.cpp) were invisible to test executables linking against bertini2.dll. Fix: revert Windows to a static library (remove BUILD_SHARED_LIBS TRUE from the early WIN32 block) and add /WHOLEARCHIVE to each test-executable linker step so lld-link pulls every object, including the data-only one. 2. Stale manylinux wheel: cibuildwheel cache key did not hash tracker_export.hpp or endgame_export.hpp, so the by-value scalar fix in 41bf852 may have been served from a pre-fix cached wheel. Added both python_bindings/include headers to the key. 3. ccache + clang-cl on Windows: add -DUSE_CCACHE=OFF to the Windows C++ test cmake invocation as a precaution against ccache/clang-cl edge cases. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…kage Replace `import bertini.foo` / `from bertini.foo import` with relative equivalents (`from . import foo` / `from .foo import`) across all python/bertini submodule __init__.py files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… choice AlgoBuilder::ClassicBuild now applies all user-specified config values to the constructed ZeroDim algorithm via a new virtual ApplyParsedConfigs(config_str). Previously, parsed configs were discarded and the algorithm ran with DefaultSetup() defaults regardless of what was in the input file. - Add virtual AnyZeroDim::ApplyParsedConfigs; implement in ZeroDim using a C++17 fold-expression helper InjectParsedTuple that applies a parsed std::tuple<Ts...> to any Configured<>-derived target via Set<T>. Injects ZeroDim-owned configs, tracker configs (SteppingConfig/NewtonConfig/Predictor via Setup()), endgame configs (via AlgoTraits<EndgameType>::NeededConfigs), and MidPathConfig. MPI-ready: each rank calls the same method on the broadcast config_str. - Add PrecisionType::FixedMultiple; fix Spirit parser (mptype:1 was incorrectly mapped to Adaptive); add three-way tracker dispatch in ClassicBuild. type::Tracker::FixedMultiple and ZeroDimSpecifyTracker dispatch were already correct. - Add EndgameChoice enum + EndgameChoiceConfig struct (endgamenum: 1=PSEG, 2=Cauchy); add ConfigSettingParser specialization; add to AllConfsD; wire dispatch in ClassicBuild. ZeroDimSpecifyEndgame dispatch was already correct. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds ENABLE_MPI=OFF cmake option. When ON, FindMPI locates OpenMPI/MPICH (no Boost.MPI needed) and defines BERTINI2_HAVE_MPI. New parallel infrastructure (all behind #ifdef BERTINI2_HAVE_MPI): - parallel/mpi_utils.hpp: mpi_send/recv_serialized + mpi_broadcast_string bridging Boost.Serialization and plain C MPI - parallel/path_result.hpp: PathBeforeEGResult<C>, Phase2Task<C>, PathDuringEGResult<C> with Boost.Serialization serialize() methods - parallel/manager.hpp: RunManagerLoop<TaskT,ResultT> — dynamic assignment - parallel/worker.hpp: RunWorkerLoop<TaskT,ResultT> ZeroDim::Run() dispatches to RunParallel(WorldComm()) when Size() > 1. RunParallel() runs two-phase manager-worker: before-EG then during-EG, with EGBoundaryAction (midpath check) on rank 0 between phases. Phase2Task carries boundary point+stepsize so workers are self-contained. parallel::Initialize() checks MPI_Initialized() before MPI_Init for mpi4py compatibility. Serial path is 100% unchanged; all tests pass. Blackbox: rank 0 reads files and broadcasts config_str/input_str; rank-0- only guards on file output writes. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Exposes the C++ MPI manager-worker infrastructure to Python so users
can pass an mpi4py communicator to solver.solve() for parallel execution,
or call serial solver.solve() as before (backward compatible).
New public API:
bertini.parallel.{rank, size, is_manager, is_worker, initialize, finalize}
solver.solve(communicator=MPI.COMM_WORLD) # parallel
solver.solve() # serial (unchanged)
Key fixes needed during implementation:
- +lambda trick: Boost.Python get_signature() can't introspect C++ lambdas;
convert stateless lambda to function pointer with unary + prefix
- Root CMakeLists now auto-detects MPI and sets ENABLE_MPI=ON before
add_subdirectory(core) so libbertini2 and _pybertini stay in sync
- Rank()/Size() guard with MPI_Initialized() check so importing bertini
in plain Python (no mpirun) is safe and returns serial defaults
Tests: 4 passing under mpirun -n 2; excluded from default pytest run via
norecursedirs in pyproject.toml (require mpi4py + mpirun).
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Minimal-blast-radius fixes: loop variable types int->size_t where bounds are size_t, explicit static_casts at signedness boundaries, constructor initializer-list reordering, commented-out unused parameter names, and range-loop-construct fixes. No member types, signatures, or return types changed (Degree() keeps its int/-1 sentinel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rent modules
The config sub-submodules were awkward (bertini.endgame.config.Endgame was
a config class confusingly named just 'Endgame'). Config classes now live
directly in bertini.tracking and bertini.endgame, with Config-suffixed names:
- endgame.config.Endgame -> endgame.EndgameConfig
- endgame.config.Security -> endgame.SecurityConfig
- endgame.config.{PowerSeries,Cauchy}Config -> endgame.* (unchanged names)
- tracking.config.* -> tracking.* (unchanged names)
BREAKING CHANGE: bertini.tracking.config and bertini.endgame.config no
longer exist; import the config classes from the parent modules.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… to mpfr_float
The rational type was meant to give exactness guarantees, but the guarantee
was already void: the classic settings parsers laundered every value through
double (NumTraits<double>::FromString) before storing into the rational, and
every comparable knob is already NumErrorT = double. mpfr_float is chosen
over double for its exponent range -- min_step_size can legitimately go
below double's ~1e-308 floor in high-precision AMP runs.
- SteppingConfig::T and EndgameConfig::sample_factor are now mpfr_float;
min_step_size default expressed as T("1e-100")
- amp_tracker: FromRational(x, prec) -> mpfr_float(x, prec) at use sites
- classic parsers now parse these settings via NumTraits<mpfr_float>::
FromString, so input files are read at full precision (no more double
laundering)
- python config enhance layer: update()/from_dict() accept strings for
numeric fields (update(max_step_size="0.05")), coerced exactly to
multiprec.Float; plain Python floats stay rejected by policy
- settings_test: mpfr_float expecteds; fixes latent mpq_rational(1/2)
integer-division bug (compared against 0)
Also fixes Float/Complex equality in Python, which was entirely broken:
Float == Float hit RecursionError because ExposeFloat registered GreatLess
visitors but no Equality visitors, so == fell through to a recursing eigenpy
ufunc slot. Added EqualitySelfVisitor/EqualityVisitor to Float and Complex.
ctest 10/10 suites pass; pytest 145 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… dispatch Each MPI worker rank now tracks paths on OMP_NUM_THREADS std::threads, for both the pre-endgame and endgame phases of ZeroDim. Verified: serial, MPI-serial-workers, and MPI-threaded produce identical solutions/residuals (differences only in RNG-probed condition numbers); 2.2x wall-clock on 4 threads for an 81-path quartic (use `mpirun --bind-to none` — OpenMPI's default per-rank core binding pins all threads to one core). Parallel layer: - parallel/thread_pool.hpp: ThreadSafeQueue + WorkerThreadPool with a per-thread state factory (System clone + Tracker copy + observers). - Credit-based manager<->worker protocol (new TAG_CAPACITY): each worker announces its thread count; the manager keeps that many tasks in flight per rank, retiring ranks with a sentinel at queue-empty + zero outstanding. - RunWorkerLoopThreaded: MPI_Iprobe event loop (a blocking recv would deadlock against the credit protocol); all MPI on the main thread under MPI_Init_thread(MPI_THREAD_FUNNELED). - ZeroDim: Phase1/Phase2ThreadState factories; thread variants of the per-path track functions with per-thread precision observers; phase-2 workers now set the newton_during_endgame tolerance (serial workers previously inherited the pre-EG tolerance). Thread-safety groundwork (value semantics, so copies are simply safe): - Tracker's predictor/corrector: shared_ptr -> by-value mutable members; copies were silently sharing work buffers with their source. - Observable copies no longer inherit the watcher list. - bertini::Clone() rebuilds derivatives + SLP from the deserialized tree and normalizes precision, instead of trusting the archived SLP. Thread-local precision (SetThreadPrecision/ThreadPrecision): - All tracking/endgame-time precision writes AND reads converted from the global DefaultPrecision to thread-local, including the sneaky readers: Eigen::NumTraits<mpfr> epsilon/dummy_precision/digits10, bertini NumTraits NumDigits/NumFuzzyDigits, SLPCompiler::Compile, TotalDegree start-point generation, Number node eval, RandomMp. - RNG engines made thread_local (RandomUnit is drawn every Newton step). Latent core bugs found and fixed along the way: - System::operator*=/+= mutated the SHARED Function nodes of the system they were called on, so forming a homotopy corrupted the target and start systems in place. Operators now create new Function wrappers. - ApplyParsedConfigs re-ran DefaultSystemSetup unconditionally, double- wrapping the homotopy (6 paths instead of 2 for a circle/line system, in every mode including serial). Now re-runs only when the parsed path variable name differs. - System::Homogenize() is now idempotent. - System::precision() now syncs the SLP regardless of differentiation state; the always-false PleaseAssumeUniformPrecision family is removed. - Unsuccessful endgames no longer crash metadata collection or the classic-output writers (guarded by SuccessCode). Verification: ctest 10/10; pytest 140 passed; MPI pytest 4x3 ranks passed; cross-mode equivalence on 2-path and 81-path systems. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…GABRT manylinux_2_28 (AlmaLinux 8) ships MPFR 3.1.6, which causes an unconditional abort() during AMP path tracking when bundled into the Linux wheel. Boost.Multiprecision 1.90.0 + Bertini's precision-changing arithmetic is incompatible with MPFR 3.1.6 at runtime. macOS (MPFR 4.2) and Windows (MPFR 4.x) are unaffected; the C++ tests on Ubuntu pass because the GitHub runner's apt provides MPFR 4.1. Switching to manylinux_2_34 (AlmaLinux 9, MPFR 4.1.0) resolves this without building from source. Wheels now require glibc ≥ 2.34 (Ubuntu 22.04 LTS / Debian 12+). Also commit the type-indexed observer dispatch optimization (observable.hpp, observer.hpp, observers.hpp): observers that override SubscribedEventTypes() are routed only to matching events; catch-all observers (including all Python observers via the empty-vector default) are unaffected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…pt-out
Previously ENABLE_MPI defaulted OFF, so users with MPI installed got a
serial exe unless they explicitly passed -DENABLE_MPI=ON. Now find_package(MPI)
runs unconditionally and option() defaults to ${MPI_FOUND}, giving auto-detect
with opt-out semantics. Also removes the root-level FORCE block that was
preventing -DENABLE_MPI=OFF from working.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…on in splash - OUTPUT_NAME bertini2 on the bertini2_exe CMake target so the installed binary is named `bertini2`, not `bertini2_exe` - --help now shows MPI + OMP_NUM_THREADS guidance (guarded by BERTINI2_HAVE_MPI so serial builds get an appropriate message) - --version now prints full DependencyVersions including the MPI library string instead of a bare "Bertini2" label - splash.hpp: add MPIVersion() using MPI_Get_library_version; include it in DependencyVersions(); guard <mpi.h> include with BERTINI2_HAVE_MPI - python_bindings/info.cpp: expose mpi_version() in the info submodule Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…CLI/parallelism section to landing page Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New doc_resources/landing/cli/index.html covers basic usage, MPI ranks, std::thread (OMP_NUM_THREADS), and serial-build fallback - Landing page now has three cards: Python, C++, CLI - style.css gains prose/pre/code/breadcrumb styles for subpages Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Register AnyEvent and 11 tracker event types per tracker variant (TrackingStarted, TrackingEnded, SuccessfulStep, FailedStep, StepsizeDecreased/Increased, InfinitePathTruncation, PrecisionChanged, PrecisionIncreased, PrecisionDecreased) in observers.amp/double/multiple submodules; enables isinstance() type discrimination in Python Observe() overrides - Expose tracker() accessor on TrackingEvent-derived types so Python observers can read live tracker state during an event - Expose previous()/next() on PrecisionChanged events - Fix ObserverWrapper::Observe to pass the event via boost::ref so Boost.Python's to_python_indirect path is taken, enabling RTTI-based polymorphic dispatch to the correct Python event class - Add CallbackObserver to observers.amp/double/multiple: pure-Python wrapper with .on(event_type, callback) for function-handle-based observation without subclassing - Fix TrackingStarted never being emitted: fire it from TrackPath in base_tracker.hpp after TrackerLoopInitialization succeeds (before first step), so FirstPrecisionRecorder and Python observers see it - 8 new observer tests; 153 total pass, no regressions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nput files All on_error handlers now print to stderr, include line number, column, expected token, and a truncated snippet of what was found, then throw std::runtime_error so callers get actionable exceptions rather than silent stdout output. Adds FormatParseError/ReportParseError helpers in qi_files.hpp used via phx::bind across all 20 grammar on_error sites. Fallback messages in system_parsers.hpp and settings_parsers.hpp also include the unconsumed remainder. Config parsing in algorithm_builder.cpp is now caught and returned as a non-zero exit code. Three negative test cases added to test_blackbox to verify throws on malformed input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ndomseed: parser Consolidates 5+ isolated static thread_local mt19937 engines into one canonical per-thread engine (ThreadEngine() in random.cpp) controlled by three new functions: SetGlobalSeed(seed) — seed 0 draws from entropy and reports effective seed GetGlobalSeed() — returns effective (always non-zero) global seed ReseedThisThread(key) — splitmix64(global_seed ^ key) into ThreadEngine() All tracking-phase random draws (RandomInt, RandomRat, RandomUnit<dbl>, rand_complex, rand_unit) route through ThreadEngine(). Fixes the non-thread_local bug in double_extensions.hpp::RandReal(). Routes PSEG SetRandVec and Cauchy rand_vector away from Eigen::Random to an explicit loop via RandomUnit<ComplexT>() so they are governed by the canonical engine. Per-path reseeding: ReseedThisThread(soln_ind) is called at the top of every TrackSinglePath* function (serial and threaded) so condition-number probes and PSEG random vectors are path-indexed and scheduling-independent — raw_data is now diff-able across serial/threaded/MPI modes with the same seed. RandomConfig struct + ConfigSettingParser<RandomConfig> specialisation (parses `randomseed: N;`). Added to Configs::Algorithm<T> so it participates in the existing blackbox config pipeline. The parsing test already contained `randomseed: 72;` — now asserts it parses to 72. MPI: manager calls SetGlobalSeed from parsed config, then broadcasts the effective seed to all workers via MPI_Bcast so cross-rank per-path streams are identical. Effective seed printed at startup: `bertini: random seed = N`. Python bindings: set_random_seed(seed=0) / get_random_seed() added to bertini.random. Note: RandomMp<N>() (compile-time-precision, used for patch/slice setup) keeps its own independent_bits_engine for now — setup draws are best-effort deterministic if SetGlobalSeed is called before system construction. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Python script wraps the bertini2 CLI to sweep MPI rank × OMP thread combinations, timing each run and computing speedup vs. serial baseline. Includes five diagonal sample systems (27–19683 paths) for small-to-cluster scale testing, and a SLURM job template for cluster submission. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…symbols lld-link won't pull explicit_predictors.obj (which holds all ExplicitRKPredictor Butcher-tableau statics) from libbertini2 unless forced. The same fix already existed for the C++ test targets in core/CMakeLists.txt; extend it to the Python binding module. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nly feature branches - Linux C++ tests: install openmpi-bin + libopenmpi-dev so MPI is compiled in and tested - macOS C++ tests: brew install open-mpi for same reason - Windows: add msmpi + mpi4py to conda env so FindMPI picks up MS-MPI - Wheel builds intentionally stay MPI-free (can't bundle mpirun; PyPI wheels must be self-contained) - build_macos_ubuntu_wheels now needs test_cpp_unix; build_windows_wheels needs test_cpp_windows so a C++ failure aborts the expensive wheel matrix immediately (!failure() guard preserves minimal-mode behavior where C++ tests are skipped) - Feature branches: reduce OS matrix from [ubuntu, macos] to [ubuntu] only; macOS still runs for PRs targeting develop/main and on those branches directly Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… duplicate Boost.Serialization symbols /WHOLEARCHIVE forces every object file from libbertini2 into the executable. Boost.Serialization instantiates oserializer/singleton templates in each TU that includes a serialization header, so the same symbol appears in multiple .obj files. Normal linking silently picks one; /WHOLEARCHIVE exposes them all and lld-link rejects them as duplicates. /FORCE:MULTIPLE restores the expected "first-definition wins" behavior for these identical template instantiations. Applied to both the C++ test targets (core/CMakeLists.txt) and the Python extension module (python_bindings/CMakeLists.txt). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous approach added /WHOLEARCHIVE:bertini2.lib as a link option while bertini2 was still in the normal LINK_LIBRARIES list, causing the archive to appear twice. lld-link then saw hundreds of duplicate Boost.Serialization symbols and either rejected them (as errors) or, with /FORCE:MULTIPLE, resolved them incorrectly — dropping other symbols and producing undefined-reference failures for System, ExplicitRKPredictor, parallel::Size, etc. Fix: strip "bertini2" from each target's LINK_LIBRARIES before adding /WHOLEARCHIVE:bertini2.lib, then re-add bertini2's INTERFACE_LINK_LIBRARIES (gmp, mpfr, boost, MPI, etc.) so transitive deps are preserved. The archive now appears exactly once. Applied to both the C++ test targets (core/CMakeLists.txt) and the Python extension module (python_bindings/CMakeLists.txt). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n from static init Root cause: EndgameConfig::sample_factor was mpfr_float, and DefaultConstruct<EndgameConfig>::value is a non-local static initialized at program startup when BMP global default precision is 20 (the BMP startup default), not the test's 16. With preserve_related_precision enabled (ET on), this stale-precision sample_factor contaminated time computations in ComputeInitialSamples and AdvanceTime, propagating precision 20 through TrackerLoopInitialization -> delta_t_ -> FullStep stage arithmetic -> SetVariables, causing a runtime_error in fixed-multiple precision endgame tests. Fix: change EndgameConfig::sample_factor from mpfr_float to double. Converting from double at use sites (RealT(sample_factor) / static_cast<RealT>(sample_factor)) creates an mpfr_float at the current DefaultPrecision(), always the right precision. 0.5 is exactly representable as double, so the default value is correct. Also adds explicit precision resets in MultiplePrecisionTracker::TrackerLoopInitialization before assigning start_time/end_time to member variables, so that any stale precision on incoming time values cannot propagate into the tracker's internal state via preserve_related_precision. All 353 endgame tests pass; all 10 ctest suites green. NOTE: double is not ideal for sample_factor -- 0.1 is not exactly representable. A follow-up commit will change sample_factor back to mpq_rational (exact arithmetic, no precision to go stale) which was its type before a434145. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…arser
The previous commit changed EndgameConfig::sample_factor from mpq_rational
to double as an expedient fix for the stale-precision bug rooted in
DefaultConstruct<T>::value being initialized at program startup. But double
is unsatisfactory: values like 0.1 or 0.3 are not exactly representable,
so a user setting SampleFactor: 0.647 would silently lose precision.
mpq_rational is the correct type:
- It is exact (GMP integer arithmetic, no floating-point at all).
- It carries no MPFR precision state, so it cannot go stale via
preserve_related_precision when DefaultConstruct<EndgameConfig>::value
is initialized at startup.
- It converts to mpfr_float at the correct DefaultPrecision() at use sites.
The prior mpq_rational(1)/2 NSDMI was broken (integer division → 0);
fixed by using the two-argument constructor mpq_rational{1, 2}.
The config-file parser now converts decimal strings (e.g. "0.647", "8e-3")
to mpq_rational exactly via decimal_str_to_rational(), which counts decimal
places and exponent to construct numerator/denominator as GMP integers,
avoiding both double's limited precision and GMP's octal-prefix ambiguity
(leading zeros are stripped before mpz_int construction).
All 10/10 ctest suites pass (353 endgame tests, settings parser tests,
blackbox parsing tests).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ng test targets The MSVC /WHOLEARCHIVE block called target_link_libraries with PRIVATE, but the test targets were already using the plain (keyword-less) signature. CMake rejects mixing the two styles on the same target. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rvers AMPObserver::Observe() calls t.RemoveObserver(*this) when it detects a precision increase, erasing itself from the exact vector being iterated by NotifyObservers. This is iterator invalidation — undefined behavior that manifested as heap corruption and SIGABRT on macOS (libc++ detects the invalid iterator), while silently "working" on older Linux (libstdc++ happened to not crash, though it skipped the next observer in the list). The same latent bug existed in the old single-list code, but the new type-indexed dispatch (added in 315ce14) uses separate per-type buckets in an unordered_map, making the UB more consistently fatal. Fix: snapshot the ObserverList before the notification loop so that removals during Observe() do not affect the live container. The copy is O(n_observers) per event but n_observers is always tiny (0–2 in normal usage), and self-removal happens only on precision-increase events which are rare during a track. Also explains the ubuntu SIGABRT at manylinux_2_28: that was a separate MPFR 3.1.6 incompatibility (fixed by 315ce14 manylinux upgrade), but the cache-hit restore at 315ce14 served a stale wheel, masking the fix. Our new commits force a cache miss so the fix takes effect in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y once The previous attempt broke because the library appeared twice (normal link + /WHOLEARCHIVE), and /FORCE:MULTIPLE then dropped needed symbols. Now that bertini2 appears only via /WHOLEARCHIVE (normal link entry stripped), /FORCE:MULTIPLE safely handles the Boost.Serialization void_cast_register and oserializer templates that are legitimately instantiated in multiple TUs within the archive — the same deduplication that ELF/Mach-O linkers perform via weak/COMDAT symbols on Linux and macOS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…2 archive target_link_options(/WHOLEARCHIVE:path) puts the flag in the linker OPTIONS section, not the library inputs — so lld-link saw the flag but never received bertini2.lib as an actual input, leaving all its symbols undefined. CMake 3.24+ $<LINK_LIBRARY:WHOLE_ARCHIVE,bertini2> correctly places the archive adjacent to other library inputs with /WHOLEARCHIVE: applied, ensuring lld-link both links the archive and forces all object files (including data-only objects like explicit_predictors.obj) to be included. Applies to both C++ test executables (core/CMakeLists.txt) and _pybertini.pyd (python_bindings/CMakeLists.txt). /FORCE:MULTIPLE is retained for Boost.Serialization duplicate template instantiations exposed by WHOLEARCHIVE. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s + verbose) print(system) / str(system) was poor for block-composed systems: GetNaturalFunctions() returns only the PolynomialBlock rows, so structured-block rows were invisible (a randomized system printed "2 functions:" then nothing; a poly+slice system dropped the slice); polynomial rows were double-named with "unnamed_function"; and debugging leftovers (the differentiated flag, a dump of the working dbl/mpfr variable vectors) leaked into the output. Give each block a Describe(out, row, vars, verbose) method; System::Describe loops the blocks and operator<< / Python str call the terse form. Terse uses placeholder symbols so only structure shows; verbose reveals the actual numbers and the underlying functions of randomization/blend. - PolynomialBlock: f_k = <expression> (via EntryNode; clean, single label). - LinearFormsBlock: terse f_k = c.[x, y, 1]; verbose the affine combination 2*x + 1*y - 1. - ProductsOfLinearsBlock: terse "prod of k linear forms"; verbose the product of factors. - RandomizationBlock: terse "f_a..f_b = R . g (R: nxN)" with the underlying g_j indented; verbose also prints R's entries. - BlendBlock: terse "f = (1-t)*A + gamma*t*B (blend of k systems)"; verbose lists operands. - System::Describe drops the differentiated flag and the current-variable-values dump, suppresses zero-count sections, shows the path variable only when defined, and prints the patch tersely (verbose shows the matrix). Python: system.describe(verbose=False); str/repr -> terse. Printing is human-facing only (not re-parseable; a classic-format / HC.jl exporter is a separate future feature). Tests: system_printing_test.cpp + python/test/classes/system_printing_test.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…grabbag ZeroDim: crossed-path resolution, speculative-full-path parallelism, reproducibility & MPI-portability fixes
…merge PR #21 (merged to develop) introduced ADRs 0022–0024 (path crossings, speculative-full-path parallelism, condition probe). This PR's ADRs collided on 0022/0023; renumber them to 0025 (randomization block) and 0026 (moving-row homotopy). No code references the numbers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ylinux container test_crossing_is_detected_then_resolved runs several cyclic-5 Euler solves (loose tolerance, order-1 predictor -- deliberately slow). It passed on macOS CI but exceeded its 360s per-test safety-net timeout inside the manylinux test container, which is markedly slower per-core; that hung the Linux wheel-test job and cascaded a cancel onto the Windows test job. The work is deterministic (fixed seeds) and completes -- it is just slower than 360s there -- so raise the safety net to 900s. Not a performance assertion; a true hang is still caught. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…arch The cyclic-5 seed-search crossing test timed out in the manylinux container (>15 min, up to 12 Euler solves of a 120-path system) because *which* seed crosses depends on platform floating point, so it had to search at runtime. Replace it with a tiny hand-built cubic whose crossing is a portable geometric fact: three paths A(t) (curved), B (flat), C (far); under coarse Euler the straight-line prediction of the curved path overshoots into the flat path's basin, the corrector lands on it exactly (accepted, so adaptive stepping does not rescue it), two paths funnel onto B and A(0) is lost -- a detectable, recoverable crossing. Exact rational coefficients + a macroscopic overshoot make it identical on every platform. Runs in ~0.02s instead of >15 min; no randomness, no seed search, no pytest timeout band-aid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ectly on Python 3.10 The enhanced config classes pickled via a to_dict/from_dict __reduce__, and let copy.deepcopy fall through to that generic path. copy.deepcopy then deep-copied the to_dict() mapping's *values* -- the multiprecision scalars -- which alias and corrupt on Python 3.10 (the SteppingConfig mpq_rational fields collapsed to 0.5, failing test_config_struct_round_trips[SteppingConfig] on the 3.10 wheels while 3.11-3.14 passed). Pickle was unaffected (it serializes each value on its own). Give the enhanced classes an explicit __deepcopy__ (and __copy__) that rebuild through from_dict(to_dict(self)) -- the same equality-preserving round-trip pickle already uses -- so the immutable scalar field values are re-fetched from the getters rather than deep-copied. Removes the corrupting operation entirely and is portable across interpreter versions; pickle behaviour is unchanged. Pre-existing bug from the config-pickling work (fbe6271); surfaced on PR #19's CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Randomize overdetermined systems via a first-class RandomizationBlock
i find versioning hard. when should i be doing this? can i automate this? it runs an entirely new CI, so that's pretty wasteful. boo.
the parenthetical was not so good
again, i didn't like the ()
it was getting in the way. Gone! 🔥
more documentation improvements
… condition number, norms) Adds Python bindings for CurrentStepsize, DeltaT, LatestConditionNumber, LatestNormOfStep, and LatestErrorEstimate on the tracker, alongside the existing current_point/current_time/current_precision. These let a Python observer read the full per-step state during a SuccessfulStep event, which is what the path-visualization (meta-)observer needs to collect a time series for plotting. Covered by a new test in observer_test.py that reads every accessor inside a SuccessfulStep callback and confirms each casts to a plain python number. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…point 4) Events are stack temporaries that live only for the duration of the NotifyObservers/Observe call, and their payload is held by const& into the emitter's internal buffers. Spell this out on AnyEvent and on the Python ObserverWrapper so future observer authors (especially Python ones that keep data) copy values out during Observe() rather than storing dangling refs. Documentation only; no functional change. Deliberately does NOT switch events to by-value: the held const& payload would still dangle, and by-value would break the boost::ref RTTI path the Python isinstance dispatch relies on. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tiniteam#255 pts 1 & 4) Observe() now returns ObserveResult { KeepObserving, Unsubscribe } instead of void. An observer asks to be dropped by returning Unsubscribe; the observable applies the removal after the notification loop finishes, so observers no longer mutate the watcher list mid-dispatch. Observable now defers ALL add/remove requests made during a notification (a depth-counted dispatching guard + an ordered pending-ops queue, drained when the outermost notification unwinds). This makes it safe for an observer to attach or detach other observers from inside Observe() -- the basis for a "meta-observer" that spawns a fresh per-path collector on TrackingStarted and harvests/detaches it on TrackingEnded. The old per-list snapshot copies in NotifyObservers are gone (the band-aid the deferral replaces). - FirstPrecisionRecorder now `return ObserveResult::Unsubscribe` instead of calling t.RemoveObserver(*this) mid-dispatch; all other observers and MultiObserver (AND-reduce: unsubscribe only when every child is done) updated. - Python: ObserverWrapper returns ObserveResult, translating a python None return to KeepObserving so existing observers/CallbackObserver are unchanged; a python observer may `return bertini.tracking.ObserveResult.Unsubscribe` to self-detach. The enum is exported and surfaced as bertini.tracking.ObserveResult. Documents the AnyEvent lifetime contract (bertiniteam#255 pt 4): events are stack temporaries holding payload by const& into emitter buffers, so observers must copy out anything they keep. Tests: new C++ cases (self-unsubscribe-via-return, meta-observer attaches a child mid-dispatch) in path_observers.cpp; new python cases (step-diagnostics accessors, python self-unsubscribe). test_tracking_basics (99) and test_nag_algorithms green; observer_test.py (10) green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…aths Adds two observers (per precision: amp/double/multiple), built the same factory way as CallbackObserver: - PathDataCollector (B): attach to a tracker; on each SuccessfulStep it records the time, the space point, and step diagnostics (condition number, precision, stepsize). Adaptive-precision trackers hand back mpfr numbers that don't pack into one numpy array, so each value is cast to plain python complex/float as it's collected. Exposes typed arrays times()/points()/diagnostics() and an optional as_dataframe() (pandas). - PathCollectionObserver (A): the meta-observer. Attach to a tracker (e.g. zd.get_tracker()) before a solve; on every TrackingStarted it spins up a fresh B, attaches it, and on TrackingEnded harvests it into .series and detaches it. The attach/detach happen from inside Observe() and rely on the observable's deferred-mutation machinery (the bertiniteam#255 refactor). Per-path state isolation comes for free: each path gets its own fresh collector. Because a solver reuses one tracker for the homotopy paths AND the endgame sub-tracks, .series holds more than one entry per solution; each collector is tagged with its track's start_time so the main homotopy paths (start at |t|=1) are trivially separable from endgame sub-tracks. Tests (meta_observer_test.py): bare-tracker one-series-per-track, array shapes/dtypes, as_dataframe (skipped without pandas), no-leak after harvest, and a full ZeroDim solve collecting every path with start_time filtering. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New tutorial python/docs/source/tutorials/observers_and_path_data.rst (added to the toctree). It builds from a one-line Python observer up to the meta-observer: writing an Observe() method, reading per-step tracker state, the ready-made CallbackObserver, collecting a path into numpy with PathDataCollector, and then PathCollectionObserver attaching/detaching per-path collectors across a whole ZeroDim solve. Ends by dehomogenizing and plotting the six homotopy paths of z^6 - 2z^2 + 2 = 0 in the complex plane (committed figure observers_and_path_data.png). Honest about two real wrinkles a reader will hit: mpfr values are cast to plain floats for plotting, and the solver reuses its tracker for endgame sub-tracks (so paths are separated by start_time). Builds clean under sphinx; the code mirrors what the meta_observer tests and figure generator actually run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cast (bertiniteam#255 pt2) Adds TypedObserver<Derived, ObservedT, ConcreteEvents...>: a CRTP base that implements SubscribedEventTypes() from its event list and the single Observe(AnyEvent const&) virtual once, doing the one downcast centrally and dispatching to a typed OnEvent(ConcreteEvent const&) handler on the derived observer. The hand-written SubscribedEventTypes() + dynamic_cast chains are removed from FirstPrecisionRecorder, MinMaxPrecisionRecorder, AMPPathAccumulator and StepFailScreenPrinter, which now just write their typed OnEvent overloads. The handler is named OnEvent (not an Observe overload) so it doesn't hide the Observe(AnyEvent const&) virtual -- Observe stays the one dispatch entry point and there's no -Woverloaded-virtual noise. Dispatch semantics are unchanged (still a dynamic_cast, just centralized). PrecisionAccumulator and both GoryDetailLoggers stay untyped on purpose: the former matches the TrackingEvent *base* (a hierarchy the typed path can't express) and the latter wants every type plus an unknown-event fallback. MultiObserver now routes children through the AnyObserver interface so it can glue typed observers (whose OnEvent overloads would otherwise be irrelevant and whose virtual is the right entry) as well as untyped ones. Also relaxes the step-diagnostics test: condition number/stepsize/precision are asserted positive, but error-estimate may legitimately be NaN/unset on a trivial step, so it's only checked to be a readable float. test_tracking_basics (99), test_nag_algorithms, and the python tracking suite (74) all green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rd (bertiniteam#255 pt3) Two lifetime/safety improvements at the observable boundary, the part of bertiniteam#255 that most affects the Python API. 1. Co-owned (shared_ptr) observers. AddObserver gains a shared_ptr overload; the observable stores the shared_ptr and co-owns the observer, so it stays alive and keeps firing for as long as it is attached -- you can attach an observer and drop your own reference without dangling ("attach it and forget it"). It is released on RemoveObserver or when the observable dies. The existing reference overload stays non-owning for stack-allocated C++ observers (which can't be shared_ptr-held); those remain caller-managed as before. (This realizes bertiniteam#255 point 3's lifetime-safety goal via ownership rather than the literal weak_ptr, which doesn't fit stack observers and would silently stop a forgotten observer instead of keeping it working.) Python: the observer wrapper classes now use a shared_ptr holder and add_observer extracts a shared_ptr, so python-defined observers are co-owned too. Proven by a test that attaches an observer keeping NO python reference and confirms it still fires (previously a dangling crash). 2. Incompatible-attach guard. AnyObserver::ObservedKind() (typeid of the observed type; Observer<T> reports typeid(T), default wildcard typeid(void)) and Observable::ObservableIsA() let AddObserver reject an observer built for a different observable -- which used to silently never fire. Throws IncompatibleObserver, translated to a python TypeError. Catches e.g. attaching a double-precision observer to an AMP tracker. C++ tests: incompatible attach throws; an owned observer outlives the caller's reference and is released on remove. test_tracking_basics (31), test_endgames, test_nag_algorithms green; full python suite (420) green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"Abstract" was misleading: the base is instantiable (you subclass it and it constructs fine). CustomObserver says what it's for -- subclass it to make your own observer. Renamed the exported class for trackers and endgames (amp/double/ multiple + the endgame variants), and updated the CallbackObserver/PathDataCollector/ PathCollectionObserver factories, tests, and the tutorial. No compatibility alias (no external users to break yet). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…kObserver.on
Tracker events are templated on the emitter, so each precision exposes its own
SuccessfulStep/TrackingStarted/... class under observers.amp/double/multiple --
forcing users to know the precision just to name an event. Two ergonomic fixes,
both pure-Python (the templated-on-emitter C++ design that gives observers full
typed access to event.tracker() is untouched):
- bertini.tracking.events.<Name> bundles the three precision variants into a
tuple, which works directly with isinstance() and CallbackObserver.on(). So
`events.SuccessfulStep` works regardless of precision.
- CallbackObserver.on() now also accepts the event name as a string, resolved
against the observer's own precision: obs.on("SuccessfulStep", cb).
Tests cover the tuple-union handle and the string form.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds nag_algorithms/events.hpp with four events emitted during a solve: AlgorithmStarted / AlgorithmComplete (around the whole Solve loop) and PathBeginning / PathComplete (around ExecuteOnePath, the per-start-point unit; PathComplete fires even when the pre-endgame leg fails). PathBeginning/Complete carry the path index. They are emitted on the non-templated AnyZeroDim base, so a SINGLE observer type can watch every templated ZeroDim variant (no per-instantiation explosion). An observer recovers the concrete solver from event.Get() -- in C++ via dynamic_cast, in Python (next commit) automatically via RTTI. ZeroDim overrides ObservableIsA to accept AnyZeroDim observers (in addition to its exact type). The point: between a PathBeginning and its PathComplete, the solver runs everything for that path -- main homotopy track AND endgame sub-tracks -- so an observer can group them as one complete solution path (the basis for SolutionPathCollector). (Event names are working placeholders; more events can be added by following the same pattern.) C++ tests: a ZeroDim observer sees AlgorithmStarted once, AlgorithmComplete once, and matched PathBeginning/PathComplete per path; a tracker observer attached to a ZeroDim is rejected. test_nag_algorithms (22) green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…erver + lifecycle events)
The Python ZeroDim solvers now have add_observer/remove_observer (ObservableVisitor
on ZDVisitor), and a new bertini.nag_algorithms.observers submodule exposes the
single AnyZeroDim-based observer surface for every solver variant:
- CustomObserver (subclassable base, shared_ptr holder -> co-owned when attached)
- AlgorithmStarted / AlgorithmComplete / PathBeginning / PathComplete
(PathBeginning/Complete carry .path_index()).
AnyZeroDim is registered and the concrete ZeroDim classes now declare
bases<AnyZeroDim>, so event.solver() (statically AnyZeroDim&) resolves via RTTI to
the concrete solver with its full python API -- confirmed by a test reading
e.solver().solutions() inside Observe(). The attach guard distinguishes observables:
a nag observer is rejected on a tracker and a tracker observer on the solver
(TypeError).
Tests in meta_observer_test.py cover the lifecycle event counts/indices, the
RTTI concrete-solver recovery, and the cross-observable guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…game) in one observer bertini.nag_algorithm.SolutionPathCollector is the two-level meta-observer the dream was after. Attach it to a ZeroDim solver: on each PathBeginning it spins up a fresh tracking PathDataCollector and attaches it to the solver's tracker, and on the matching PathComplete it harvests that collector into .series and detaches it. Since the solver reuses one tracker for a path's main homotopy track AND its endgame sub-tracks, the per-path collector captures the entire journey to t -> 0 -- one clean series per solution path, endgame included, with no start-time filtering. It picks the right precision's PathDataCollector via tracker.observers, and recovers the tracker from event.solver().get_tracker() (RTTI-resolved concrete solver). The tutorial now leads with SolutionPathCollector (attach to the solver) as the clean way to collect and plot every path, and the regenerated figure shows each of the six z^6-2z^2+2 paths running all the way into its solution (the endgame carries the last leg). The tracker-level PathCollectionObserver is kept and mentioned as the lower-level building block. Tests: one series per solution path with the expected indices; whole-path series capture at least as many steps as the main-track-only filter. Full observer/nag/zero_dim suites green; docs build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ax.set_aspect(1.0) so Re/Im are scaled equally; regenerated the figure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…on number New section solving the cyclic-3 system in three variables and plotting the paths in (Re x, Re y, Re z) space, each drawn as a Line3DCollection coloured along its length by the condition number (log scale, shared colorbar) -- showing where the tracking is worst-conditioned (near the solutions / in the endgame). Uses the same SolutionPathCollector and the diagnostics() condition-number column, with a 1:1:1 box aspect. Committed figure cyclic3_paths.png; docs build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…orn) section, SVG+PNG figures - Seed every example with bertini.random.set_random_seed(...) so a reader reproduces exactly the pictures shown. - New section "Watching the Cauchy endgame at a singular solution": solves Griewank-Osborn (triple root at the origin), selects the singular paths from the solver's own metadata (m.is_singular / m.path_index -- no hand-rolled coordinate thresholds), and plots one path's x-coordinate on a LOG-RADIAL scale so the geometrically-shrinking Cauchy loops stay visible as a spiral winding into the singularity (coloured by |t|). - Render every tutorial figure as SVG (crisp in the built docs) and also keep a PNG alongside for local viewing; figure directives reference the SVG. - z^6 plot: add set_box_aspect(1) so the box is square (with 1:1 data scaling). Docs build clean; figures regenerated from seeded scripts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consistent lifecycle-event naming: AlgorithmStarted/AlgorithmComplete and PathStarted/PathComplete. Renamed the C++ event class, the emission point, the python-exposed class, SolutionPathCollector, and the tests/tutorial. C++ test_nag_algorithms and the python meta_observer suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Observer subsystem refactor (bertiniteam#255) + Python path-visualization observers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.