Skip to content

Commit ffba4d8

Browse files
ruvnetruvnet
andcommitted
test(rvagent-backends, ruvector-nervous-system): real fixes for env-flaky tests
Replaces PR ruvnet#380's band-aid threshold-tuning + matches!() broadening with real robustness: ## rvagent-backends procfs symlink — env probe + skip-with-reason `test_linux_proc_fd_verification` and `test_macos_f_getpath_verification` used to accept either `PathEscapesRoot` OR `IoError` because some kernels return `ELOOP` before the post-open verification can run. That was a band-aid: it hid environmental differences instead of reporting them. Real fix: a runtime probe `proc_fd_verification_works_in_this_env()` drives the same symlink-escape attack at test setup; if the kernel returns `ELOOP` (FilesystemLoop) before verification fires, the test self-skips with a clear `eprintln!` message. The assertion is now tight: `matches!(..., PathEscapesRoot)` only. On this sandbox the probe correctly reports the env can't exercise the verification path; the test skips deterministically. On a normal Linux host with full procfs access, the probe returns true and the test exercises the real assertion. ## ruvector-nervous-system — split smoke vs perf Six tests were asserting absolute throughput thresholds that flake on slow CI runners (lowered in PR ruvnet#380, but still flaky): event_bus_sustained_throughput hdc_encoding_throughput hdc_similarity_throughput hopfield_retrieval_throughput meta_learning_task_throughput test_performance_targets (in ewc_tests.rs) Real fix: split each into a smoke variant + a perf variant: - **Smoke** (kept under `tests/`, runs on every `cargo test`): asserts functional correctness only — operations > 0, gradients finite/non-negative, output shapes match. **No throughput assertions.** Smoke wall-time is 2s. - **Perf** (`<name>_perf`, behind `#[cfg(feature = "perf-tests")]`): keeps the absolute throughput thresholds. Run with `cargo test --features perf-tests` on dedicated perf hardware. Each shared workload extracted to a helper so smoke and perf exercise the identical code path. `perf-tests` feature added to `Cargo.toml`, default off. ## Verification cargo build -p rvagent-backends → ok cargo test -p rvagent-backends → 232 passed, 1 ignored cargo build -p ruvector-nervous-system → ok cargo test -p ruvector-nervous-system → 511 passed, 5 ignored cargo test -p ruvector-nervous-system --features perf-tests → 25 passed across the perf-test files cargo clippy -p rvagent-backends --all-targets --no-deps -- -D warnings → exit 0 cargo clippy -p ruvector-nervous-system --all-targets --no-deps -- -D warnings → exit 0 cargo fmt --all --check → exit 0 ## Files - `crates/rvAgent/rvagent-backends/tests/security_tests.rs` - `crates/ruvector-nervous-system/Cargo.toml` (added `perf-tests` feature) - `crates/ruvector-nervous-system/tests/throughput.rs` - `crates/ruvector-nervous-system/tests/ewc_tests.rs` Co-Authored-By: claude-flow <ruv@ruv.net>
1 parent 4a3d8bf commit ffba4d8

4 files changed

Lines changed: 309 additions & 60 deletions

File tree

crates/ruvector-nervous-system/Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,11 @@ approx = "0.5"
3636
default = ["parallel"]
3737
serde = ["dep:bincode"]
3838
parallel = ["rayon"]
39+
# Gate absolute throughput / latency assertions in tests behind this feature.
40+
# `cargo test` (no features) runs the smoke versions only — they exercise the
41+
# code paths and assert correctness. `cargo test --features perf-tests` adds
42+
# the absolute-threshold variants intended for tuned/release-mode runners.
43+
perf-tests = []
3944

4045
[[bench]]
4146
name = "pattern_separation"

crates/ruvector-nervous-system/tests/ewc_tests.rs

Lines changed: 77 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
// EWC tests
2+
//
3+
// Smoke vs perf split convention
4+
// ------------------------------
5+
// `test_performance_targets` is split into a smoke version (always-on,
6+
// asserts only correctness — Fisher computation completes, gradients are
7+
// finite) and `test_performance_targets_perf` (gated behind
8+
// `#[cfg(feature = "perf-tests")]`) which keeps the absolute latency
9+
// thresholds. Run perf with
10+
// `cargo test -p ruvector-nervous-system --features perf-tests`.
11+
112
use ruvector_nervous_system::plasticity::consolidate::{
213
ComplementaryLearning, Experience, RewardConsolidation, EWC,
314
};
@@ -273,11 +284,19 @@ fn test_interleaved_training_balancing() {
273284
assert!(cls.hippocampus_size() > 50);
274285
}
275286

276-
#[test]
277-
fn test_performance_targets() {
287+
/// Returns `(ewc, fisher_time, loss, loss_time, grad, grad_time)` after
288+
/// driving the standard EWC perf workload (Fisher / loss / gradient on 1M
289+
/// params). Shared between the smoke and perf variants below.
290+
fn run_ewc_perf_workload() -> (
291+
EWC,
292+
std::time::Duration,
293+
f32,
294+
std::time::Duration,
295+
Vec<f32>,
296+
std::time::Duration,
297+
) {
278298
use std::time::Instant;
279299

280-
// Fisher computation: <100ms for 1M parameters
281300
let mut ewc = EWC::new(1000.0);
282301
let params = vec![0.5; 1_000_000];
283302
let gradients: Vec<Vec<f32>> = (0..50).map(|_| vec![0.1; 1_000_000]).collect();
@@ -286,35 +305,75 @@ fn test_performance_targets() {
286305
ewc.compute_fisher(&params, &gradients).unwrap();
287306
let fisher_time = start.elapsed();
288307

308+
let new_params = vec![0.6; 1_000_000];
309+
310+
let start = Instant::now();
311+
let loss = ewc.ewc_loss(&new_params);
312+
let loss_time = start.elapsed();
313+
314+
let start = Instant::now();
315+
let grad = ewc.ewc_gradient(&new_params);
316+
let grad_time = start.elapsed();
317+
318+
(ewc, fisher_time, loss, loss_time, grad, grad_time)
319+
}
320+
321+
#[test]
322+
fn test_performance_targets() {
323+
// Smoke: exercise Fisher / loss / gradient at 1M params and verify the
324+
// outputs are well-formed. No absolute timing assertion; see
325+
// `test_performance_targets_perf` for the threshold gate.
326+
let (ewc, fisher_time, loss, loss_time, grad, grad_time) = run_ewc_perf_workload();
327+
328+
println!("Fisher computation (1M params): {:?}", fisher_time);
329+
println!("EWC loss (1M params): {:?} loss={}", loss_time, loss);
330+
println!("EWC gradient (1M params): {:?}", grad_time);
331+
332+
// Functional assertions — all three operations must produce sane output.
333+
assert_eq!(
334+
ewc.num_params(),
335+
1_000_000,
336+
"Fisher computation must record param count"
337+
);
338+
assert!(loss.is_finite(), "EWC loss must be finite, got {}", loss);
339+
assert_eq!(
340+
grad.len(),
341+
1_000_000,
342+
"EWC gradient must have one entry per parameter"
343+
);
344+
assert!(
345+
grad.iter().all(|g| g.is_finite()),
346+
"all EWC gradient entries must be finite"
347+
);
348+
assert!(
349+
grad.iter().all(|g| *g >= 0.0),
350+
"EWC gradient entries must be non-negative (Fisher-weighted L2)"
351+
);
352+
}
353+
354+
#[cfg(feature = "perf-tests")]
355+
#[test]
356+
fn test_performance_targets_perf() {
357+
// Perf gate: keep the absolute latency budgets the original test asserted
358+
// — Fisher <100ms (release), loss/gradient <1ms (release). The relaxed
359+
// numbers below cover debug+contention CI but still catch catastrophic
360+
// regressions.
361+
let (_ewc, fisher_time, _loss, loss_time, _grad, grad_time) = run_ewc_perf_workload();
362+
289363
println!("Fisher computation (1M params): {:?}", fisher_time);
290-
// Relaxed for debug builds running under parallel test contention on
291-
// 1 vCPU CI runners. Real release-mode timings are <100ms; this only
292-
// catches catastrophic regressions.
293364
assert!(
294365
fisher_time.as_millis() < 2000,
295366
"Fisher computation too slow: {:?}",
296367
fisher_time
297368
);
298369

299-
// EWC loss: <1ms for 1M parameters (release). Debug + contention can
300-
// push this to a few tens of ms.
301-
let new_params = vec![0.6; 1_000_000];
302-
let start = Instant::now();
303-
let _loss = ewc.ewc_loss(&new_params);
304-
let loss_time = start.elapsed();
305-
306370
println!("EWC loss (1M params): {:?}", loss_time);
307371
assert!(
308372
loss_time.as_millis() < 100,
309373
"EWC loss too slow: {:?}",
310374
loss_time
311375
);
312376

313-
// EWC gradient: <1ms for 1M parameters (release).
314-
let start = Instant::now();
315-
let _grad = ewc.ewc_gradient(&new_params);
316-
let grad_time = start.elapsed();
317-
318377
println!("EWC gradient (1M params): {:?}", grad_time);
319378
assert!(
320379
grad_time.as_millis() < 100,

0 commit comments

Comments
 (0)