You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: Comprehensive update to acknowledgements and credits
- Expanded README.md with detailed 'Inspirations & Acknowledgements' section covering MuZero, Mamba, ToT, RepE, RAP, AlphaLLM, RLHF, and PPLM.
- Updated ARCHITECTURE.md with clarified Gated Injection note and expanded references.
- Updated ROADMAP.md with disclaimer on theoretical benefits.
- Updated technical_report.md with theoretical disclaimer and inspiration summary.
- Updated README.md tone to be semi-formal and humble regarding claims.
This acts as a "low-pass filter" for thoughts. It allows the planner to gently nudge the model's intuition towards a better trajectory without disrupting the stability of the underlying state manifold.
32
32
@@ -130,6 +130,46 @@ I believe this transparency is important to accurately represent the collaborati
130
130
131
131
---
132
132
133
+
## 🧠 Inspirations & Acknowledgements
134
+
135
+
This project is an experimental synthesis of existing breakthrough research. It attempts to combine these distinct ideas into a unified architecture. I claim no credit for the foundational concepts, only for the specific implementation of their integration (CRSM).
136
+
137
+
### Core Theoretical Foundations
138
+
***[MuZero (Schrittwieser et al., DeepMind)](https://arxiv.org/abs/1911.08265)**: The primary inspiration for performing **Monte Carlo Tree Search (MCTS)** entirely within a learned **latent space**, without decoding back to observations. CRSM adapts this "planning in latent space" concept to the continuous state of a language model.
139
+
***[Mamba (Gu & Dao)](https://arxiv.org/abs/2312.00752)**: The efficient **State Space Model (SSM)** backbone is the engine of this architecture. Its fixed-size, linear-time state enables the direct state manipulation and injection that would be computationally prohibitive with the KV-cache of Transformers.
140
+
***[Tree of Thoughts (Yao et al.)](https://arxiv.org/abs/2305.10601)** & **[Chain of Thought (Wei et al.)](https://arxiv.org/abs/2201.11903)**: The inspiration for treating reasoning as a search problem over a space of intermediate steps. CRSM attempts to make this search internal and continuous rather than external and discrete.
141
+
142
+
### Cognitive Frameworks
143
+
***[System 1 & System 2 (Daniel Kahneman)](https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow)**: The guiding conceptual framework.
144
+
***System 1 (Intuition):** Represented by the Mamba backbone (fast, heuristic generation).
145
+
***System 2 (Deliberation):** Represented by the Asynchronous MCTS planner (slow, logical search).
146
+
***[Global Workspace Theory (Baars)](https://en.wikipedia.org/wiki/Global_workspace_theory)**: The idea of a "working memory" where conscious processing occurs inspired the design of the **Latent State** as a shared workspace that both the planner and generator can access and modify.
147
+
148
+
### Emerging Research
149
+
***[Coconut (Chain of Continuous Thought)](https://arxiv.org/abs/2412.06769)**: A parallel line of research exploring reasoning in continuous latent space. While Coconut feeds the last hidden state back as input to the next step, CRSM modifies the internal state directly in real-time during the generation process.
150
+
151
+
### Architectural Components
152
+
***[JEPA (LeCun)](https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/)**: The design of the **Latent Dynamics Model** is heavily influenced by Joint Embedding Predictive Architectures—learning to predict the representation of the next state rather than the pixel/token details.
153
+
***[World Models / Dreamer (Ha & Schmidhuber, Hafner et al.)](https://arxiv.org/abs/1803.10122)**: The concept of learning a compact model of the environment to simulate futures ("dreaming") for planning is directly implemented in CRSM's dynamics distillation pipeline.
154
+
155
+
### Related Mechanics
156
+
***[State Delta Communication (Tang et al.)](https://aclanthology.org/2024.findings-emnlp.665/)**: While CRSM uses "state deltas" for *intra-agent* self-correction (Planner → Backbone), Tang et al. explore a similar mechanic for *inter-agent* communication, passing "state deltas" between models to convey reasoning dynamics that are lost in discrete token communication.
157
+
158
+
### Related Methodologies
159
+
***[Representation Engineering (RepE) (Zou et al.)](https://arxiv.org/abs/2310.01405)**: The concept of "Top-Down" control of model behavior by manipulating the latent space is central to CRSM. Our "Gated Injection" can be viewed as a control-theoretic application of RepE, where the control vector is dynamically generated by the planner rather than a static concept vector.
160
+
***[Reasoning via Planning (RAP) (Hao et al.)](https://arxiv.org/abs/2305.14992)** & **[AlphaLLM (Tencent AI Lab)](https://arxiv.org/abs/2404.05584)**: These works pioneered the integration of MCTS with Large Language Models to enable self-improvement and strategic planning. CRSM builds on this by moving the planning process into the *asynchronous* and *continuous* domain.
161
+
***[Plug and Play Language Models (PPLM) (Dathathri et al.)](https://arxiv.org/abs/1912.02164)** & **[Activation Addition (Turner et al.)](https://arxiv.org/abs/2308.10248)**: These works established the foundation for steering model generation by modifying hidden states (via gradients or vector addition). CRSM extends this by using a *dynamic planner* to generate the steering vectors in real-time, rather than using static vectors or classifiers.
162
+
***[RLHF (Christiano et al. / OpenAI)](https://arxiv.org/abs/1706.03741)**: The methodology of training a separate **Value Head** to estimate the utility of a language model's state is adapted directly from the foundational work on Reinforcement Learning from Human Feedback.
163
+
164
+
### Mathematical & Engineering Parallels
165
+
***[Speculative Decoding (Leviathan et al.)](https://arxiv.org/abs/2211.17192)**: The "draft-then-verify" computational pattern in speculative decoding shares DNA with CRSM's asynchronous design. In CRSM, the "dynamics model" acts as a latent drafter, while the MCTS planner acts as a verifier/improver running in parallel.
166
+
***[Polyak Averaging (Lillicrap et al.)](https://arxiv.org/abs/1509.02971)**: The Gated Injection formula ($h_{new} = (1-\tau)h + \tau h_{target}$) is mathematically identical to the "soft target updates" used in DDPG and other RL algorithms. We apply this standard control-theory technique to maintain stability in the language model's latent manifold.
167
+
***[Quiet-STaR (Zelikman et al.)](https://arxiv.org/abs/2403.09629)**: This work explores generating "internal thoughts" at every token step to improve reasoning. CRSM shares this goal but seeks to make these thoughts continuous and asynchronous rather than discrete and interleaved.
168
+
169
+
I am deeply grateful to the researchers behind these works for sharing their code and insights with the open-source community.
Copy file name to clipboardExpand all lines: docs/ARCHITECTURE.md
+5Lines changed: 5 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -49,6 +49,8 @@ Linear layer projecting from hidden dimension to vocabulary:
49
49
50
50
### 6. Gated State Injection (Stability Mechanism)
51
51
52
+
**Note:** The primary innovation of CRSM is the **asynchronous interaction loop** between the System 1 backbone and the System 2 planner. The Gated Injection formula below is simply the stabilizing mechanism that makes this loop mathematically viable on a continuous manifold.
53
+
52
54
To safely integrate the asynchronous thought vectors into the sensitive Mamba manifold, CRSM employs a **Gated Injection** mechanism rather than simple additive perturbation.
53
55
54
56
The update rule is:
@@ -202,4 +204,7 @@ Expected throughput (on A100):
202
204
203
205
-**S4**: [Efficiently Modeling Long Sequences with Structured State Spaces](https://arxiv.org/abs/2111.00396)
204
206
-**Mamba**: [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.08956)
207
+
-**MuZero**: [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://arxiv.org/abs/1911.08265)
208
+
-**RepE**: [Representation Engineering: A Top-Down Approach to AI Transparency](https://arxiv.org/abs/2310.01405)
209
+
-**AlphaLLM**: [AlphaLLM: Monte Carlo Tree Search with Large Language Models](https://arxiv.org/abs/2404.05584)
205
210
-**Scaling Laws**: [Training Compute-Optimal Large Language Models](https://arxiv.org/abs/2203.15556)
Copy file name to clipboardExpand all lines: docs/ROADMAP.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ This phase involved the initial design, implementation, and debugging of the cor
17
17
18
18
## Phase 2: Initial Training and Validation (🚀 IN PROGRESS)
19
19
20
-
This phase focuses on moving from a structurally complete prototype to a model that demonstrates meaningful capabilities.
20
+
This phase focuses on moving from a structurally complete prototype to a model that demonstrates meaningful capabilities.**Note: At this stage, the performance benefits of the MCTS planner are theoretical. Empirical validation requires the large-scale training scheduled for Phase 4.**
21
21
22
22
***[DONE]****Architecture Verification**: Diagnosed "State Explosion" issue and implemented **Gated State Injection** to solve it.
23
23
***[DONE]****Safety Mechanisms**: Implemented Confidence Scaling to prevent model lobotomization by untrained planners.
Copy file name to clipboardExpand all lines: docs/technical_report.md
+8-2Lines changed: 8 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,9 +38,15 @@ This acts effectively as a low-pass filter for reasoning signals. The planner ge
38
38
## 4. Current Status
39
39
***Stability:** Verified via stress tests (1000+ continuous injections). The Gated Injection mechanism successfully maintains numerical stability.
40
40
***Safety:** A "Confidence Scaling" mechanism ensures that if the planner's Value Head is uncertain, the injection rate ($\alpha$) drops to zero, preventing the planner from degrading the backbone's performance.
41
-
***Validation:** Preliminary tests verify that *if* the planner identifies a state with lower expected loss, the injection mechanism successfully steers the generation towards it.
41
+
***Validation:** Preliminary tests verify that *if* the planner identifies a state with lower expected loss, the injection mechanism successfully steers the generation towards it.**Note: The net positive impact on downstream reasoning tasks is currently theoretical and awaits large-scale training (Phase 4 of Roadmap).**
42
42
43
-
## 5. Current Constraints & Challenges
43
+
## 5. Inspirations & Parallels
44
+
This architecture draws heavily from existing research:
45
+
***Representation Engineering (RepE):** The idea of steering generation via latent space manipulation.
46
+
***MuZero & AlphaLLM:** The application of MCTS to latent states and language models.
47
+
***Polyak Averaging:** The mathematical basis for the stable "soft update" injection.
48
+
49
+
## 6. Current Constraints & Challenges
44
50
While the architecture functions, several practical bottlenecks remain:
45
51
46
52
1.**Global Interpreter Lock (GIL):** The implementation uses Python's `asyncio`, which is not truly parallel due to the GIL. This causes minor stuttering in generation when the planner is under heavy load. A robust implementation would require the planner to be implemented in C++ or Rust.
0 commit comments