Skip to content

Commit 79c2466

Browse files
committed
docs: Comprehensive update to acknowledgements and credits
- Expanded README.md with detailed 'Inspirations & Acknowledgements' section covering MuZero, Mamba, ToT, RepE, RAP, AlphaLLM, RLHF, and PPLM. - Updated ARCHITECTURE.md with clarified Gated Injection note and expanded references. - Updated ROADMAP.md with disclaimer on theoretical benefits. - Updated technical_report.md with theoretical disclaimer and inspiration summary. - Updated README.md tone to be semi-formal and humble regarding claims.
1 parent 5bb811c commit 79c2466

4 files changed

Lines changed: 55 additions & 4 deletions

File tree

README.md

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ Standard Transformers typically face a latency trade-off: to perform "System 2"
2626

2727
### 1. Gated State Injection (Stability Control)
2828
Directly modifying a model's high-dimensional latent state often leads to "state explosion."
29-
My solution is **Gated Injection**:
29+
The proposed mechanism is **Gated Injection**:
3030
$$h_{t} \leftarrow (1 - \alpha_{eff}) \cdot h_{t} + \alpha_{eff} \cdot h_{target}$$
3131
This acts as a "low-pass filter" for thoughts. It allows the planner to gently nudge the model's intuition towards a better trajectory without disrupting the stability of the underlying state manifold.
3232

@@ -130,6 +130,46 @@ I believe this transparency is important to accurately represent the collaborati
130130

131131
---
132132

133+
## 🧠 Inspirations & Acknowledgements
134+
135+
This project is an experimental synthesis of existing breakthrough research. It attempts to combine these distinct ideas into a unified architecture. I claim no credit for the foundational concepts, only for the specific implementation of their integration (CRSM).
136+
137+
### Core Theoretical Foundations
138+
* **[MuZero (Schrittwieser et al., DeepMind)](https://arxiv.org/abs/1911.08265)**: The primary inspiration for performing **Monte Carlo Tree Search (MCTS)** entirely within a learned **latent space**, without decoding back to observations. CRSM adapts this "planning in latent space" concept to the continuous state of a language model.
139+
* **[Mamba (Gu & Dao)](https://arxiv.org/abs/2312.00752)**: The efficient **State Space Model (SSM)** backbone is the engine of this architecture. Its fixed-size, linear-time state enables the direct state manipulation and injection that would be computationally prohibitive with the KV-cache of Transformers.
140+
* **[Tree of Thoughts (Yao et al.)](https://arxiv.org/abs/2305.10601)** & **[Chain of Thought (Wei et al.)](https://arxiv.org/abs/2201.11903)**: The inspiration for treating reasoning as a search problem over a space of intermediate steps. CRSM attempts to make this search internal and continuous rather than external and discrete.
141+
142+
### Cognitive Frameworks
143+
* **[System 1 & System 2 (Daniel Kahneman)](https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow)**: The guiding conceptual framework.
144+
* **System 1 (Intuition):** Represented by the Mamba backbone (fast, heuristic generation).
145+
* **System 2 (Deliberation):** Represented by the Asynchronous MCTS planner (slow, logical search).
146+
* **[Global Workspace Theory (Baars)](https://en.wikipedia.org/wiki/Global_workspace_theory)**: The idea of a "working memory" where conscious processing occurs inspired the design of the **Latent State** as a shared workspace that both the planner and generator can access and modify.
147+
148+
### Emerging Research
149+
* **[Coconut (Chain of Continuous Thought)](https://arxiv.org/abs/2412.06769)**: A parallel line of research exploring reasoning in continuous latent space. While Coconut feeds the last hidden state back as input to the next step, CRSM modifies the internal state directly in real-time during the generation process.
150+
151+
### Architectural Components
152+
* **[JEPA (LeCun)](https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/)**: The design of the **Latent Dynamics Model** is heavily influenced by Joint Embedding Predictive Architectures—learning to predict the representation of the next state rather than the pixel/token details.
153+
* **[World Models / Dreamer (Ha & Schmidhuber, Hafner et al.)](https://arxiv.org/abs/1803.10122)**: The concept of learning a compact model of the environment to simulate futures ("dreaming") for planning is directly implemented in CRSM's dynamics distillation pipeline.
154+
155+
### Related Mechanics
156+
* **[State Delta Communication (Tang et al.)](https://aclanthology.org/2024.findings-emnlp.665/)**: While CRSM uses "state deltas" for *intra-agent* self-correction (Planner → Backbone), Tang et al. explore a similar mechanic for *inter-agent* communication, passing "state deltas" between models to convey reasoning dynamics that are lost in discrete token communication.
157+
158+
### Related Methodologies
159+
* **[Representation Engineering (RepE) (Zou et al.)](https://arxiv.org/abs/2310.01405)**: The concept of "Top-Down" control of model behavior by manipulating the latent space is central to CRSM. Our "Gated Injection" can be viewed as a control-theoretic application of RepE, where the control vector is dynamically generated by the planner rather than a static concept vector.
160+
* **[Reasoning via Planning (RAP) (Hao et al.)](https://arxiv.org/abs/2305.14992)** & **[AlphaLLM (Tencent AI Lab)](https://arxiv.org/abs/2404.05584)**: These works pioneered the integration of MCTS with Large Language Models to enable self-improvement and strategic planning. CRSM builds on this by moving the planning process into the *asynchronous* and *continuous* domain.
161+
* **[Plug and Play Language Models (PPLM) (Dathathri et al.)](https://arxiv.org/abs/1912.02164)** & **[Activation Addition (Turner et al.)](https://arxiv.org/abs/2308.10248)**: These works established the foundation for steering model generation by modifying hidden states (via gradients or vector addition). CRSM extends this by using a *dynamic planner* to generate the steering vectors in real-time, rather than using static vectors or classifiers.
162+
* **[RLHF (Christiano et al. / OpenAI)](https://arxiv.org/abs/1706.03741)**: The methodology of training a separate **Value Head** to estimate the utility of a language model's state is adapted directly from the foundational work on Reinforcement Learning from Human Feedback.
163+
164+
### Mathematical & Engineering Parallels
165+
* **[Speculative Decoding (Leviathan et al.)](https://arxiv.org/abs/2211.17192)**: The "draft-then-verify" computational pattern in speculative decoding shares DNA with CRSM's asynchronous design. In CRSM, the "dynamics model" acts as a latent drafter, while the MCTS planner acts as a verifier/improver running in parallel.
166+
* **[Polyak Averaging (Lillicrap et al.)](https://arxiv.org/abs/1509.02971)**: The Gated Injection formula ($h_{new} = (1-\tau)h + \tau h_{target}$) is mathematically identical to the "soft target updates" used in DDPG and other RL algorithms. We apply this standard control-theory technique to maintain stability in the language model's latent manifold.
167+
* **[Quiet-STaR (Zelikman et al.)](https://arxiv.org/abs/2403.09629)**: This work explores generating "internal thoughts" at every token step to improve reasoning. CRSM shares this goal but seeks to make these thoughts continuous and asynchronous rather than discrete and interleaved.
168+
169+
I am deeply grateful to the researchers behind these works for sharing their code and insights with the open-source community.
170+
171+
---
172+
133173
## Reference (If you find this useful)
134174

135175
```bibtex

docs/ARCHITECTURE.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ Linear layer projecting from hidden dimension to vocabulary:
4949

5050
### 6. Gated State Injection (Stability Mechanism)
5151

52+
**Note:** The primary innovation of CRSM is the **asynchronous interaction loop** between the System 1 backbone and the System 2 planner. The Gated Injection formula below is simply the stabilizing mechanism that makes this loop mathematically viable on a continuous manifold.
53+
5254
To safely integrate the asynchronous thought vectors into the sensitive Mamba manifold, CRSM employs a **Gated Injection** mechanism rather than simple additive perturbation.
5355

5456
The update rule is:
@@ -202,4 +204,7 @@ Expected throughput (on A100):
202204

203205
- **S4**: [Efficiently Modeling Long Sequences with Structured State Spaces](https://arxiv.org/abs/2111.00396)
204206
- **Mamba**: [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.08956)
207+
- **MuZero**: [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://arxiv.org/abs/1911.08265)
208+
- **RepE**: [Representation Engineering: A Top-Down Approach to AI Transparency](https://arxiv.org/abs/2310.01405)
209+
- **AlphaLLM**: [AlphaLLM: Monte Carlo Tree Search with Large Language Models](https://arxiv.org/abs/2404.05584)
205210
- **Scaling Laws**: [Training Compute-Optimal Large Language Models](https://arxiv.org/abs/2203.15556)

docs/ROADMAP.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ This phase involved the initial design, implementation, and debugging of the cor
1717

1818
## Phase 2: Initial Training and Validation (🚀 IN PROGRESS)
1919

20-
This phase focuses on moving from a structurally complete prototype to a model that demonstrates meaningful capabilities.
20+
This phase focuses on moving from a structurally complete prototype to a model that demonstrates meaningful capabilities. **Note: At this stage, the performance benefits of the MCTS planner are theoretical. Empirical validation requires the large-scale training scheduled for Phase 4.**
2121

2222
* **[DONE]** **Architecture Verification**: Diagnosed "State Explosion" issue and implemented **Gated State Injection** to solve it.
2323
* **[DONE]** **Safety Mechanisms**: Implemented Confidence Scaling to prevent model lobotomization by untrained planners.

docs/technical_report.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,15 @@ This acts effectively as a low-pass filter for reasoning signals. The planner ge
3838
## 4. Current Status
3939
* **Stability:** Verified via stress tests (1000+ continuous injections). The Gated Injection mechanism successfully maintains numerical stability.
4040
* **Safety:** A "Confidence Scaling" mechanism ensures that if the planner's Value Head is uncertain, the injection rate ($\alpha$) drops to zero, preventing the planner from degrading the backbone's performance.
41-
* **Validation:** Preliminary tests verify that *if* the planner identifies a state with lower expected loss, the injection mechanism successfully steers the generation towards it.
41+
* **Validation:** Preliminary tests verify that *if* the planner identifies a state with lower expected loss, the injection mechanism successfully steers the generation towards it. **Note: The net positive impact on downstream reasoning tasks is currently theoretical and awaits large-scale training (Phase 4 of Roadmap).**
4242

43-
## 5. Current Constraints & Challenges
43+
## 5. Inspirations & Parallels
44+
This architecture draws heavily from existing research:
45+
* **Representation Engineering (RepE):** The idea of steering generation via latent space manipulation.
46+
* **MuZero & AlphaLLM:** The application of MCTS to latent states and language models.
47+
* **Polyak Averaging:** The mathematical basis for the stable "soft update" injection.
48+
49+
## 6. Current Constraints & Challenges
4450
While the architecture functions, several practical bottlenecks remain:
4551

4652
1. **Global Interpreter Lock (GIL):** The implementation uses Python's `asyncio`, which is not truly parallel due to the GIL. This causes minor stuttering in generation when the planner is under heavy load. A robust implementation would require the planner to be implemented in C++ or Rust.

0 commit comments

Comments
 (0)