Lec 54: LLM Assisted SVA generation with Montecarlo Tree Search

Lec 54: LLM Assisted SVA generation with Montecarlo Tree Search

by Prof. Chandan Karfa (NPTEL IIT Guwahati) · View the channel on PSI

Applied Sciences & Engineering AI — Models & Research UYQArtificial intelligence (AI)UYQFGenerative artificial intelligence
🎙 Prof. Chandan Karfa (NPTEL IIT Guwahati) 👥 230K 📅 September 18, 2026 ⏱ 26 min 👁 0 📄 original study 🧭 2026-09-18
Available in: English (current) Français

Keywords

LLMassertion generationMonte Carlo Tree Searchformal verificationSystemVerilog

Summary

The lecture, part of the course ‘Formal Methods for System Verification’ at NPTEL IIT Guwahati, presents a novel method called SAUM (SystemVerilog Assertion generation via Monte Carlo Tree Search with self-refinement) for automatically generating SVA assertions from natural language specifications. The lecturer, Prof. Chandan Karfa, first recaps the traditional verification flow and highlights the limitations of existing LLM-based assertion generation tools, which typically generate a single set of assertions and refine iteratively. SAUM addresses this by employing a pre-processing stage with signal, specification, and waveform analyzers to build a structured information bank. The core uses Monte Carlo Tree Search (MCTS) with four phases—selection, expansion, evaluation, and back-propagation—to explore diverse assertion sets for each signal. A critic LM scores each node, and back-propagation updates path values. Finally, assertions from all nodes are combined, syntax-corrected, and deduplicated. Experiments on two designs (I2C and RB timer) using DeepSeek and Jasper show higher branch, property, and toggle coverage compared to prior work, with about 152 and 46 assertions generated respectively. The lecture concludes with an example of an assertion found by SAUM but not by earlier tools, and notes that the code is publicly available. The work was published at ICLAD.

197 words

Critical Evaluation

Value of the Information & Strength of the Argument

The lecture provides a compelling argument for using MCTS to overcome the limitations of single-path refinement in LLM-based assertion generation. The method is clearly explained with a step-by-step breakdown of the MCTS phases, and the idea of integrating a critic LM for scoring is a practical contribution. The experimental results, showing significant coverage improvements over the closest prior work (Chira), substantiate the claims. The presenter articulates the motivation well, addressing the pitfalls of existing approaches such as vacuous assertions due to signal name mismatches. The argumentation is logical and well-structured, making a strong case for the SAUM framework.

Scientific Rigor, Source Quality, Title Accuracy

The lecture is methodical, presenting the problem, proposed solution, and results in a clear sequence. The presenter references prior works (Chira, LagRV, assert LM) and mentions that SAUM is published, but does not provide citations or URLs within the lecture. The description provides links to the NPTEL course and playlist, which serve as additional resources. The title accurately reflects the content. No contradictions or unsupported claims were noted; the methodology is described in sufficient detail for a lecture, and the results are plausible. The public availability of the code adds to the scientific rigor. The lecture does not engage with potential criticisms or alternative methods beyond a brief mention, but within the scope of a lecture, the presentation is rigorous.

233 words

Title / Content Match

The title accurately reflects the content: the lecture indeed discusses LLM-assisted SystemVerilog Assertion generation using Monte Carlo Tree Search. The title is concise and matches the lecture's focus.

Quality & Reliability

8/10

The lecture is a formal academic presentation of a research method, with clear motivations, detailed steps, and quantitative results. The presenter is a professor at IIT Guwahati, and the work is published (ICLAD), lending credibility. However, lack of external references and limited experimental details slightly reduce the score.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

Contribution & Novelties

The lecture presents an original method (SAUM) that applies Monte Carlo Tree Search to explore the space of possible assertion sets, guided by a critic LM, thereby overcoming the one-shot generation limitation of prior works. The integration of signal mapping and waveform analysis in pre-processing is a practical enhancement. The contribution is demonstrated with concrete coverage gains.

Pour aller plus loin :

  • Monte Carlo Tree Search — MCTS is a well-known algorithm for decision-making; understanding it is essential to grasp the SAUM search strategy.
  • SystemVerilog Assertions — The formal language for assertions used in hardware verification; the lecture focuses on generating such assertions.
  • Formal verification — The broader field that encompasses assertion-based verification; this lecture contributes to automated property generation.

120 words

Radar Profile

The radar profile shows high scores across all dimensions, with a peak in technical level (9) reflecting the lecture's focus on algorithmic details. Quantity and quality are both strong, indicating a dense yet clear presentation. Overall robustness is high, making the content a valuable educational resource.

Reliability 8/10