How AI is uncovering hidden truths in medical device development

Software as a Medical Device (SaMD) development depends on dashboards, status reports, and requirements databases to monitor progress. Yet these systems often focus on project documentation rather than the actual product, creating gaps between recorded progress and what has been built. Project dashboards, sprint boards, and status reports attempt to answer this question, but in complex SaMD development, these tools frequently describe the state of the project rather than the state of the product. A new method leveraging large language models (LLMs) seeks to bridge that divide by exposing the true condition of SaMD implementations instead of just their reported status.

Current development tools rely on manual updates from team members. Developers mark tasks as finished, engineers log requirements, and testers document verification results. The issue arises when these inputs fail to align. A requirement may be marked implemented even though its associated risk control has not been fully integrated, or a software story may be closed because the core application logic is complete while the user workflow, data handling, exception management, or verification evidence remains unfinished. Conversely, functionality may already exist in the codebase but remain disconnected from requirements, tests, or planning artifacts. These discrepancies are often discovered during system integration, formal verification, or design reviews—precisely when correcting them becomes more expensive.

LLMs Move Beyond Code Generation to Implementation Tracking

Discussions about generative AI in SaMD have primarily emphasized productivity gains: faster code writing, debugging, and unit test creation. However, LLMs could deliver greater value by analyzing repositories, interfaces, and commit histories to determine the actual state of the product. This analysis could answer essential questions about implementation:

Which clinical workflows are fully operational and which remain partially supported?
Are temporary functions or simulated integrations still present in the codebase?
Does verification documentation correspond with the current code version?
Which completed work items contain unresolved dependencies?
Has a risk control been implemented across every relevant execution pathway?
Which requirements appear to have code changes without updated tests?

This implementation awareness does not replace formal traceability or engineering expertise. Instead, it serves as an additional verification layer, helping teams identify discrepancies earlier. The objective extends beyond progress tracking to ensure that what is built matches the intended design. The most important contribution of LLMs to regulated software development may not be generating code faster, but helping engineering teams understand what has actually been built, what remains incomplete, and where implementation has drifted from documented intent.

Take a remote cardiac-monitoring system designed to detect arrhythmias and alert medical staff. Documentation might show the feature as complete—the algorithm executes, the dashboard displays results, and most tasks are marked done. Yet an LLM-assisted review could reveal hidden issues:

Signal-quality checks were bypassed during manual file processing.
A clinician interface displays confidence scores, but one code path defaults to a fixed value.
Notification service failures went unrecorded.
Tests covered standard detection but ignored degraded-signal scenarios.
A risk-control requirement linked to the feature lacked verification tests.
Placeholder functions or mocked integrations remained in the codebase.

These are not trivial oversights. They represent risks to safety, traceability, and usability—the exact type of problems implementation awareness can uncover before they worsen. The capability is not merely about identifying “80% completion,” but pinpointing specific gaps that affect critical system behaviors.

Vertical Slicing Enhances AI Evaluation Precision

LLM-based analysis functions most effectively when development follows vertical slicing, end-to-end functional units covering all necessary layers. Traditional segmentation by frontend, backend, or algorithms can obscure whether a capability is fully usable, safe, and verifiable. Vertical slicing addresses this limitation by structuring work around coherent system capabilities rather than isolated technical components.

In the cardiac-monitoring example, the relevant unit is not just the “arrhythmia algorithm” or “clinician dashboard.” It is the complete workflow: data collection, analysis, prioritization, clinician review, and event recording. Organizing work this way provides the AI with clearer criteria for evaluation. Rather than verifying individual components, it assesses whether the entire capability operates correctly across all layers, including data completeness checks, preprocessing logic, and exception handling pathways.

Initial deployment of AI-assisted workflows would likely begin with advisory rather than automated functions. Teams would define a functional slice, requirements, tests, and interfaces, and submit it to the LLM. The system would then compare intended design with available evidence, flagging discrepancies without altering controlled records. Example output might include:

Signal processing: Missing checks for manual-reprocessing scenarios.
Clinician interface: Inconsistent confidence-score handling in one code branch.
Notification service: Unlogged failures found in test logs.
Verification: Absent tests for risk controls in delayed-alert cases.
Traceability: Code changes without corresponding updates to linked requirements.

Instead of a generic completion percentage, the assessment would pinpoint specific gaps, converting vague progress estimates into actionable findings. The focus remains on supporting human judgment with data-driven visibility into actual implementation status, including how recent code changes may have invalidated existing verification assumptions.

The difference between reported progress and product readiness is fundamental in SaMD development. Most planning tools rely on human-reported status, where developers update work items, systems engineers maintain requirements, and test engineers record verification results. Each activity is necessary, but the resulting view is only as accurate as the synchronization between the underlying artifacts. In a large SaMD program, relevant information may be distributed across requirements systems, architecture documents, continuous integration pipelines, and test-management tools. No single artifact provides a complete picture of product readiness, as the source code may show capability exists without confirming it meets requirements, or tests may pass without validating recent changes.