Click Review to open the workspace, which includes three different table views: “Comparison”, “Base Table”, “GenAI Table”. Additionally, the side panel menu will feature the Metadata Curator, SQL Panel, and Tasks Panel, allowing users to review the metadata of each table.
Comparison Mode
Discrepancies are highlighted. Mismatched cells show base table values (in red) and GenAI Table values (in black). Any cell displaying different values between the two tables is considered a discrepancy. For each discrepancy, an automated task is created and can be found under the tasks section on the right side menu.
Table review in comparison mode
Base Table View
Displays the output uploaded table without overlays.
Table review in base mode
GenAI Table View
Displays the generated table by the Verify system, produced from ADaM and extracted metadata by AI models.
Table review in GenAI mode
Metadata Corrector
To support efficient iteration and reduce the need for external programming adjustments, GenAI Validation introduces the Metadata Corrector.
The Metadata Corrector enables users to update metadata definitions directly within the platform using an interactive metadata editor.
This capability allows users to refine metadata interpretation when discrepancies or incomplete interpretations are identified during review.
Metadata corrections are applied directly to the generation logic and can be used to regenerate the table using the updated definitions.
Users can open the Metadata editor via the edit Button at the footer of the panel or via the change view button at the table panel header.
Within the editor, users can update:
- Source dataset used for the analysis
- ADaM variables and values used for calculations
- Statistical calculation methods
Multiple metadata updates can be applied within a single editing session.
After metadata changes are applied, the system regenerates the table using the updated configuration.
Metadata Editor UI initial view (before the user enters a modification)
Guided Metadata Commands
To simplify metadata editing, the editor provides a guided command interface.
Users can access metadata operations through a command menu that supports structured metadata modifications, including:
- Dataset selection
- Variable selection
- Statistical method definition
Metadata component updates
This structured approach helps ensure that metadata changes remain consistent with standard clinical reporting structures.
Metadata reference menu
Table Regeneration
After metadata updates are saved & applied, GenAI Validation regenerates the table using the updated metadata definitions.
The regeneration process includes:
- Applying updated metadata definitions
- Recalculating table values from the ADaM dataset
- Rebuilding the table structure
- Updating the comparison baseline
If the regeneration process encounters any issues during the regeneration process, then the user will be notified and the changes will not be applied.
Regeneration failed error message
Activity Log Traceability
All actions performed within the Metadata Editor are automatically recorded in the Activity Log to ensure full traceability of metadata modifications and table regeneration activities.
When a user clicks the "Save and Apply" button, any modifications to metadata components within the editor are recorded. This action logs the metadata change request and its application.
Each modification entry records separately with the following information:
- Location of the change (table ID)
- User performing the change
- Timestamp of the modification
- User request prompt
These records ensure that all metadata updates performed through the Metadata Editor are fully documented and traceable within the project’s Activity Log, supporting regulatory compliance and validation governance requirements.
The metadata edits are captured in the Project’s Activity Log
SQL Panel
Shows the SQL used to generate the GenAI table.
Users can copy or download the SQL.
Open SQL panel
Issues & Tasks Panel
Displays discrepancies and manually created issues. Users can set priority, status, and navigate tasks. The tasks can also be managed via the Task Manager. An empty state appears when no tasks exist.
Task Panel empty state
Acceptance / Rejection of Metadata
Users can now formally accept or reject the metadata interpretation:
Accept: Acceptance confirms that the AI interpretation aligns with expectations.
Reject: Rejection enables further review and investigation prior to proceeding with comparison outcomes. Users can select a reason for the rejection, such as wrong/missing source data or incorrect statistical specifications. All actions are recorded in the Activity Log, fully traceable for audit purposes, and associated with the relevant Generation analysis.
Metadata Report
Rather than manually curating alignment, GenAI Validation now provides full transparency into the explanatory metadata used to generate tables. This report enables users to review how the AI interpreted title components, population definitions, statistical calculations, ADaM variable mappings, and column and row structures. This transparency strengthens trust, auditability, and governance of AI-driven generation.
The Metadata Report includes 2 panels:
1. Table Metadata Panel: Provides full transparency into how the table was generated. Includes:
- Source Data: Datasets used (e.g., Big N dataset).
- Variables: Mapping of variables used to calculate the table, filters applied, and derived calculation logic.
- Assumptions: Assumptions taken by the agent.
- Statistics Specification: Statistical methods used.
2. Confidence & Gaps Panel: Provides visibility into how the system handled uncertainty during table generation. Includes:
- Ambiguities: Key assumptions, interpretation choices, and areas requiring user confirmation.
- Data Quality Verification: Checks on data consistency, completeness, and alignment with expected standards.
This panel helps users focus their review on areas that may impact accuracy, supporting confident and efficient validation.
Confidence & Gaps Panel
Workflow Integration
GenAI Validation is fully integrated into the Verify task management system. All discrepancies identified during table comparison are converted into structured tasks and managed through existing Verify workflows. Furthermore, all Generation-related actions (such as table generation initiation, metadata acceptance/rejection, discrepancy creation, and task updates/closures) are recorded in the Activity Log to ensure full compliance and traceability.
How to Use the GenAI Validation Page [Step-by-Step]
- Navigate to the GenAI Validation page to review the AI-generated tables and their metadata.
- Open the Metadata Report to review both the Table Metadata Panel and the Confidence & Gaps Panel to understand the AI's interpretations, variables, assumptions, and any flagged ambiguities.
- Choose to either Accept or Reject the metadata interpretation based on your review.
- If you select Reject, you can optionally provide a reason for the rejection (e.g., Wrong/missing source data, Incorrect statistical specifications) before saving.
- If accepted, proceed to manage any identified discrepancies that are now converted into structured tasks within the Verify workflow.
Comments
0 comments
Please sign in to leave a comment.