Clinical Data Management: Everything You Need to Know

Clean, consistent, high-quality data is the lifeblood of a clinical trial.

Clinical data management (CDM) is the collection, validation, quality management, integration, and delivery of clinical data to deliver high-quality, reliable data for statistical analysis, regulatory compliance and submission, medical decision-making, and further clinical research.

Originally, CDM was manual, paper-based, and error-prone. In the 1990s, paper case report forms (CRFs) began to be digitally transcribed into CDM databases, but high error rates persisted as a result of duplicate and incorrect entries. Over the last few decades, advancements have transformed CDM beyond recognition through:

• Technology advancements: Cloud solutions, clinical trial platforms, centralized data, artificial intelligence (AI)  machine learning (ML), wearable devices, etc.
• Advanced quality and CDM: Principles, standards, and methodologies–quality by design (QbD), risk-based quality management (RBQM), CDISC, and others.^1,2,3
• Advances in clinical research processes and methodologies: Adaptive studies and decentralized trials (DCT).
• The shifting regulatory landscape: Regulatory practices have evolved in line with advancements in technology and methodologies.
• Data interoperability standards and initiatives: These advancements have resulted in unprecedented volumes of data being produced at high velocity. The scope of CDM has also expanded to inform and support other clinical study activities.

The resulting risks and challenges faced include:

• Data overload and system compatibility issues
• Evolving regulatory demands across jurisdictions
• Ensuring consistency in multi-center/global trials
• Limited CDM expertise in emerging markets
• Interoperability between disparate systems
• Cybersecurity and patient privacy threats

Data is collected from many disparate sources: patients, sites, labs, EHRs, sensors, imaging, and others. This includes semi-structured or unstructured data, which is inconsistent, e.g., physician notes, patient-entered data, and erroneous data entry. Data sources, such as different EHRs, may contain different data for the same patient, producing inconsistencies in medical histories and leading to potential patient safety risks.

Data collection, reconciliation, standardization, quality checks, and medical coding are manual and semi-automated processes that are resource-hungry, time-consuming, and error-prone.

Higher volumes of data present valuable benefits for clinical research, but they also present challenges in managing that data, ensuring patient safety, and meeting regulatory compliance.

To overcome these challenges, the industry must move away from outdated, siloed approaches toward a more fully integrated, platform-based approach that leverages AI-powered solutions and automation to connect study workflows into a unified experience. Data integration and simplified workflows throughout the trial lifecycle can significantly enhance collaboration among sponsors, CROs, sites, and patients, promoting faster and more informed decision-making and better compliance with regulatory requirements.

Medidata is leading this transformation through the delivery of unified experiences on the Medidata Platform. The Medidata Patient, Study, and Data Experiences, powered by automation and AI, improve patient engagement through digital technologies, enhance data quality with actionable insights, and optimize study design with predictive analytics. The Medidata Data Experience simplifies and accelerates clinical data management workflows from study build to data acquisition, data integration, data review, and database lock.

Read on for a comprehensive overview of CDM—the lifecycle of the data management process, tools and technologies, regulatory compliance, data standards, use cases, and the future of CDM.

The Clinical Data Management Lifecycle and Best Practices

Both the Association for Clinical Data Management (ACDM) and Society for Clinical Data Management (SCDM) are key sources of guidance.^4,5  The following stages and steps are drawn from their guidance for best practices and data management planning.

Clinical Data Management Stages

Strategy and Planning

A data strategy is created for the study and data quality management methodologies; frameworks are agreed upon for different aspects of the trial.

Clinical research teams are increasingly moving from siloed working practices to collaborative frameworks such as RBQM, while also leveraging centralized data management resources and systems. An integrated quality management plan and data management plan are created, then used by clinical data management, risk management, central monitoring, medical monitoring, and other stakeholders/participants. 

The CDM lifecycle has three main stages: study set up, study conduct, and close out.

Within these stages, the steps are as follows:

1. The Study Set Up Stage

This stage lays the foundations for a clinical trial.

The data management team reviews the protocol and creates a clinical study protocol summary. Based on input from the study team, a study timeline is also created.

The data management team creates records that include key contacts, study documentation, data management tools and systems to be used in the study, system integrations, and medical coding dictionaries.

Document management is set up with an electronic trial master file (eTMF). 

Systems, libraries, and integrations, including EDC systems and global libraries, are checked to make sure they’re validated and compliant. Vendor compliance is also checked.

Other setup tasks in this stage are:

Randomization (except in open-label studies): Randomization is conducted by a randomization and trial supply management/integrated randomization tool (RTSM/IRT) integrated with the clinical database. All data management activities are performed in a blinded state from the randomized data, which is then reviewed by the study team prior to go-live and documented within the unblinding plan. 

System integrations: Common system integrations used in studies include a clinical trial management system (CTMS), coding tools, EDC, and statistical computing environments (SCE). More on technology appears later in this article.

Medical coding: Standardizing medical terms whilst maintaining accuracy, consistency, and preserving context is critical.

Medical coders also create proprietary synonym lists that are used and maintained to speed up the complex and time-consuming task of coding.

Data validation specification: A DVS document lists all data cleaning activities to be performed. It is also known as a data handling plan or edit check specification. The aim is to ensure that critical data is accurate and clean in preparation for statistical analysis.

Case report form design and completion: eCRFs are designed to collect study-specific data from patients that align with the protocol/study endpoints. As they are patient-facing, the clarity of materials is important, as is the logical flow of the form design.

Links to guidance on eCRF completion are provided for the data management team.

Transfer agreements (non-CRF data): Agreements and specifications on how, what, and when non-CRF data is transferred by third-party vendors are filed. Non-CRF data is data from other sources such as labs, ePRO/eCOA, imaging, sensors, etc.

Clinical database design, initial release, and change management: The clinical database design and build is structured to reflect the CRF logic and meet regulatory standards, then integrated with EDC and coding dictionaries.

It is recommended that a ‘go-live’ initial release checklist be followed, including validation and user acceptance tests (UATs) for integrated tools.

A change log must be created and approved for any changes after the initial release of the clinical database.

2. The Conduct Stage

The conduct stage begins after approval of the clinical study database.

The steps are: 

User access to the clinical database: Relevant stakeholders, such as data managers, biostatisticians, and clinical data coders, are granted system-specific user access.

Data extraction: Automated or manual data extraction of the clinical database is carried out on an ad hoc or regular basis for analysis, quality checks, validation, and compliance.

Data cleaning: Validation and cleaning of clinical data is required to ensure data completeness and integrity for audit trails—and documentation for transparency and compliance. This stage involves edit checks, discrepancy management, and query resolution.

Continuous data review: Data review is conducted continuously during and after source data validation (SDV) and at the end of every cohort, part, and study close.⁶ The data manager also reviews the system and programmable checks. During this process, data reconciliation and validation are carried out, and areas with high data query rates are identified.

Metrics: Standard reports, metrics, and key performance indicators (KPIs) help teams to understand the status of a study, identify and manage issues, and measure efficiency and quality. Example KPIs include:

• Query rate per CRF page
• Time to resolve queries
• Data entry errors
• Percentage of clean data at interim/final database lock
• Data points created large volumes of queries

Medical coding review: The medical coder uses MedDRA, WHODrug, or similar dictionaries to review the coding, raising queries directly within the clinical database for clarification at the sites.^7,8

All complete and applied terms are approved, documented, and filed within the eTMF prior to interim or full database lock.

Serious adverse event (SAE) reconciliation: SAE data and actions are reconciled between the clinical and safety databases.

Non-CRF data reconciliation: Non-CRF data is reconciled regularly throughout the study conduct stage.

Risk assessment & action/decision logs: Risk assessments are carried out, reviewed, and documented in a risk assessment log.

Interim lock: An interim analysis can be carried out during the conduct of the study; it’s typically a manual, time-consuming process. Often, sites can’t continue to enter data for a period of time while analysis takes place.

Protocol amendment or mid-study design changes: Mid-study design changes or protocol amendments are quite common in studies—particularly in phase II and III studies. Read more about their impact and how to manage changes here.

3. Closeout

Database lock: Database lock is a major clinical trial milestone. Final checks are made before locking.

Some databases have a two-step lock: a soft lock (data can still be queried and edited) and a hard lock (no editing or access is possible).

Preparing data for regulatory submission: Data is standardized to allow for interoperability between systems, analysts, and organizations. It’s also a regulatory requirement. In preparation for regulatory submission, SDTM datasets are applied for organizing and formatting data following database lock, though some of this can be carried out pre-lock. SAS programming and tools are used to generate SDTM datasets, generate Define.xml files, verify data, and generate reviewers’ guides. Analysis data model (ADaM) numerical datasets provide the connection between SDTM datasets and final statistical analysis, and TLF (Tables, Listings, and Figures) datasets summarize datasets into an easily readable format.

Database unlock: If modifications are needed that are impactful to analysis after the database lock, approval needs to be sought to unlock the database to make changes. This will be documented and requires approval from all stakeholders. Once modifications are made, the data is quality checked and the database re-locked.

Post-lock steps include data extraction and transfer, data archiving, and decommissioning.

Once the database is locked, data is securely extracted for a final analysis within a statistical computing environment (SCE). All study data documentation is archived. Submission-ready subject data is uploaded to the eTMF or designated repository. The clinical study database is decommissioned after the submission-ready subject data files are created, QC checked, and received by the clinical study site.

Tools & Technologies in Clinical Data Management

Technology plays an essential role in enabling and empowering stakeholders across the entire CDM lifecycle. Manual processes are removed wherever possible due to the risk of human error and data duplication, and their inability to manage vast volumes of data. Electronic collection and management tools enable real-time data capture from many disparate sources, direct entry into key systems, and automated data management capabilities such as validation, cleaning, and analysis.

The ideal CDM system involves integration and data interoperability between the following technologies and other systems (such as those used by labs), bringing all the data into a centralized system of data management.⁹

The most commonly used tools and systems are (in alphabetical order):

Clinical data management systems (CDMS): A system for centralized data integration, review, reconciliation, and analytics, such as Medidata Clinical Data Studio.

Clinical/medical dictionaries and technology: Technology such as Medidata Rave Coder+ provides a level of automated medical coding alongside tools that browse and search industry-standard dictionaries. Rave Coder+ also creates and maintains proprietary synonym lists and uses AI to predict codes for verbatim terms.

eConsent: eConsent replaces paper consent forms, simplifying patient enrollment and onboarding and improving patient engagement. It also provides consent tracking management and reduces administrative burdens and informed consent errors.

EHRs: Access to a clinical trial patient’s EHR is vital for informing clinical researchers and patient safety. EHRs often hold source data, such as lab results, vital signs, medications, and medical history, that is required by eCRFs in the EDC system. EHR-EDC solutions such as Medidata Rave Companion can automate this process.

Electronic clinical outcome assessments (eCOAs) and electronic patient reported outcomes (ePROs): Systems that collect electronic data using industry-standard questionnaires from patients, physicians, and caregivers, such as myMedidata.

EDC systems: EDCs capture, manage, clean, and report site, patient, and lab-reported data. Advanced EDCs, such as Medidata Rave EDC, streamline workflows, eliminate manual data reconciliation, and deliver insights.

Imaging: Medical imaging, such as scans and x-rays, is an important and complex source of data. Systems such as Medidata Rave Imaging support the management and review of images.

RTSM/IRT: These technologies, such as Medidata Rave RTSM, are used for randomized patient/subject allocation, inventory management, and logistics for the delivery of drugs or devices.

Sensors and wearable devices: Advances in sensor technology, such as wearable devices, have enabled episodic or continuous real-world data collection/monitoring using sensor capabilities on clinical trial platforms such as the Medidata Platform

Statistical computing environment (SCE): These are advanced analytics systems that include data ingestion, complex and predictive data modeling and analysis, and data visualizations.

Standalone Software vs Clinical Data Technology Platforms

Implementing disparate standalone software offerings and services for clinical trials is known to introduce a high risk of implementation delays and integration problems.

The most successful approach has been to implement an established, unified platform with proven third-party integration and data interoperability capabilities.

Regulatory Compliance and Data Standards

Data accuracy, consistency, and completeness are crucial to making sure that submissions to regulators are successful.

Any technology solutions implemented must also comply with strict regulatory requirements, including validation, security, and audit trails.¹⁰

CDM is subject to many regulatory requirements and data standards at the global and national levels. These include (but are not limited to) ICH E6(R3)/E8 (modernized guidelines and risk-based approaches), FDA 21 CFR Part 11 (digital signature validation and audit trails), and GDPR & HIPAA (data privacy considerations in multinational trials).^11,12,13,14 

Use Cases and Case Studies

Having supported over 2,300 customers in 36,000+ clinical trials involving 11 million+ participants, Medidata has many data management case studies and testimonials to share.

The Medidata Big Book of CDM Case Studies provides a few examples. One example includes the transformational project with PHASTAR, a biometrics contract research organization (CRO) that partnered with Medidata to improve the visibility and oversight of data collection, speed up trial implementation, and enhance patient engagement.

Luke Gregory, Senior Director of Clinical Systems at ICON plc, discusses here how Medidata Clinical Data Studio helped ICON harmonize its vision of clinical data science within its organization.

Swathi Vasireddy, Associate Director, Clinical Data Management, Corcept Therapeutics, discusses visibility of real-time data and anomalies, tracking data trends, resolving issues, cleaning data, and shortening the time to database lock in her testimonial here.

Further clinical data management testimonials can be seen here.

Future of Clinical Data Management

‘The focus of clinical data management teams is shifting from reactive, exhaustive data cleaning to proactive, risk-based clinical data science.’

The Medidata article Clinical Data Science 101: The New Clinical Data Management and the associated webinar with the SCDM provide insights into the industry's shift from data management to data science.¹⁵

Supporting this evolution for data managers, the SCDM has published reflection papers on the transition from CDM to clinical data science (CDS) since 2019.¹⁶ Four main themes have remained the core drivers for this evolution:

• The complexification of clinical trial designs (e.g., adaptive, master protocols, synthetic control arms)
• The decentralization of clinical trials (DCT)
• The adoption of risk-based CDM approaches that foster quality by design (QbD) and the focus on what matters most
• The automation of CDM activities

Other factors include the integration of real-world data (RWD) and wearable technology and data standardization (e.g., CDISC) across global sponsors.

The SCDM defines CDS as:

“Clinical data science is an evolution of clinical data management. Clinical data science encompasses domain, process, and technology expertise, as well as data analytics skills and good clinical data management practices essential to prompt decision making throughout the lifecycle of clinical research. Clinical data science can be defined as the strategic discipline enabling the execution of complex protocol designs in a patient-centric, data-driven, and risk-based approach, ensuring subject protection as well as “the reliability and credibility of trial results.”

The Five V’s of Clinical Data (Volume, Variety, Velocity, Veracity, and Value) form part of the framework that clinical data managers are using in their transition to clinical data scientists. An excellent resource on this topic is the eBook Driving Data Quality 5Vs, which is freely available here.

Since 2019, CDM challenges have increased, and there is a clear need for a shift to CDS that uses proactive, targeted, and risk-based approaches combined with centralized data management and automated, intelligent workflows and analytics.

Data managers are already able to leverage AI and advanced technology for complex tasks such as anomaly detection, coding, audit trail reviews, and data reconciliation. Soon, this will extend across all areas of a study, from protocol design to preparing data for regulatory submission, vastly reducing the time to database lock.

Conclusion

Technology delivers transformational benefits for clinical research. The increase in volume and velocity is such that advanced technology, targeted risk-based methodologies, and centralized data management are needed to effectively manage and leverage that data.

At present, technology empowers data managers in many key areas of CDM. Soon, it will be able to support every area of the data management lifecycle.

Data management has evolved into data science. Data managers are upskilling as they transform into clinical data scientists and strategic advisors.

The Medidata Data Experience connects every part of the clinical data management process. From study build to data capture to database lock, it leverages built-in AI and automation to simplify workflows, reduce manual effort, improve data quality, and shorten trial timelines, turning raw data into real insights faster.

Learn more about how we can help you advance your clinical data management practices by speaking to an expert here.

References:

1. Quality by design | European Medicines Agency (EMA)
2. RBQM - The Role of Source Data Verification (SDV) and Source Data Review (SDR) in Driving Clinical Trial Data Quality
3. CDISC
4. ACDM. Guides and Templates
5. Journal for the Society for Clinical Data Management. Data Management Plan
6. ibid., RBQM - The Role of Source Data Verification (SDV) and Source Data Review (SDR) in Driving Clinical Trial Data Quality
7. MedDRA
8. UMC's Drug Dictionary WHODrug Global (UMCDD)
9. Interoperability in Healthcare - Everything You Need to Know
10. Guideline on computerised systems and electronic data in clinical trials | EMA
11. ICH E6 Good clinical practice - Scientific guideline | European Medicines Agency (EMA)
12. FDA Part 11, Electronic Records; Electronic Signatures - Scope and Application
13. GDPR - Regulation - 2016/679 - EN - gdpr - EUR-Lex
14. Summary of the HIPAA Privacy Rule | HHS.gov
15. Clinical Data Science 101: The New Clinical Data Management
16. The Evolution of Clinical Data Management into Clinical Data Science An SCDM Position Paper on how to create a Clinical Data Science Organization