The 2024 ESA Dry Run tested over 1,000 financial institutions for compliance with DORA, revealing critical gaps in data quality, third-party oversight, and reporting accuracy. Here's what you need to know:
- Top Challenges: Missing SLAs (27%), invalid Legal Entity Identifiers (32%), and formatting errors caused 41% of rejections.
- Sector Error Rates: Credit institutions (1.9%), investment firms (2.4%), and insurance companies (3.3%) faced varying issues.
- Improvement Areas: Better tools for data validation, third-party data collection, and compliance software are essential.
- Upcoming Changes: New validation rules (v4) and centralized DORA reporting portal launch in 2025.
Quick Action Plan for 2025 Compliance:
- Adopt Two-Stage Validation: Technical and data quality checks.
- Leverage Tools: Use DORApp or advanced compliance software.
- Monitor Metrics: Track Third-party Dependency Ratio (<15%), Contract Coverage Index (>95%), and Validation Success Rate (>98%).
Stay ahead of DORA requirements by addressing the 2024 findings and implementing these steps to ensure compliance in 2025.
Recording of the DORA Dry Run Summary workshop
2024 ESA Dry Run Results
Participation Results by Sector
The exercise revealed clear differences in performance across sectors, driven by operational challenges unique to each. These findings point to areas where compliance teams should prioritize improvements before the 2025 rollout:
| Financial Sector | Error Rate | Primary Challenge |
|---|---|---|
| Credit Institutions | 1.9% | Contract date formatting |
| Investment Firms | 2.4% | Service scope classifications |
| Insurance Companies | 3.3% | Termination clause documentation |
Organizations that participated in ESA workshops showed much better outcomes, cutting their error rates by 40% compared to those that didn’t attend [2][4]. Timing also played a role - entities submitting in early June averaged 2.3 resubmissions, while those filing in August (after updated guidance) reduced this to 1.5 [2].
Data Quality and Reporting Results
Participation trends directly impacted data quality, with several recurring issues identified. ESA’s three-stage validation process uncovered the following consistent reporting flaws:
- Documentation Gaps: 22% of submissions were missing key fields, such as contract start and end dates.
- Category Inconsistencies: 18% misclassified ICT services.
- Location Codes: 13% included incorrect country identifiers [2][4].
Updates to reporting tools in Q3 2024 helped mitigate these problems, especially for institutions dealing with legacy contracts that lacked essential metadata [2][1].
The most pressing challenges for participants were:
| Challenge Area | Percentage of Queries |
|---|---|
| Legacy System Integration | 42% |
| Third-party Data Collection | 33% |
| Reporting Format Requirements | 25% |
These ongoing issues will shape the ESAs' updated 2025 guidance, which is discussed in upcoming sections.
Main Problems Found in 2024
Data Collection Issues
Several systemic issues related to data collection came to light, adding to the sector-specific challenges already identified. A significant number of submissions failed basic integration checks due to inconsistent file naming and accidental inclusion of test files [2].
Here’s a breakdown of the most common data collection errors:
| Error Type | Frequency |
|---|---|
| Missing SLAs | 27% |
| Business continuity gaps | 19% |
| Invalid dates | 14% |
Third-Party ID Problems
Identity validation proved to be a major weak point, with 32% of submissions containing invalid Legal Entity Identifiers (LEIs) for ICT providers [2].
"The dry run revealed significant challenges in identifier consistency, particularly regarding non-EU third party providers using alternative identification systems." - ESMA Technical Report [2]
Some firms complicated the process further by using tax IDs without proper validation mappings, which made ESA monitoring much harder [4]. These inconsistencies created significant reconciliation challenges for the centralized monitoring systems used by ESAs [4].
Validation Tool Limitations
The current validation tools fell short of meeting ESA's technical standards. In fact, 41% of rejected files were due to formatting errors in mandatory fields like "contract_termination_conditions", caused by these tools [2]. Metadata-related problems were among the most frequent issues [2].
Key shortcomings included:
- Inability to handle ESA-required CSV/XML formats.
- Missing 15% of errors during validation.
- Lack of necessary validation layers [2][5].
To address these gaps, ESAs plan to roll out updated validation rulesets (v4) in Q2 2025, followed by sandbox testing environments in Q3 [4][5]. These updates highlight the pressing need for improved compliance software, as discussed in the next section.
sbb-itb-107f699
Steps for 2025 DORA Compliance
Data Management Steps
To address the challenges posed by 2024's validation tool limitations and gaps in data, financial entities should adopt a two-stage validation process. This approach ensures compliance with the European Supervisory Authorities' (ESA) rigorous standards.
The first stage focuses on technical validation, ensuring file integrity and proper encoding. The second stage targets data quality, applying 116 predefined checks to improve accuracy and reliability [2].
| Validation Layer | Required Actions | Compliance Target |
|---|---|---|
| Technical | File integrity checks, UTF-8 encoding with BOM headers | Q1 2025 |
| Data Quality | LEI validation, date format standardization | Immediate |
Compliance Software Options
Meeting these technical requirements calls for specialized compliance software. The right choice depends on your organization's size and complexity. ESA's analysis highlights different needs for different types of entities [2][3].
Large institutions, for instance, often require advanced tools like AI-powered anomaly detection and automated XBRL solutions to handle intricate networks of providers. Top-tier software solutions typically include:
- Automated alignment with EBA taxonomy
- Multilingual submission capabilities across the EU
- Version control with ESA update tracking
- Fully compliant audit trails [2][3]
Template Updates and Changes
The introduction of the TPR-045 field addresses gaps in cybersecurity documentation by mandating the inclusion of cybersecurity certification status [2]. Organizations will need to adjust their reporting processes to meet this new requirement while ensuring data precision.
A structured validation strategy has proven highly effective. Organizations that refine their processes have achieved error rates below 0.5% by their third review cycle [5].
| Metric | Target Threshold | 2024 Sector Benchmark |
|---|---|---|
| Data Completeness | >99.5% | Credit institutions: 98.1% |
| First-Attempt Success | >97% | - |
| ICT Provider Coverage | 100% | - |
To manage templates effectively, organizations should conduct biweekly data quality audits. Leveraging ESA's error code matrix (REF-ERR-001 to REF-ERR-116) can help maintain high accuracy and compliance levels [2].
Summary and Next Steps
Key Test Findings
Credit institutions showed fewer errors (1.9%) compared to insurance firms (3.3%), highlighting sector-specific readiness gaps [2]. Investment firms, which made up 28% of submissions, reported a moderate error rate of 2.4% [2].
Data quality was the biggest hurdle, with only 62% of submissions including valid Legal Entity Identifier (LEI) codes for third-party providers [2]. Smaller payment institutions faced the most technical formatting issues, responsible for 41% of rejected files [2].
Regular Monitoring Requirements
Starting Q1 2025, organizations must track three mandatory metrics:
- Third-party Dependency Ratio (TPDR): Critical operations relying on single vendors must stay below 15% [3]. Real-time LEI validation APIs linked to the GLEIF registry are recommended for accurate tracking.
- Contract Coverage Index: At least 95% of active agreements must meet DORA-compliant terms [1]. Automated tracking tools with alerts at 90, 60, and 30 days can help ensure compliance.
- Validation Success Rate: A pass rate above 98% per submission batch is required [5]. Machine learning-based error detection systems should align with ESA's upcoming validation ruleset v4 to meet this target.
These metrics build on the operational benchmarks outlined in the Data Management Steps document.
The ESAs will introduce a centralized DORA reporting portal by March 2025. This portal will include built-in validation checks, expanding the earlier biweekly audit recommendations and providing a structured escalation process for recurring issues.
FAQs
What is a requirement for operational resilience testing under DORA?
Under DORA, all critical ICT systems and processes must undergo testing annually [1]. For institutions managing assets over €100 billion, the rules are stricter, requiring biannual threat-led simulations [3]. These guidelines build on insights from the 2024 Dry Run exercise.
The annual tests aim to ensure businesses can continue operating during disruptions. The EBA also mandates that at least 0.25% of the IT budget be allocated to resilience testing activities [1].
Key testing activities include:
| Testing Element | Frequency | Requirements | Priority |
|---|---|---|---|
| Comprehensive System Testing | Annual | All critical ICT systems | High |
| Vulnerability Assessment | Quarterly | Core infrastructure | Medium |
| Threat Monitoring | Continuous | SIEM solution usage | High |
To address the gaps identified in 2024, organizations are encouraged to focus on implementing SIEM solutions and using standardized assessment protocols [6][1].
The ESAs have incorporated these testing requirements into their validation framework, offering automated systems to help organizations meet compliance goals [2][5].
Related Blog Posts
- XBRL Reporting for DORA: A Complete Implementation Guide
- DORA Compliance Checklist: 10 Key Requirements for 2025
- Is excel an appropriate tool to maintain the DORA ICT Third-Party Register of Information?
- Overview of DORA’s Draft Regulatory Technical Standards (RTS) and Implementing Technical Standards (ITS)
