Critical Data Elements Interim Report
Best Practice Scribes
Mark McQueen, EDM Council, Senior Advisor-DCAM
Michael Atkin, EDM Council, Strategic Advisor
The issues surrounding the designation and management of critical data elements ranked as the number one priority in the 2017 benchmarking study conducted by the EDM Council. Because of its importance, we formed a working group of interested members. We conducted 14 in-depth interviews with member firms to frame the issue. We organized our basic research to define the challenges financial institutions are facing with definition, lineage traceability, governance, and quality management. This is the interim report. Members are invited to join the working group as we move this activity from framework to recommendations about best practice.
What is a Critical Data Element
Criticality is determined from the perspective of the data consumer. The highest priority candidates are those that have an impact on a regulatory report, financial calculation, performance indicator or risk measure. One of the biggest challenges is the lack of consistency in distinguishing a granular data attribute from a derived or calculated business measure. Many firms are using the same nomenclature to describe logical concepts, business objectives, calculation processes, derived elements, and physical expression. The concepts described above are all real and important “things” – but they are not the same thing – and by calling them all “critical data elements” we add confusion to an already difficult task.
How are CDEs Designated
Defining criticality is dependent on understanding the business processes associated with how the data is used in the real world. Many organizations start with a “candidate CDE” that originates from the data requirements capture and verification process. This process is frequently the result of an internal “data forensics” and “negotiation” mechanism between the data consumer and the data producer. The forensics (or discovery process) is driven by the knowledge of subject matter experts/business analysts. It is dependent on an analysis of data flow. Tracing the flow of data is relatively straightforward to envision but compounded by the challenges of fragmentation with multiple repositories, incomplete inventories, hard to obtain business knowledge and the plethora of end user spreadsheets/local data repositories. Organizations are using their formal governance structures to approve the designation of criticality and to reconcile gaps in understanding about granular data inputs versus business process outputs.
CDE Implications
The designation of a data element as critical means it is governed by organizational policy and standards. This translates into increased data management rigor being applied to the attribute. The following were common themes across the universe of firms involved in the initial discussions:
- Definition: Designation as a CDE reinforces the importance of alignment to precise meaning for all front-to-back applications, all business processes and all derived formulas that consume the attribute. The CDE must be understood in context of how it is used. Locking down meaning, unraveling front-to-back nuance, managing misuse of data elements and preventing users from changing meaning to fit the requirements of applications remain the top challenges to address.
- Lineage: Unraveling technical and business lineage requires significant effort. The forensics of data flow, rationalizing across IT systems and unwinding transformations are accomplished based on interviews between producers, consumers and “armies of consultants” that all need to collaborate to stitch the baseline together. Not only is lineage complex, it can be inhibited by the lack of standard approaches, confusion on terms/meaning, questions about value and incomplete levels of expertise. The translation of the “tangled web” of business processes and outputs into data precision is not always obvious (particularly for legacy products). Lineage is compounded because it is done at a specific “point-in-time” and not aligned with the realities of dynamic and interconnected business/IT environments. Financial institutions are using various starting points (i.e. unravel reports, trace from logical model, reverse engineer technical systems) to align data with precision.
- Data Quality: Designation as a CDE means subjugation to a higher level of data quality control. This includes negotiating agreement across multiple stakeholders on the expression of fit-for-purpose criteria, quality tolerance ranges/thresholds, business rules, testing requirements, and measurement criteria. For most organizations, data accuracy is NOT the main problem. Many organizations are implementing quality control processes (i.e. triangulation, source comparison, ADS provisioning rules) to ensure accuracy. And while accuracy is critical, the biggest data quality challenges appear to be associated with ensuring alignment to precise meaning, unraveling complex data manufacturing processes and performing root cause analysis needed to drive remediation.
- Governance: Managing the relationships between data producers and data consumers is the most intensive part of the governance challenge because it requires collaboration across multiple stakeholders and precious resource (time) commitments. Designation of data attributes (and business outcomes) as CDEs require accountability and agreement on prioritization. The essence of governance means the performance of due diligence over the CDE management process. This can be daunting because managing meaning can be “tedious” and “boring” making it hard to capture the attention of senior business participants. The importance of the CDE process is only recognized when it becomes a priority and stakeholders only care up to a certain point. Identification of the “point of caring” and managing incentives correctly is the art of governance and the key to success.
- Metadata: CDE management is expressed as metadata. This includes metadata about business concepts and technical flows as well as the attributes themselves. Metadata about CDEs includes (but is not limited to): term names, business definitions, identifiers, aliases, authorized data domains, authorized provisioning points, owners, stewards, custodians, operational processes/data flows (authoritative source to consumption), timestamps, lineage diagrams (including derived processes), accuracy monitoring, origination (source of data), consuming applications, logical to physical mapping, allowable values, inventory locations, classification of criticality and classification of sensitivity. This is where the maturity of the metadata tools and their ability to integrate into the fabric of the organization becomes essential.
CDE Management Challenges
Every financial institution that we interviewed is deeply engrossed in the designation and management of CDEs – and progress is both clear and substantial. The number one issue (across the board) is the challenge of locking down the precision of meaning and harmonizing nomenclature. The implementation of governance and the creation of “control processes” within federated environments remains a daunting activity because these are big “change management” tasks that translate into robust producer, consumer and process accountability requirements. These governance challenges are compounded by the difficulties of keeping processes current, managing turf and reining in rogue operations.
The EDM Council’s CDE Working Group is operational. We have finished the issue framework phase and are set to translate this essential capability into best practice recommendations. We invite you to join us.
The Council would like to thank the following people for their participation in the CDE research or as members of the CDE Working Group: Bala Ayyar (Société General), Sathya Bala (Deutsche Bank), John Bottega (EDM Council), Lisa Davis (TIAA), Genevy Dimitrion (State Street), Michael Finnen (Mitsubishi UFJ Financial Group), Christopher Giardina (IBM), Andrew Gordon (Deutsche Bank), Steve Goulas (State Street), Allie Harris (Bank of Montreal), Matthew Hawkins (Goldman Sachs), Gareth Isaac (Ortecha), Denise Jeffries (Regions Bank), Kathryn Klaentschi (Deutsche Bank), Andrew Lawson (Brickendon), Ari Marcus (Citi), Yogan Naidoo (KPMG), Kamal Panda (Wells Fargo), Tamzin Rayleigh (Macquarie Group), Peter Serenita (Scotiabank), Suriya Subramanian (independent), Tim Swan (Union Bank), Alec Tang (ADIA), Mildred Townsend (Charles Schwab), John Yelle (DTCC)
Respectfully Submitted,
Michael Atkin, Mark McQueen
EDM Council, November 2017
Revision History
| Date | Author | Description |
| November 2017 | Michael Atkin, Mark McQueen | Initial Publicaiton |
| September 2019 | Mark McQueen | Knowledge Portal Release |
