skip to content

Technical Challenges associated with record linkage

Presented by: 
James Boyd
Tuesday 13th September 2016 - 10:00 to 10:30
INI Seminar Room 1
Background: The task of record linkage is increasingly being undertaken by dedicated record linkage units with secure environments and specialised linkage personnel. In addition to the complexity of undertaking record linkage, these units face additional technical challenges in providing record linkage ‘as a service’. The extent of this functionality, and approaches to solving these issues has had little focus in record linkage literature.

Methods: This session identifies and discusses the range of functions that are required or undertaken in the provision of record linkage services. These include managing routine, on-going linkage; storing and handling changing data; handling different linkage scenarios; accommodating ever increasing datasets. Current linkage methods also include analysis of data attributes such as data completeness, consistency, constancy and field discriminating power. This information is used to develop appropriate linkage strategies.

Results: In order to maximise matching quality and efficiency, linkage systems must address real-world operational requirements to manage linked data over time. By maintaining a full history of links, and storing pairwise information, many of the challenges around handling ‘open’ records, and providing automated managed extractions are solved. Automation of linkage processes (including clerical processes) is another way of ensuring consistency of results and scalability of service. Several of these solutions have been implemented as part of developments by the PHRN Centre for Data Linkage in Australia.

Conclusions: Increasing demand for and complexity of, linkage services present challenges to linkage units as they seek to offer accurate and efficient services to government and the research community. Linkage units need to be both flexible and scalable to meet this demand. It is hoped that the solutions presented will help overcome these difficulties.
The video for this talk should appear here if JavaScript is enabled.
If it doesn't, something may have gone wrong with our embedded player.
We'll get it fixed as soon as possible.
University of Cambridge Research Councils UK
    Clay Mathematics Institute London Mathematical Society NM Rothschild and Sons