NDF2019

50 years of New Zealand music from over 5,000 artists from a major New Zealand publisher on one hard drive!  As the largest digital deposit received under Legal Deposit legislation this was both tremendously exciting and overwhelmingly large. This presentation will describe the processes and tools NLNZ used to convert a spreadsheet and 3.5Tb of 250,000 files into ~6,000 bibliographic records and digital items.  We’ll discuss the broad approach, highlighting where things did / didn’t go to plan, and concentrate on a few tools and techniques we used to turn this large data store into a well described collection. Richard will get into the record keeping side of the problem – how did we find and link duplicate bibliographic records? How did we deal with name authorities? How did we automate the creation of thousands of new records? Jay will get into the technical side of the problem – how did we link items on a spreadsheet to a files on the hard-drive? How did we pull data about a file when it wasn’t included in a listing? How did we break the collection into manageable chunks? How did we use other people’s data sources to help verify our approach for making sense of the multi-terabyte file store?