Transforming data prior to loading the data into a data warehouse. Software of the invention partitions data records received from a plurality of servers and performs sequential file management operations and identifier management operations on each of the partitions prior to loading the data records into the data warehouse. Data records transformed according to the invention are easier to load into the data warehouse and easier to manipulate after loading. The invention enables analysis and data mining of tens of terabytes of user level detail data and summary data.