Generating masks for de-duplication in a database where distributed entities provide activity data for said database. Determining from activity input data which entities add variable data to a given data field. Generating a list of the masks which effectively remove the variable data portion in the field. Consolidating input data using the generated masks.