A system for improving sensor-based decision making provides for the automatic submission of data obtained locally from instrumentation (such as image data) together with the interpretation of that data, which can be the output of some software which has been checked and possibly corrected by a user according to his/her expertise, to a remote database via an internetwork. The submission to the remote database is preferably automatic so that the remote database grows over time. The local site can access the remote database to retrieve information to assist in interpretation of the locally produced data (for example similar images and their corresponding interpretations), or can retrieve updated or improved software or parameters improving the software used for processing the data. The information on the remote database can also be reprocessed by software agents to provide statistical information based on information from a variety of such local sites. The system is particularly useful in improving the interpretation of data which is difficult to interpret such as medical image data (e.g. mammographic or cardiac ultrasound data).