Methods and systems for providing support for large data sets are
provided. Example embodiments provide a Big Data Object Library "BDOL,"
which defines data structures and routines for handling big data objects
using out of memory techniques. In one embodiment, the BDOL defines a
bdFrame object which stores the data in binary form in a cache on an
external storage medium, such as a file on a disk. The example BDOL
provides support for user defined block processing a bdFrames using a
pipeline engine. Also, the BDOL provides for Trellis plots, and other
charts, of big data objects using hexagonal binning. This abstract is
provided to comply with rules requiring an abstract, and it is submitted
with the intention that it will not be used to interpret or limit the
scope or meaning of the claims.