For the past couple of years, it seems that “Big Data” has been a buzzword in the industry. We have more and more data coming in from more and more places, and it is our job to figure out how best to handle it. One way to attempt to organize data is with arrays – but what do you do when the array you are attempting to populate is so large that it cannot be handled in memory. Further, how do you handle a large array when most of the elements are missing?
This presentation deals with the concept of a Sparse Matrix – that is, a large array with relatively few actual elements. We will address methods such a construct be handled while keeping memory, CPU, clock, and programmer time to their respective minimums.