In 2015 we saw strong incremental progress on several technical fronts that should make 2016 an interesting year for those of us in the data trenches. The volume of data is ballooning across the board. Unstructured data is becoming more structured (making it easier to use), so we’re witnessing an intriguing convergence point where there is vastly more data to process and substantially more powerful tools to do the necessary work.
Here are the trends that I see continuing in 2016:
- As more data is “born digital” and designed to be user-friendly at birth, it will demand less conversion and other data wrangling. The generally higher volume of data, however, will mean that we’ll actually be doing even more data processing.
- Crowdsourcing techniques will increasingly incorporate AI to optimize results. We’ve been combining OCR, harvesting, and other processes before and after crowdsourced routines to speed up the work, lower costs, and improve quality.
- The proliferation of open data will go on at a brisk pace. This poses a challenge to some value-added resellers of government information, but also opens up important opportunities for the creative reuse of many new datasets by emerging information service firms.
- The convergence of CRM and competitive intelligence data, epitomized by companies like InsideView and Owler, will speed up in 2016. This will unlock more of the enormous potential behind predictive analytics.
- The amount of metadata (web traffic logs, purchasing behavior, etc.) will continue to explode and firms will find different ways to tap the incredible value hidden in these massive datasets, allowing improved corporate decision-making.
Here’s to a better, cheaper, faster 2016!