We’ve mentioned big data and its benefits in numerous blog posts, but we might not benefit from the analysis of the data as BI professionals are spending majority of their time cleaning the data. A study conducted by data integration platform provider Xplenty surveyed 200 BI professionals and found that a third of them spend 50-90% of their time cleaning raw data. Furthermore, nearly £40 of respondents claim data cleaning is their biggest challenge. The study focused on ETL areas such as perceived challenges and preferences for on-premises or cloud-based solutions and identify 97% individuals who noted ETL was essential for their BI efforts.
“BI professionals should be spending the majority of their time evaluating data and deciphering patterns gleaned through the analytics process–not readying data for analytics. The more time they spend making raw data analytics usable, the less time they have to generate real value from it. ”
Yaniv Mor, CEO & Co-Founder of Xplenty.
Some of the challenges BI professionals encountered include data integration with 55% of individuals reporting this is their biggest challenge when making data ‘analytics-ready’. Other challenges include integration relational (32%), and the high volume of data (21%). Yanvis Mor also noted “While many organizations still rely heavily on existing on-premises IT for ETL, the desire to shift to a more cloud-based model has never been stronger” after the report found 51% want to move to cloud data preparation.