The future of big data lies in real-time decision making combined with machine learning and natural language processing.

To gather insights on the state of big data today, we spoke with 22 executives from 20 companies who are working in big data themselves or providing big data solutions to clients.

Here’s what they told us when we asked, “Where do you think the biggest opportunities are in the continued evolution of big data?”

• Tie in real-time operations to telecom providers in Japan automatically directing antennae for cell phones based on need. React in real time based on where the users are located.
• Managing things centrally makes sense. All companies are scrambling to become part of Amazon. We want to make our product available on AWS. Redundant, virtualized systems in the public cloud. This is a Capex versus Op Ex versus core competency decision. Offload your IT architecture to AWS and focus on your business. Trust someone else to maintain the environment and use as a service.
• Moore’s law in computing power continues to enable the handling of larger datasets, containers, and microservices. Integration challenges are being solved. IT needs to focus on the right problem, set-up, configure, maintain, and manage disparate data sources to innovate like Tesla and Uber.
• Continue to help companies take different formats – normalize, tag, and put in a repository to use and analyze. Analysis of data by computers by enriching the data by enabling lookups.
• The immediate opportunity is to find effective, efficient ways to correlate use cases, and leverage common big data stores to address multiple needs. The concepts behind big data are fabulous for collecting a wide range of data, and now the challenge is to break away from legacy siloed thinking to recognize and leverage data relationship to solve the remaining hard problems that are out there today, such as complex multi-domain proactive incident prevention and multi-faceted cybersecurity threats.
• Insights everywhere for everyone – not just the elite. Take big data, business intelligence, and analytics to 100% of the population. Everyone can use analytics just like everyone can read and write.
• Democratization of data. Able to get useful information from data via the clould and solutions as a service (SaaS).
• Catalog data in NoSQL and Cloud, in addition to Hadoop so analysts can get access to all big data.
• We will be collecting more data, getting actionable insights, with repeatable (automated) processes, to see near-term value from data. In the next generation, we’ll have real-time streaming and decision making. More of a real-time view.
• Data is king. Evolution of tools with machine learning and NLP. Make it easy for people to use our infrastructure and focus on data mining. Cloud, Flash, soft edifying makes infrastructure invisible with APIs.
• For risk-averse markets, how to incorporate detailed data to do more experimentation and test more hypotheses. Enable experiments to go into rapid prototyping. Don’t trigger false positives. Don’t open security issues. Take advantage of big data technology. Plan for voice of the customer, sensor data, biometric signals with changes. Don’t hard code the data feed, stay flexible. Think ahead about how our kids are the next generation and how the user interface will change.
• Anyone involved realizes we’re in a boom time. Remain adaptable and flexible. Don’t be dogmatic about a particular solution. Evolution of production analytics. Predictability, scalability and how to get value out of network technologies. Embedding of knowledge to make real-time decisions and have real-time knowledge.
• Convergence of software, data, and machine learning. AR/VR via mobile. Blockchain is getting less buzz but being able to share data to get an outside perspective without moving the data is a tremendous opportunity. Attributes and scores are based on data that’s secure and encrypted.
• More real-time convergence of disparate data sets for real-time analytics. Machine learning. Being able to ask questions of big data and get answers back. IoT will provide real-time traffic, threats, and traffic. Make computer systems smarter. Build enterprise data hub architecture. Average users employ Hadoop to do one thing realizing the multi-tenant state of the data. Open source will continue to drive big data. Need to move to the cloud but maintain flexibility with hybrid solutions.
• The distant future is the application of machine learning, AI, and automation. There’s still a way to go for this not to be science fiction. We get there by having tools make the analysis and collection of data more similar. Machine learning is currently siloed away from the analysis tools. This will need to change for machine learning to be integrated into real-time decision making.
• Life sciences – struggle with large data sets. Able to predict microbiotics’ impact on humans. Learning systems across industries are building cognitive systems where all training and machine learning is based on user behavior in real-time.
• Resurgence of small businesses integrating with online businesses. It’s easy to open a store online, look at the analytics and then replicate in brick and mortar. Bonobos is doing this as are craft businesses in coffee, chocolates, and bedding. Retail will be able to use the same technology developed for casinos (e.g., Space Meter) to track walk byes and drive byes. There will be more analytics involved in monitoring the performance and potential of retail stores even down to the use of shelf space.
• Lower barrier to entry for majority of organizations and users, so that Big Data analytics can truly be integrated into day-to-day operations.
• Automation of the generic use cases will change the perception of big data projects and how they are carried out in the next couple of years. Additionally, moving away from custom implementation to more generic implementations, which in turn can be tuned to the customers’ needs.
• Better education at the grassroots level. Computer engineering degrees need to teach the concepts and provide future computer engineers with experience in big data processing. It takes a significant mind shift to move away from traditional data processing concepts, such as relational database, and to move toward big data processing concepts. Cloud providers need to keep refining their tools to make them easier and easier to use. To be fair, they are doing this. Machine learning and advanced analysis as a follow-on phase to Big Data processing is already a revolution. I believe that the longer-term impact of this technology will be politically and economically profound and is currently grossly under-estimated by most people.

Where do you see the biggest opportunities for big data?

By the way, here’s who we talked to!

• Nitin Tyagi, Vice President Enterprise Solutions, Cambridge Technology Enterprises.
• Ryan Lippert, Senior Marketing Manager and Sean Anderson, Senior Product Marketing Manager, Cloudera.
• Sanjay Jagad, Senior Manager, Product Marketing, Coho Data.
• Amy Williams, COO, Data Conversion Laboratory (DCL).
• Andrew Brust, Senior Director Market Strategy and Intelligence, Datameer.
• Eric Haller, Executive Vice President, Experian DataLabs.
• Julie Lockner, Global Product Marketing, Data Platforms, Intersystems.
• Jim Frey, V.P. Strategic Alliances, Kentik.
• Eric Mizell, Vice President Global Engineering, Kinetica.
• Rob Consoli, Chief Revenue Officer, Liaison.
• Dale Kim, Senior Director of Industrial Solutions, MapR.
• Chris Cheney, CTO, MPP Global.
• Amit Satoor, Senior Director, Product and Solution Marketing, SAP.
• Guy Levy-Yurista, Head of Product, Sisense.
• Jon Bock, Vice President of Product and Marketing, Snowflake Computing.
• Bob Brodie, CTO, SUMOHeavy.
• Kim Hanmark, Director of Professional Services EMEA, TARGIT.
• Dennis Duckworth, Director of Product Marketing, VoltDB.
• Alex Gorelik, Founder and CEO and Todd Goldman, CMO, Waterline Data.
• Oliver Robinson, Director and Co-Founder, World Programming.

Link:

https://dzone.com/articles/big-data-opportunties