Rango's Dog House

An inspired read from Koverse CEO Paul Brown, posted this week to the Cloudera website after it was announced that Koverse and Cloudera are partnering around Apache Accumulo development:

Paul Brown: You Won’t Need a Cannon

For all of the growth within the big data world, we have only started to realize its full potential.

I have spent the last decade focusing on one such opportunity – the idea that big data and actionable insights are not limited to expert organizations and departments but can be provided to all parts of a business in a self-service manner. Organizations need to be able to develop their individual sources and methods. Giving organizations the ability to do this themselves is a game changer. The environment today is right for this disruption. Technology has matured to a sufficient level and the need to provide actionable insights on big data is bigger than ever.

Data, both public and organization-specific, are available. The potential value of this data – high. They are varied, complex, constantly changing, and ever increasing in size. The challenge of aligning changing data to evolving business needs is like trying to hit multiple moving targets with a cannon. You may hit a target or two, but the cannon is too unwieldy to continually re-aim. Alignment requires agility.  Data systems need to be agile to keep up with changing needs, which in turn helps organizations be more agile.

Creating systems with this agility is costly and technically risky. It takes money to acquire the various technologies, specialized skills to build solutions and the right vision to ensure success. It also takes time to build and deploy such solutions, and too often once the software is deployed it falls dramatically short of expectations.

When we founded Koverse we asked ourselves what we needed to do to help organizations deploy a solution like this more quickly and for less money and risk, without specialized expertise. For us the solution is driven by five core principles that a system must embrace to make it perform in this manner:

1. Ability to store and process large amounts of data inexpensively, quickly and securely. This has less to do with very large raw data sets and more because organizations need the flexibility to ingest new datasets and materialize the various data alignments without being constrained by system capacity.
2. Ability to change a data processing approach and reprocess data and insights from raw sources in less than a day. This allows known insights to adapt to changes in source data.
3. Ability to load new, previously unknown data sets in a matter of hours. This is more than simply storing files. This means loading in data and discovering the intrinsic structure automatically so it can be leveraged immediately.
4. Ability to interactively query all data within the system regardless of schema or structure. This is critical to both data scientists exploring data, as well as, to the large number of users that need access to final analytical results.
5. Ability to support thousands of inter-dependent datasets within a single system such that the right insights can be made accessible to the right people at the right time. Collecting and hosting the data once and making it available across the organization lowers time to and cost per insight because the cost of the system can be spread across multiple organizations. A key principal to achieve this ability is role based access and fine grained labeling of data.

For Koverse, our foundation for providing this capability to our customers is Apache Hadoop and Apache Accumulo. As one of the first advocates of Accumulo, I was both proud and excited when Cloudera announced its support. Excited because Cloudera’s support of Accumulo provides the foundation for delivering on our core principles.

Cloudera’s Distribution including Apache Hadoop, CDH,  delivers proven performance when it comes to scale and processing speed, which are critical to meeting cost-effectiveness and time-to-insight requirements.

To that, Accumulo adds the ability to do sub-second queries across data sets of multiple schema types and sensitivity levels, making it possible to deliver results to a large number of users within the same system. In addition, proper use of Accumulo means security concerns no longer force data sets to be stored on separate systems, and data sets can be combined to produce value while respecting legal and policy requirements.

We are excited to form a partnership with Cloudera in support of Accumulo and to help accelerate their contributions to the Accumulo project. By taking advantage of the combined capabilities of Cloudera’s products and Accumulo we have been able to create a platform for separating signal from noise and deliver actionable insights “out of the box,” significantly reducing the time and expense to organizations.

I can’t wait to show you what we can accomplish together.

Paul Brown, CEO, Koverse. Follow him on Twitter: @paulbrown_pnw

Originally posted to the Cloudera website here:

http://vision.cloudera.com/you-wont-need-a-cannon/

Read more about the Koverse / Cloudera parternship arround Accumulo here:

http://www.koverse.com/cloudera

Koverse & Cloudera