Use Case Decomposition for Big Data

**** Looking BACK to 2013 ****

As I begin to describe the value of a successful ASSESS, ALIGN, ASSERT and ACHIEVE big data methodology, I am reminded about a fundamental prerequisite: Use Cases.






From Wikipedia: 
"a list of steps, typically defining interactions between a role or actor and a system, to achieve a goal."

I am convinced that Big Data benefits from Use Case Decomposition.  It's a predecessor - and now an integral part of -  Agile Software Construction.  I am continually surprised at how well Use Cases assist in the analysis of uncertain data.

The Decomposition part is also worth mentioning. Moving from top to bottom, decomposition let's you break complex tasks into smaller, more understandable elements.  When reviewing results, you get the chance to assemble all the composite pieces of information to ensure fundamental requirements have been met.  It is also a really nice way for every member of a development organization to say:  'I understand what you are requesting and here's what I plan to do about it'.

Business Analytics for Big Data in 4 Words

**** Looking BACK to 2013 ****

Readers of this blog probably know the 4 V's of Big Data:

Volume, Velocity, Variety and Veracity (trust)


I would like to offer the 4 A's for Analytics:

Assess - Figure out what you need and where the data resides. Start with visualizations to gain understanding of large datasets.  Look for opportunities to connect data.  Having lots of data and attributes at this stage is a good thing.

Align - Align data with existing dimensions, metrics and measures to start building better sources of trusted data. Early association of key attributes improves the accuracy of text and entity analytics.

Assert - Here comes the statistics part: Create analytic data from sources and attributes identified in the previous steps. Find new connections with exploration and discovery. Achieve quantitative insight as you create new columns of data from old.  Revisit previous assumptions to ensure you have consistent data. The more the data aligns with your assertions, the more trusted you new data becomes.  But don't worry: inconsistent data is also good.  It helps you find outliers that suggest your assertions might need to be revisited.

Achieve - Take action using the new data you created (obviously). But you also need to share analytic discoveries including data and procedures. Reuse these assets within your enterprise to build more advanced analytics. Continue revisiting and applying analytics to new data to build greater accuracy and trust.

My goal in the next few weeks is to continue differentiating Analytics from more traditional Business Intelligence while showing how solutions that blend both disciplines can offer some of the best insight available.

What is an Analytic?

**** Looking BACK to 2013 ****

I work in the rapidly growing field of Business Analytics (BA) . It is the natural evolution of Business Intelligence (BI) that we started two decades ago. Follow the links above and you see that BA is roughly described as:

"the skills, technologies, applications and practices for continuous iterative exploration and investigation of past business performance to gain insight and drive business planning"

and

"Business analytics makes extensive use of data, statistical and quantitative analysis, explanatory and predictive modeling, and fact-based management to drive decision making."

That's a great start, but I still have trouble understanding the value of BA in the context of the reporting, spreadsheets and databases that I have used for many years. In other words, what do I get with Business Analytics that I didn't get from Business Intelligence? And how does Big Data change things?

Here is a series of slides have been working on. I hope they help.

We use BI built on top of an IT infrastructure to prepare, cleanse and conform raw data into curated SQL (rows & columns) and OLAP (cubes and slices) tables. These warehouses and datamarts provide aligned data that is generally accurate, understandable and relevant within the confines of the applications they were meant to serve. 

Here's a fundamental element to remember: BI organizes, summarizes and ultimately reduces the original volume of data. 

Business Analytics adds the functions and procedures to augment BI data to reflect a wider domain of understanding. It literally expands data volumes to reflect greater amounts inferred knowledge.

We explore data to find new associations. We extend data using various quantitative, qualitative and predictive functions to create new data from existing BI data. Together we get the insight that hopefully lets us make better decisions.

At least that's how things worked before the arrival of Big Data.

The players are similar but the overall process often changes in a few important ways.

Data volume is the same or higher. Aligned and curated IT data is replaced by the different sources that are Big Data. We still prepare, cleanse and prepare data although actual sources are less likely to be moved to physical warehouse files. Data stays in place while virtual warehouses and datamarts are created on demand and live only as long as necessary. 

Big Data introduces a new wrinkle: Previously, IT built most of the data used by BA. Now much of the data is referenced directly by Line-of-Business users. This adds great flexibility and speed. But the data can be suspect. More on this in a moment.

Analytic functions extend virtual data to reflect better understanding. The parameters to these functions are more diverse since data is less structured than before.

I define Business Analytics for Big Data as an extension to traditional analytics that offer an important new feature: Virtual data is related to existing BI data to improve alignment, accuracy, understanding and relevance. This step cannot be overemphasized. Without relating new data to old, it becomes much harder to trust diverse but uncurrated big data sources.