Custom Data Mining

Today, I'm introducing our new consulting service: "Custom Data Mining".

What? You have a mission critical question about your business, as well as some data that might bear on the question. You'd like to see how the data could answer your question.

Who? You're a manager at a company or other institution, and you have P&L responsibility. The data you have — for example, invoices to customers or from outsourced providers — bears on the "P" and/or the "L".

Why? You want an answer from the data because the data isn't biased by its role within your organization, or incomplete, like human answers might be. The data could provide a more rigorous answer.

How? The image shows the steps. Starting with your question and data, we clean the data to generate a master data file. Then we extract "variables" from the data, where each variable is a column of numbers, each number representing an observation or recording. We then perform statistical analysis of the variables with the goal of generating a model that serves as the source of the (statistically) best answer to your question. Finally we translate that model (a math equation) into English paragraphs "written by your data" that help you with forward planning.

This work requires three forms of expertise: business, statistics, and technology. In our patent analysis line of work, the three required skillsets are: business, IP law, and technology. So for data mining, we swap out the patent lawyer for the statistician. Fair trade? :)

As for the name of our new service, "data mining" speaks for itself (see, e.g., the image in this post). The term "custom data" is meant to distinguish it from "big data". Big data is typically uniform, or capable of being unified and normalized, otherwise, tech companies couldn't scale as Big Data service providers.

This service we're offering is meant for those rather unique data sets that collect within your organization. The data is unique enough, that even if one were to string together all the companies doing your kind of work, and gathering your kind of "custom data", there's not enough of you to scale a web-based analytics business.

So, with a turnkey web solution to your data question unavailable, you'd be stuck with one of two choices: (1) bugging your busy engineers to work on your problem as a side project; or (2) retaining a big, costly outside consulting shop that has coders, statisticians, and project managers on their payroll.

However, there is now a third option. Us. We provide a quick, agile, and cost-effective Custom Data Mining service.