Data Science Prep

In an earlier post, I mentioned that, among my other work for Tonal, I'm helping the company data scientist. This help involves preparing data so that he can conduct his analysis. This post addresses the upside when preparation work requires "heavy lifting."

For almost any kind of work, some level of planning, organizing, and preparing helps to ensure success. But the degree of this preparation is variable, ranging from quick and easy, to difficult and time-consuming. For data science, data preparation is often the latter.

Another example of work that demands time-consuming preparation is a personal project of mine: renovating my family's old barn. In the previous post, I mentioned that we live in a clearing surrounded by forest. On our property is a barn that we're told dates back to 1886.

To us, the barn is beautiful. Its bones are of old growth redwood, its siding is chipped and peeling, and it is held together mainly by character. It creaks, it tilts, has survived two major earthquakes, and is host to a variety of critters — including skunks. It is our little bit of mountain history, and worthy of saving.

So, to save our barn, we need to carefully disassemble it, plank by plank, saving the beautiful old wooden boards, and the antique hardware.

Then, we can lay down a concrete platform, and build a strong, true frame for the renovated barn. On this new frame, we would attach the salvaged wood.

Nice idea. We'd have our 1886 barn, and it would be safe and sturdy.

There's only one problem: the barn is full of stuff. Beyond our own stuff, there are left overs from previous owners, and still even more stuff "gifted" us by our extended family.

So before we can even get started renovating our barn, we need to go through this accumulation of stuff, item by item, sorting it into three "buckets:" junk, give away, or keep. Then, for the stuff we're keeping, we need to find other places on our property to store it. Much of that latter stuff is construction tools.

Faced with this daunting task, I realized there was no feasible way that we were going to be able to start the barn renovation before winter. So we have the entire fall and winter seasons to empty the barn, and organize the stuff.

Realizing that I had ample time, I took a fresh look at the problem of storing our tools. I noticed in the hardware area of the Tonal offices, the employees have a wall-mounted steel pegboard on which they hang their tools. See, e.g., Pegboard Tool Organizer. For about $180, I could buy from Amazon what we need for our purposes.

But instead, because I have more time, it occurred to me that I could combine the tool storage prep work with another of my interests: 3D printing. The picture to the right shows our MakerGear M2 printing out tool holders.

The picture to the left shows our tool hanging approach. I found a 4'x8' OSB sheet left over from an earlier project, and nailed it to a wall in our old carport. On that sheet, I attached the 3D printed tool holders, and then mounted our hand tools.

One reason I used this approach is that, as my wife will attest, I'm a cheapskate. But the main reason is because I wanted to become more proficient with our 3D printer. Before this tool hanging project, I had printed only a few things.

One thing about 3D printers is that they are not quite ready for "prime time". Not only are they expensive, but they are quite complex, requiring great attention to configuration. If you're just a little off in the configuration, your prints become a mess.

I saw this tool storage project as a way to bring my knowledge up the curve on 3D printing. I probably spent an extra $20 on 3D printer PLA filament due to some trial and error in my printing efforts. Still, even with these errors, the cost of my approach was much lower than the Amazon $180 price.

But best of all, after this experience, I can say with confidence that I now know much more about what I'm doing with 3D printing.

So tying up this allegory, renovating our antique barn requires significant preparation. The approach I took to one part of that prep work gave me the opportunity to improve my knowledge of 3D printing — knowledge that I'm sure will become useful in the near future.

How does this allegory map to data science preparation? As with renovating the old barn, data science requires prep and planning. But we certainly don't have the fall and winter to do our data science prep work.

Even on an accelerated schedule, there's no getting around the need to prepare. Some of that preparation involves editing a C++ code base that controls a game sensor; writing a python script to start and stop this game sensor automatically; physically setting up the sensors and coordinating this setup with other departments in the company; and writing other python code to parse and upload the sensor data.

With the data parsed and uploaded, our data scientist can now do the science.

Without any help, he would spend his time on these time-consuming, "menial," yet critical preparation steps — that is, when he's not being pulled onto front-line software release fire drills.

So my role and goal as "data science helper" is to do the prep work so that our data scientist can do the critical work, which is why he joined the company. It's a win win for everyone.

Additional upside for me? For the barn renovation prep work, I learned to master our 3D printer. For some of this data science prep work, I'm learning Java and how use it to build a mobile app.

I love learning.