Streaming Algorithms/Data
Several qualities come to mind when I think about the uses of streaming algorithms or data. Not to sound like too much of a grump, but one thing that really grinds my gears these days is planned obsolescence. I have several examples that just plain sicken me, but I will give only a couple just to get my point across. I am sure no doubt you have probably had the same thoughts in this modern world we live in.
The first is the Iphone. I have been a lifelong customer of Apple. I have owned both a first generation Ipod and Iphone. Most people don’t even know what an Ipod is. That is how long I have been supporting Apple. Literally since the first Iphone came out, they have almost not changed at all. They were briefly “water proof” for one version. Now they have 15 cameras on them, which doesn’t interest me in the slightest. I take ridiculously good care of my stuff by the way. I have never broken or lost an Iphone, yet I am forced every 3–4 years to buy a new one. Is it because they have made them insanely better or added new awesome features? Nope, this is all part of their plan…
The second example is Honda. My wife and I recently bought our first new car as a young family. Our other cars were 20 year old Hondas which we’ve had since day one, and they were handed down through the family. Long story short, when we were buying our new car, numerous people at the Honda dealership told us what amazing cars our ancient Honda’s are. I agreed with that, but then they go on to tell us to not expect the same lifespan out of our new car. To which I replied, “They better last that long or Honda will lose a lifelong customer!” Now in all fairness to Apple and Honda, two companies I have loved since I was a kid, some of this is just the world we live in. Computers don’t last forever, but some of this is also them using planned obsolescence. And all I have to say is this “Shame on you”!
Ok enough complaining, let’s focus on Netflix! Another life long love of mine… Streaming in terms of data is an amazing concept for data and algorithms. And it isn’t a new one. It has been around for about fifty years. The first use that comes to mind is what I was talking about with the Iphones above, memory. As formats of data become bigger and bigger, from DVDs to Blu Ray and now 4k movies, they take up more and more space. This is a part of the reason why we have to keep upgrading our computers and phones. We are designing streaming algorithms and data these days for this very reason, so they don’t take up too much space. We will need more and more space the more detailed and complicated our calculations become!
“Well, sure, the Frinkiac-7 looks impressive, don’t touch it, but I predict that within 100 years, computers will be twice as powerful, 10,000 times larger, and so expensive that only the five richest kings of Europe will own them.”
The second important use I can think of for these enormous computers comes from my previous career and my idea for my capstone project. I used to drill oil/gas wells as a Geologist. We are constantly collecting and evaluating data as we “drill ahead”. A streaming data collection process would be perfect for such endeavors! If only we could train a computer to build a model! Wait a minute. There is already a way to stream this data to the computers on the internets. Now we just need to make a model for them to use to evaluate the test data they are receiving. In theory, maybe computers couldn’t completely take over the world, err I mean process, but they sure could help us drill more efficient wells. Since computers can interpret data so quickly, they could evaluate the whole project with every new window (designated time interval), which would make the correlations even stronger. Normally a human Geologist will only look at the last couple hundred feet of data to make a decision about their current position. Every once in a while we will have to go back and recorrelate the entire cross section. This can lead to things you missed before and completely change your mind about where you are in the stratigraphic section. If a computer could double check us every so often, it could help reduce errors tremendously. Fewer mistakes not only save money, but they keep everyone safe, from the people who drill these wells to the people that live near them.