Lyjotoblog

Sometimes we hear remarks like; “In the past 50 years we extended our knowledge and insights more than in the past 2000 years”. Although this is hard to proof, you could also claim that people didn’t grew more intelligent, compared to the past 2000 years. Of course, we gained massive amounts of knowledge and wisdom, so much so, it’s too much for one person to comprehend and understand. We are very good at collecting massive amounts of data, which collectively increase our knowledge. Better then, compared to the past 2000 years. The relative gradual progression of (scientific) history and the saltatory changes over the past decade are witness to this. The medical sciences alone demonstrate this.

Are we collecting to much data? Are we capable of handling this amount of data? No, WE CAN NOT. We need computers to do this, just as computers made us create this huge amount of data. IBM announced that 90% of data we own have been generated in the past two years. These “Big Data” are generated everywhere; from social media, sensor information, digital photos and movies, money transactions, mobile phone traffic, e-mail traffic and so on. The Art, or better, the Necessity is to create structure or meaning from these immense streams off data. The problem is, this data is not always read or interpreted correctly. So there is a growing need of making sense but also of slinking down these massive data streams. Are we maybe missing some data which a computer might not miss? It is well possible, but a computer can’t do this spontaneous? It needs the right software programmed. And if placed on the right analysis stream it might be capable of detecting (obsolescence or redundancy, too big data complexities) something we can learn a great deal from. Maybe we learn how to streamline or simplify digital processes.

A special form of sniffing in “Big Data” is called “Data Mining”, more about this subject later on.