Category Archives: Big Data

Confidence Intervals in Python

Suppose you are interested in finding out the mean weight of all Sumo wrestlers in Japan. Or the average gas consuption of Korean made automobiles… Why…? No idea, but that sort of statistics might be of interest, for someone, at … Continue reading

Posted in Big Data, development, Math, Numpy, Probability, Python, Simulation, Statistics | Tagged , , , , , , , , , | Leave a comment

Single simulation run consuming 230 h (!) of cpu-time – root cause analysis of performance

I think I’ve broken some kind of record of CPU-time consumption for a single simulation run… for a fairly trivial application… As my frequent readers know, I’ve been playing lately with a simulation of Shelling’s segration model (see previous posts … Continue reading

Posted in Big Data, development, software | Tagged , , , , | Leave a comment

Neural networks and image recognition – hit rate for target detection

In my previous post, I described some experiments with neural networks and image recognition that I entertained myself with during xmas holidays.  Basically, I wanted to see if I could create a neural network process that, given tiny (10×10, that … Continue reading

Posted in Big Data, Complex Systems, development, Math, Non Linear Dynamic Systems | Tagged , , , , , , , , , | 2 Comments

Twitter buzz about IBM Rational

Having created a Python program for “poor man’s mass surveillance”, that given a Twitter account, or set of accounts, grabs the latest tweets from that account and analyzes each tweet for mentions, I thought it might be interesting to see … Continue reading

Posted in Big Data, development, SNA | Tagged , , , | Leave a comment

Poor man’s mass surveillance:collection & analysis of Twitter data with Gephi & Python

[Each of the “dots” in the graphs above –  there are about 100.000 of them –  even the tiniest small ones barely visible at this level of zoom, represents a twitter user (blue dot), or a hashtag (red dot).  The … Continue reading

Posted in Big Data, development, SNA, Surveillance | Tagged , , , , , | Leave a comment

Artistic Twitter social network graphs by Gephi

The above image contains almost 100.000 nodes, extracted from my Twitter relationships (transitively, of course, I’m not the Lady Gaga of  social networking, after all…! 😉 The above image was obtained by allowing Gephi to execute its Force Atlas 3D … Continue reading

Posted in Big Data, development, software | Tagged , , , , , | 2 Comments

Artistic Twitter Network Displays

These #Gephi layouts of my Twitter network start to look like pieces of art… don’t know about you, but I find these displays beautiful…! There’s almost 100.000 nodes in that graph, an amount that would be totally impossible to make … Continue reading

Posted in Big Data, development | Tagged , | Leave a comment

Who has most influence within your network…?

Whether you are a leader, manager, coach, politician or dreaded marketeer, you would want to know who or which persons are the most influencial within your network, regardless of that network being a social or professional one. I just analyzed … Continue reading

Posted in Big Data | Tagged , | Leave a comment

Analysis of Twitter: relationships between tweeters, mentions and hashtags

Now things start to get really exiting: I extended my Python hack to collect information on all relationships for a set of Twitter users, more specifically, I record the following: Each tweeter becomes a node Each #hashtag becomes a node … Continue reading

Posted in Big Data, SNA, Surveillance | Tagged , , , , | Leave a comment

twitter relationships

WIth a bit of (Python) programming it is possible to create a graph with Gephi, that shows the relationships between people in your Twitter network, e.g. “who are those that you are following, following…?” More specifically: I started with the … Continue reading

Posted in Big Data, development | Tagged , , | Leave a comment