Estimation of tides

Tide estimation made simple by using “rule of 12th’s”.

Advertisements
Posted in development | Leave a comment

Central Limit Theorem illustrated

IMG_1516

A fundamental theorem of statistics is the Central Limit Theorem, which basically states that the averages (means) of a large number of samples drawn from any population converges towards a normal distribution.

The graph above demonstrates this theorem: the blue bars show a “population” of 1.000.000 instances, with a range of values of [0..9]. As can be seen by the blue histogram, these 1.000.000 values are very uniformly distributed.

From these 1.000.000 values, I randomly draw 10 values, 1000 times, and compute the averages of the samples, and plot the resulting histogram in green. Voilá, the famous Bell Curve appears!

The red line is the corresponding pdf for a normal disteibution with the corresponding mean and stddev.

Posted in development, Math, Numpy, Python, Statistics | Tagged , , , | 2 Comments

Wind,Temperature,Air Pressure statistics

lfreqlqqTfreqTqqvfreqvqq

The above graphs show statistics from SMHI, the swedish meteorological and hydrographical institute, on air pressure, temperature and wind speed.  For each categrory, (pressure,temp,wind speed) there are two graphs: the first a frequency chart, the second a Q-Q-chart, showing the correlation between the observed data and a specific theoretical distribution, which I’ve chosen to be the Normal distribution for temperature and pressure, and the Rayleigh distribution for wind speed. Note: that’s not to necessarily imply that wind, temp & pressure ARE distributed according to these theoretical models… more on this below.

As can be seen, the best match occurs for wind speed distribution, where the observed data fit the Rayleigh distribution very well – almost all observations fall onto a straight line.

Both air pressure and temperature are mapped against the Normal distribution, and as can be seen, neither of them are a perfect fit, with temperature being clearly most non-conforming to the normal distribution.

To summarize the observations from this data:

Wind speed follows the Rayleigh distribution quite well, Air pressure could perhaps be modelled by the normal distribution, despite it not being a “perfect” match – but then, in reality, no model is a perfect match for reality! – and temperature, at least at my latitudes, where there is large seasonal variation, does not fit neither Rayleigh nor Normal distributions.

Posted in development, Non Linear Dynamic Systems, Probability, Python, Statistics | Tagged , , , , , , | Leave a comment

Wind Speed Distribution – is it Normal…?

wind_hogarnasoderarm-distsoderarmhog-dist

As a sailor, I’ve become obsessed by wind. Trying to understand its behavior has kept me busy for decades, and still, I can’t claim I fully understand wind behavior.

Anyways, one of the things I’ve wondered is whether wind is governed by the normal (Gauss) distribution. It’s not. As can be seen from the graphs above, with data graciously provided by the Swedish Meteorological Service, SMHI.

Graphs 1 & 3 show a time series of wind, in terms of direction and speed, from two different stations on the swedish east cost, the lighthouses ‘Svenska Högarna’  (abt 600 data points) and ‘Söderarm’ (abt 300.000 data points). For Svenska Högarna, the time series is short, just June 2017 to October 2017, while the time series for Söderarm is from 1951 to 2017. As an aside: if you look carefully at the data for Söderarm, you can in the smoothed plots see that something seems to happen to the data in 1995… I got in touch with the good folks at SMHI, and they confirmed that in november 1995 the measurements went from manual, 3 times a day, to fully automatic, once per hour, which explains the larger fluctuations of the data after 1995.  Also, these two graphs display the data in smoothed form, where I’ve used a Hanning Window and convolution to detect the overall trend for TWS and TWD.

Graphs 2 & 4 show the relative distribution of wind speed (TWS), measured in m/s. It’s from these histograms obvious, that wind speed does not follow normal distribution – instead, wind is characterised by a Long Tail, typical for phenomena governed by different types of power laws.  Apparently the wind pro’s uses Weibull distribution to model wind speed behavior, at least according to this reference.

On top of the wind speed distribution of graphs 2 & 4, I’ve overlayed a normal probability density function, in orange, as well as a random sampling with same parameters as the wind data, by a red, dashed line. Furthermore, on the graphs the mean is illustrated by the red dashed vertical line, and the 1st, 2d & 3d standard deviations by dashed orange lines.

The raw data from SMHI came as .csv’s, and I used Python Pandas to read and manipulate the data. Pandas are great for processing large amounts of more or less structured data, it takes only about 60 lines of code to transform the raw data from the csv’s to the Matplotlib graphs.

Posted in Data Analytics, development, Nautical Information Systems, Probability, Python, software, Statistics | Tagged , , , , , , , | 3 Comments

Map Projections using Python & Basemap

The numerous external libraries available for Python continue impressing me, in scope, numbers and ease of use.  A couple of hours ago I googled for chartographic tools for Python, and found the Basemap-library, an add-on to Matplotlib. For whatever reason, it’s not available for pip install, so I had to install it from sources, which was totally painless. Then, after finding this basemap tutorial, it took just about 15 lines of code and equally long time in minutes to draw the different map projections above, all showing a flight path Stockholm – San Francisco.

Continue reading

Posted in development, Maritime Technology, Nautical Information Systems | Tagged , , , , , , , | Leave a comment

Great Circle Navigation

SYD-LHR

IMG_1500

The shortest distance between two points is a straight line, right…?

Nope, not on the surface of a sphere, like our earth. On a sphere, the shortest distance between two points is a Great Circle.  Let’s say you are a pilot and about to embark on a journey from Sydney to London. You take your Mercator chart, and plot a straight line from Sydney to London (like the red line on the map above).  Turns out that your course, following this straight line – also called “Rhumb Line” or “Loxodrome”, will result in your flight distance being about 360 Nautical Miles longer than had you followed the orange path, which is the Great Circle route. That’s about 45 min extra flight time in a typical commercial jet, which is a lot of extra fuel,  or a day and a half of extra sailing if you’d sail the same route.

Loxodrome’s (or Rhumb Lines) have the nice characteristic on Mercator charts that they are presented as a straight line, and with a constant course, in the example Sydney-London the rhumb line course is 303 degrees.

For Great Circle navigation, your course is going to vary during the journey, for instance, on the plotted journey from Sydney to London, the initial course is 318, gradually increasing to 326, and then gradually decreasing to reach the final course of 242 degrees. For practical reasons, it’s not possible to change course constantly, so for instance onboard ships, navigators typically used to change course once a day to follow an approximation of a Great Circle route.  Today, with modern technology, it should be feasible to tell the autopilot to follow a Great Circle path with much more frequent course changes. (In case any commercial pilots or professional mariners read this blog, I’d be interested in learning how you guys do onboard your ships/aircraft in order to follow a great circle).

The lower graph shows the True Vector at selected points.

(I haven’t found a way to include code in a decent way, preserving indentation etc,  in a WordPress post – without having to pay for it – so the Python hack code is in the pdf doc below).

Cudos to pythonforoceanographersgpxpy, and movabletype for inspiration, insights and new knowledge.

great_circle

 

Posted in development, Nautical Information Systems, Python | Tagged , , , , , , , | Leave a comment

Machine Learning – results

So, I reached the point where my Neural Network , after a couple of thousand training iterations, is capable of correctly identifying and separating different categories of input into their corresponding classes.

More specifically, as “targets”, I’ve chosen a number of continuous sampled waveforms, each one 1000 samples long, wheras as “noise”, I’ve simpy used equally many samples of random noise.  One could imagine that the targets represent sonar clips of submarines, sea life or something else, and the noise samples just represent the background noise in the marine environment.

Anyways, a visual illustration of the 100 target samples is below:

sinus-100.png

Next image illustrates the noise, equally 100 samples, each with size of 1000:

noise-100.png

I put these data as the training data to my neural network, in this case a 1000 x 100 x 10 x 1 network, and after a couple of hundred learning iterations the results are as below:

learning_run

So, the network is capable of providing a perfect separation between the two classes of training data, as can be seen from the upper graph. The lower graph shows how the network fairly quickly converges towards a solution: the y-axis shows the cumulative error over all the training data per iteration. As can be seen, the network is able to adjust its 101.010 (1000 x 100 + 100 x 10 + 10 x 1) weights towards a convergence.

To test the network after the training, I pass in one single target waveform, randomly chosen, and similarly, one single noise input.  The below graph shows that the network is capable of correctly classifying these two inputs.

sample_test

I’ve done this experiment using my very modest laptop – despite my very limited number crunching and data processing power, I’m able to obtain good results. Imagine then what the Big Guys, the Google’s, Facebook’s, Amazon’s and others are able to do with this technology, with their immensely more powerful machinery….

Artificial intelligence is already changing socities and human life, and it will continue doing so in an ever increasing pace. Humans will need to think carefully about how to deal with this “Pandora’s Box” – a good intro to the potential dangers of Super-AI can be found here:

Posted in AI, development, Machine Learning, Neural networks, Society, Technology | Tagged , , , , , , , | Leave a comment

Machine Learning – normalizing your inputs

Often it pays off  to pre-process/normalize your input data before passing it further down the line of your Neural Network, eg. to get the range of input values into a scale more suitable for your network.

Im experimenting with images, where each pixel is represented by 3 values (RGB), each in range 0..255. The input to my network consists of such 3-dimensional pixels, where each pixel is represented by the sum of the 3 dimensions. That means each pixel has a range of 0-765.

That’s a pretty large range to tune the networks weights to deal with. So, I needed to scale down the range.

The below image illustrates three different ways to do such a scaling:

  • Subtracting the mean of the input data
  • Dividing the input data by the input variance
  • Doing both of the above

Ideally, I’d want my inputs to be centered around zero, since the Sigmoid function, responsible of calculating the nodes of the network, has an active range centered around zero.

By normalizing my inputs by both subtracting the mean and dividing by the variance, my data has both desired properties: smaller absolute range and centered around zero.

img_1492

Posted in AI, development, Machine Learning, Neural networks | Tagged , , , , | Leave a comment

Improving Python performance by factor of 100

I’m doing a bit of Machine Learning on my spare time. A neural network that analyzes images.

Anyways, since I don’t have zillions of computing power, any decent amount of learning data takes hours and days to process thru my (very modest, 101 x 10 x 1) neural network.

So today I spent a few hours looking for ways to improve performance. In my code, there was an obvious first candidate, the function that calculates new weights for the various level nodes in the network. Basically, with 101 input nodes, 10 hidden nodes, and 1 output node, and thus 101 hidden weights, each time going thru a learning iteration there are 101 x 100 nodes to process.

My initial attempt was to do the calculations in a nested for-loop. For a fairly limited set of learning data, and equally limited set of learning iterations, the program took in the order of 10h of computing.

Now I changed the implementation of the function computing the new weights, to skip iteration all together, instead using numpy’s powerful matrix manipulation. It took a while to figure out how to do it, but boy, the difference in execution time is striking: from 10 hours to 6 minutes, i.e. a performance boost of factor 100 !
To get a feel for what factor 100 means: apply that factor – either way – to your salary, monthly bills or daily commute time… 🙂

Code below.

 
def new_weights(input,hidden,output,hw,ow,error,mu):

for h,hval in np.ndenumerate(hidden):
for i,ival in np.ndenumerate(input):
slope_o = output * (1 - output)
slope_h = hidden[h] * (1 - hidden[h])
dx3dw = input[i] * slope_h * ow[0][h] * slope_o
hw[h,i] += dx3dw * error * mu

for h,hval in np.ndenumerate(hidden):
slope_o = output * (1. - output)
dx3dw = hidden[h] * slope_o
ow[0][h] += dx3dw * error * mu
return hw,ow

def new_weights2(input,hidden,output,hw,ow,error,mu):

slope_o = output * (1 - output)
slope_h = np.array(hidden * (1 - hidden))
dx3dw = np.outer(input,slope_h) * ow * slope_o
dx3dw = dx3dw.transpose()
hw += dx3dw * error * mu
dx3dw0 = np.outer(hidden,slope_o)
dx3dw0 = dx3dw0.transpose()
ow += dx3dw0 * error * mu
return hw,ow

 

Posted in AI, Complex Systems, development, Machine Learning, Neural networks, performance, software | Tagged , , , , , , | 1 Comment

The power of Machine Learning

https://www.theguardian.com/technology/2017/sep/07/new-artificial-intelligence-can-tell-whether-youre-gay-or-straight-from-a-photograph

Machine Learning has an amazing ability to detect patterns where mere mortals fail to do so.

As always, any technology can be used for good, as well as bad.

Posted in AI, Machine Learning, Neural networks | Tagged , , , | Leave a comment