thus spake a.k.tag:www.thusspakeak.com,2013-04-24://1Movable Type 5.2.13Lucky Sevens - baron m.tag:www.thusspakeak.com,2017:/baron_m//11.2432017-11-17T20:00:00Z2017-11-17T20:05:29Z
Greetings Sir R-----! This evening's chill wind might be forgiven some of its injurious assault upon me by delivering me some good company as I warm my bones. Come, shed your coat and join me in a glass of this rather delightful mulled cyder!
Might you be interested in a little sport whilst we recover?
Excellent!
This foul zephyr puts me in mind of the infantile conflict between King Oberon and Queen Titania that was in full force during my first visit to the faerie kingdom. I had arrived there quite by accident but fortunately my reputation was sufficient to earn me an invitation to dine at the King's table. That the fare was sumptuous beyond the dreams of mortal man goes without saying, but the conflict between the King and his consort cast something of a shadow upon the evening.
baron m.
Greetings Sir R-----! This evening's chill wind might be forgiven some of its injurious assault upon me by delivering me some good company as I warm my bones. Come, shed your coat and join me in a glass of this rather delightful mulled cyder!
Might you be interested in a little sport whilst we recover?
Excellent!
This foul zephyr puts me in mind of the infantile conflict between King Oberon and Queen Titania that was in full force during my first visit to the faerie kingdom. I had arrived there quite by accident but fortunately my reputation was sufficient to earn me an invitation to dine at the King's table. That the fare was sumptuous beyond the dreams of mortal man goes without saying, but the conflict between the King and his consort cast something of a shadow upon the evening.
]]>
We're All Sorted From A To Z - a.k.tag:www.thusspakeak.com,2017:/ak//9.2422017-11-03T19:00:00Z2017-11-03T20:00:21Z
Something that I miss when programming in JavaScript is the wide variety of array manipulation functions available in my primary language, C++. We have, in fact, already implemented one of them with ak.shuffle which randomly rearranges the elements of an array. We shall be needing another one of them in the not too distant future and so I have decided to take a short break from numerical computing to add those of them that I use the most frequently to the ak library, starting with a selection of sorting operations.
a.k.
Something that I miss when programming in JavaScript is the wide variety of array manipulation functions available in my primary language, C++. We have, in fact, already implemented one of them with ak.shuffle which randomly rearranges the elements of an array. We shall be needing another one of them in the not too distant future and so I have decided to take a short break from numerical computing to add those of them that I use the most frequently to the ak library, starting with a selection of sorting operations.
]]>
On Share And Share Alike - studenttag:www.thusspakeak.com,2017:/student//12.2412017-10-20T19:00:00Z2017-10-20T19:01:22Z
When last they met, the Baron challenged Sir R----- to a wager in which, for a price of three coins and fifty cents, he would make a pile of two coins upon the table. Sir R----- was then to cast a four sided die and the Baron would add to that pile coins numbering that upon which it settled. The Baron would then make of it as many piles of equal numbers of no fewer than two coins as he could muster and take back all but one of them for his purse. After doing so some sixteen times, Sir R----- was to have as his prize the remaining pile of coins.
student
When last they met, the Baron challenged Sir R----- to a wager in which, for a price of three coins and fifty cents, he would make a pile of two coins upon the table. Sir R----- was then to cast a four sided die and the Baron would add to that pile coins numbering that upon which it settled. The Baron would then make of it as many piles of equal numbers of no fewer than two coins as he could muster and take back all but one of them for his purse. After doing so some sixteen times, Sir R----- was to have as his prize the remaining pile of coins.
]]>
Do The Evolution - a.k.tag:www.thusspakeak.com,2017:/ak//9.2402017-10-06T19:00:00Z2017-10-06T19:01:23Z
In the last few posts we have taken a look at genetic algorithms, which use simple models of biological evolution to search for global maxima of functions, being those points at which they return their greatest possible values.
These models typically represent the arguments of the function as genes within the binary chromosomes of individuals whose fitnesses are the values of the function for those arguments, exchange genetic information between them with a crossover operator, make small random changes to them with a mutation operator and, most importantly, favour the fitter individuals in the population for reproduction into the next generation with a selection operator.
We used a theoretical analysis of a simple genetic algorithm to suggest improved versions of the crossover operator, as well as proposing more robust schemes for selection and the genetic encoding of the parameters.
In this post we shall use some of them to implement a genetic algorithm for the ak library.
a.k.
In the last few posts we have taken a look at genetic algorithms, which use simple models of biological evolution to search for global maxima of functions, being those points at which they return their greatest possible values.
These models typically represent the arguments of the function as genes within the binary chromosomes of individuals whose fitnesses are the values of the function for those arguments, exchange genetic information between them with a crossover operator, make small random changes to them with a mutation operator and, most importantly, favour the fitter individuals in the population for reproduction into the next generation with a selection operator.
We used a theoretical analysis of a simple genetic algorithm to suggest improved versions of the crossover operator, as well as proposing more robust schemes for selection and the genetic encoding of the parameters.
In this post we shall use some of them to implement a genetic algorithm for the ak library.
]]>
Further Still On A Calculus Of Differences - studenttag:www.thusspakeak.com,2017:/student//12.2392017-09-15T19:00:00Z2017-09-15T19:01:15Zstudent
For some time now my fellow students and I have been whiling away our spare time considering the similarities of the relationships between sequences and series and those between the derivatives and integrals of functions. Having defined differential and integral operators for a sequence s_{n} with
Δ s_{n} = s_{n} - s_{n-1}
and
n
Δ^{-1}s_{n} =
Σ
s_{i}
i = 1
where Σ is the summation sign, we found analogues for the product rule, the quotient rule and the rule of integration by parts, as well as formulae for the derivatives and integrals of monomial sequences, being those whose terms are non-negative integer powers of their indices, and higher order, or repeated, derivatives and integrals in general.
We have since spent some time considering how we might solve equations relating sequences to their derivatives, known as differential equations when involving functions, and it is upon our findings that I shall now report.
]]>
The Best Laid Schemata - a.k.tag:www.thusspakeak.com,2017:/ak//9.2382017-09-01T19:00:00Z2017-09-01T19:18:22Z
We have seen how we can exploit a simple model of biological evolution, known as a genetic algorithm, to search for global maxima of functions, being those points at which they return their greatest values.
This model treated the function being optimised as a non-negative measure of the fitness of individuals to survive and reproduce, replacing negative results with zero, and represented their chromosomes with arrays of bits which were mapped onto its arguments by treating subsets of them as integers that were linearly mapped to floating point numbers with given lower and upper bounds. It simulated sexual reproduction by splitting pairs of the chromosomes of randomly chosen individuals at a randomly chosen position and swapping their bits from it to their ends, and mutations by flipping randomly chosen bits from the chromosomes of randomly chosen individuals. Finally, and most crucially, it set the probability that an individual would be copied into the next generation to its fitness as a proportion of the total fitness of the population, ensuring that that total fitness would tend to increase from generation to generation.
I concluded by noting that, whilst the resulting algorithm was reasonably effective, it had some problems that a theoretical analysis would reveal and that is what we shall look into in this post.
a.k.
We have seen how we can exploit a simple model of biological evolution, known as a genetic algorithm, to search for global maxima of functions, being those points at which they return their greatest values.
This model treated the function being optimised as a non-negative measure of the fitness of individuals to survive and reproduce, replacing negative results with zero, and represented their chromosomes with arrays of bits which were mapped onto its arguments by treating subsets of them as integers that were linearly mapped to floating point numbers with given lower and upper bounds. It simulated sexual reproduction by splitting pairs of the chromosomes of randomly chosen individuals at a randomly chosen position and swapping their bits from it to their ends, and mutations by flipping randomly chosen bits from the chromosomes of randomly chosen individuals. Finally, and most crucially, it set the probability that an individual would be copied into the next generation to its fitness as a proportion of the total fitness of the population, ensuring that that total fitness would tend to increase from generation to generation.
I concluded by noting that, whilst the resulting algorithm was reasonably effective, it had some problems that a theoretical analysis would reveal and that is what we shall look into in this post.
]]>
Share And Share Alike - baron m.tag:www.thusspakeak.com,2017:/baron_m//11.2372017-08-18T19:00:00Z2017-08-18T18:59:30Z
Sir R----- my fine fellow! Come join me in quenching this summer eve's thirst with a tankard of cold ale! Might I presume that your thirst for wager is as pressing as that for refreshment?
I am gladdened to hear it Sir! Gladdened to hear it indeed!
This day's sweltering heat has put me in mind of the time that I found myself temporarily misplaced in the great Caloris rainforest on Mercury. I had been escorting the Velikovsky expedition, which had secured the patronage of the Russian Imperial court for its mission to locate the source of the Amazon, and on one particularly close evening our encampment was attacked by a band of Salamanders which, unlike their diminutive Earthly cousins, stood some eight feet tall and wielded vicious looking barbed spears.
baron m.
Sir R----- my fine fellow! Come join me in quenching this summer eve's thirst with a tankard of cold ale! Might I presume that your thirst for wager is as pressing as that for refreshment?
I am gladdened to hear it Sir! Gladdened to hear it indeed!
This day's sweltering heat has put me in mind of the time that I found myself temporarily misplaced in the great Caloris rainforest on Mercury. I had been escorting the Velikovsky expedition, which had secured the patronage of the Russian Imperial court for its mission to locate the source of the Amazon, and on one particularly close evening our encampment was attacked by a band of Salamanders which, unlike their diminutive Earthly cousins, stood some eight feet tall and wielded vicious looking barbed spears.
]]>
It's All In The Genes - a.k.tag:www.thusspakeak.com,2017:/ak//9.2362017-08-04T19:00:00Z2017-08-04T19:05:22Z
Last time we took a look at the simulated annealing global minimisation algorithm which searches for points at which functions return their least possible values and which drew its inspiration from the metallurgical process of annealing which minimises the energy state of the crystalline structure of metals by first heating and then slowly cooling them.
Now as it happens, physics isn't the only branch of science from which we can draw inspiration for global optimisation algorithms. For example, in biology we have the process of evolution through which the myriad species of life on Earth have become extraordinarily well adapted to their environments. Put very simply this happens because offspring differ slightly from their parents and differences that reduce the chances that they will survive to have offspring of their own are less likely to be passed down through the generations than those that increase those chances.
Noting that extraordinarily well adapted is more or less synonymous with near maximally adapted, it's not unreasonable to suppose that we might exploit a mathematical model of evolution to search for global maxima of functions, being those points at which they return their greatest possible values.
a.k.
Last time we took a look at the simulated annealing global minimisation algorithm which searches for points at which functions return their least possible values and which drew its inspiration from the metallurgical process of annealing which minimises the energy state of the crystalline structure of metals by first heating and then slowly cooling them.
Now as it happens, physics isn't the only branch of science from which we can draw inspiration for global optimisation algorithms. For example, in biology we have the process of evolution through which the myriad species of life on Earth have become extraordinarily well adapted to their environments. Put very simply this happens because offspring differ slightly from their parents and differences that reduce the chances that they will survive to have offspring of their own are less likely to be passed down through the generations than those that increase those chances.
Noting that extraordinarily well adapted is more or less synonymous with near maximally adapted, it's not unreasonable to suppose that we might exploit a mathematical model of evolution to search for global maxima of functions, being those points at which they return their greatest possible values.
]]>
On Divisions - studenttag:www.thusspakeak.com,2017:/student//12.2352017-07-21T19:00:00Z2017-07-21T20:19:50Z
The Baron's game most recent game consisted of a series of some six wagers upon the toss of an unfair coin that turned up one side nine times out of twenty and the other eleven times out of twenty at a cost of one fifth part of a coin. Sir R----- was to wager three coins from his purse upon the outcome of each toss, freely divided between heads and tails, and was to return to it twice the value he wagered correctly.
Clearly, our first task in reckoning the fairness of this game is to figure Sir R-----'s optimal strategy for placing his coins. To do this we shall need to know his expected winnings in any given round for any given placement of his coins.
student
The Baron's game most recent game consisted of a series of some six wagers upon the toss of an unfair coin that turned up one side nine times out of twenty and the other eleven times out of twenty at a cost of one fifth part of a coin. Sir R----- was to wager three coins from his purse upon the outcome of each toss, freely divided between heads and tails, and was to return to it twice the value he wagered correctly.
Clearly, our first task in reckoning the fairness of this game is to figure Sir R-----'s optimal strategy for placing his coins. To do this we shall need to know his expected winnings in any given round for any given placement of his coins.
]]>
Annealing Down - a.k.tag:www.thusspakeak.com,2017:/ak//9.2342017-07-07T19:00:00Z2017-07-07T19:11:16Z
A few years ago we saw how we could search for a local minimum of a function, being a point for which it returns a lesser value than any in its immediate vicinity, by taking random steps and rejecting those that lead uphill; an algorithm that we dubbed the blindfolded hill climber. Whilst we have since seen that we could progress towards a minimum much more rapidly by choosing the directions in which to step deterministically, there is a way that we can use random steps to yield better results.
a.k.
A few years ago we saw how we could search for a local minimum of a function, being a point for which it returns a lesser value than any in its immediate vicinity, by taking random steps and rejecting those that lead uphill; an algorithm that we dubbed the blindfolded hill climber. Whilst we have since seen that we could progress towards a minimum much more rapidly by choosing the directions in which to step deterministically, there is a way that we can use random steps to yield better results.
]]>
Further On A Calculus Of Differences - studenttag:www.thusspakeak.com,2017:/student//12.2332017-06-16T19:00:00Z2017-06-16T19:02:42Zstudent
As I have previously reported, my fellow students and I have found our curiosity drawn to the calculus of sequences, in which we define analogues of the derivatives and integrals of functions for a sequence s_{n} with the operators
Δ s_{n} = s_{n} - s_{n-1}
and
n
Δ^{-1}s_{n} =
Σ
s_{i}
i = 1
respectively, where Σ is the summation sign, for which we interpret all non-positively indexed elements as zero.
I have already spoken of the many and several fascinating similarities that we have found between the derivatives of sequences and those of functions and shall now describe those of their integrals, upon which we have spent quite some mental effort these last few months.
]]>
Copulating Normally - a.k.tag:www.thusspakeak.com,2017:/ak//9.2322017-06-02T19:00:00Z2017-06-10T03:44:14Z
Last year we took a look at multivariate uniformly distributed random variables, which generalise uniform random variables to multiple dimensions with random vectors whose elements are independently uniformly distributed. We have now seen how we can similarly generalise normally distributed random variables with the added property that the normally distributed elements of their vectors may be dependent upon each other; specifically that they may be correlated.
As it turns out, we can generalise this dependence to arbitrary sets of random variables with a fairly simple observation.
a.k.
Last year we took a look at multivariate uniformly distributed random variables, which generalise uniform random variables to multiple dimensions with random vectors whose elements are independently uniformly distributed. We have now seen how we can similarly generalise normally distributed random variables with the added property that the normally distributed elements of their vectors may be dependent upon each other; specifically that they may be correlated.
As it turns out, we can generalise this dependence to arbitrary sets of random variables with a fairly simple observation.
]]>
Divisions - baron m.tag:www.thusspakeak.com,2017:/baron_m//11.2302017-05-19T19:00:00Z2017-05-19T19:00:27Zbaron m.
Greetings Sir R-----! I trust that I find you in good spirit? Will you join me in a draught of this rather fine Cognac and perchance some sporting diversion?
Good man!
I propose a game that ever puts me in mind of an adventure of mine in the town of Bağçasaray, where I was posted after General Lacy had driven Khan Fetih Giray out from therein. I had received word that the Khan was anxious to retake the town and been given orders to hold it at all costs.
]]>
The Cumulative Distribution Unction - a.k.tag:www.thusspakeak.com,2017:/ak//9.2292017-05-05T19:00:00Z2017-05-26T23:56:29Za.k.
We have previously seen how we can generalise normally distributed random variables to multiple dimensions by defining vectors with elements that are linear functions of independent standard normally distributed random variables, having means of zero and standard deviations of one, with
Z' = L × Z + μ
where L is a constant matrix, Z is a vector whose elements are the independent standard normally distributed random variables and μ is a constant vector.
So far we have derived and implemented the probability density function and the characteristic function of the multivariate normal distribution that governs such random vectors but have yet to do the same for its cumulative distribution function since it's a rather more difficult task and thus requires a dedicated treatment, which we shall have in this post.
]]>
On Turnabout Is Fair Play - studenttag:www.thusspakeak.com,2017:/student//12.2272017-04-21T19:00:00Z2017-04-21T19:04:40Z
Last time they met, the Baron challenged Sir R----- to turn a square of twenty five coins, all but one of which the Baron had placed heads up, to tails by flipping vertically or horizontally adjacent pairs of heads.
As I explained to the Baron, although I'm not at all sure that he was following me, this is essentially the mutilated chess board puzzle and can be solved by exactly the same argument. Specifically, we need simply imagine that the game were played upon a five by five checker board...
student
Last time they met, the Baron challenged Sir R----- to turn a square of twenty five coins, all but one of which the Baron had placed heads up, to tails by flipping vertically or horizontally adjacent pairs of heads.
As I explained to the Baron, although I'm not at all sure that he was following me, this is essentially the mutilated chess board puzzle and can be solved by exactly the same argument. Specifically, we need simply imagine that the game were played upon a five by five checker board...
]]>