For the adventures that come, I hope to share them with you at “A Smoother Pebble.” Thanks for all those who read and commented on this blog!

1. Order a double expresso (which is usually < $2).
2. Ask for a cup of ice.
3. Add your favorite quantities of sugar/spice in the expresso.
4. Add liberal amounts of cream/milk to the expresso.
5. Pour the expresso into your cup of ice.
6. Iced Latté!

Which experiment is more probable?
A. You roll 6 fair die and get at least 1 dice that shows up as “6″;
B. You roll 12 fair die and get at least 2 die that show up as “6″;
C. You roll 18 fair die and get at least 3 die that show up as “6″.

The correct answer turns out to be experiment A. Based upon the correspondence between Newton (who provided the correct solution) and Pepys, the wagerer was not immediately convinced of the answer. When challenged, Newton attempted to give an intuitive explanation of the answer, though it is technically incorrect/incomplete (Newton probably knew this). Here it is, paraphrased from the Ye Olde Englishe version:

Imagine that B and C toss their dice in groups of six. A is most favorable because it requires a 6 in only one group toss, while B and C require a 6 in each of their group tosses.

Astute readers will recognize that this intuitive explanation neglects situations where C might get two 6′s in the first or third set, for example.

This problem has been discussed often enough that it has been titled the “Newton-Pepys Problem“, and one is likely to find it given as an elementary probability exercise. I myself came across a more general version of Pepys problem: I win if, in 6n independent rolls of dice I acquire at least n 6′s. You win if, in 6(n + 1) independent rolls of dice you acquire at least n + 1 6′s. Who is more likely to win?

When n = 1 or 2, we know from Newton that I am more likely to win (yay!), but what about for all n? The answer is of course that I still am more likely to win, but how goes the proof? It is unclear whether Newton had a general solution, as his letters to Pepys indicate a purely numerical approach (he calculated the probabilities by hand for A, B and C). My guess is that he had an elegant solution that unfortunately could not fit in the margin of his letter.

And surprisingly, the solution to this general Pepys’s problem cannot fit within the confines of a Wikipedia entry or Wolfram Mathworld page either! The latter aludes to a general solution, which involves this horrifying “regularised hypergeometric function” – what the hell is that?! No, no, there has to be a better way.

Let be the random variable that counts the occurrences of 6′s in independent dice rolls (fair die). Then we are comparing probabilities vs. . Thanks to independence, we can write .

We can forget that the dice have six sides, and instead pretend they’re coins, where you get heads (6) with probability 1/6, and get tails (1,…, 5) with probability 5/6. Then the has a binomial distribution:

with . But treating as a convolution of and , we can instead write:

By implementing the change of variables , we have

where . That is the first trick.

Note that , so in order to compare   vs. we can compare against .

Assuming that (we’ll leave cases to the reader), for .  Here comes the second trick. Note that , which follows from this identity. Thus, we know that in the sum, the are being weighted by numbers that sum up to 1, and so the average must be smaller than the largest probability, which is . Thus, , and so . I win!

The Evolution of Solution

Throughout the history of mathematics, a central goal of its practioners has been to find simple expressions for solutions to equations. Although there is no rigorous definition of “simple expression”, mathematicians have nonetheless spent enormous amounts of effort seeking solutions to their analytical problems that satisfy their aesthetic of simplicity. This aesthetic has evolved over time. In antiquity, “simple” meant rational: the ancient Greeks conducted a (misguided) search for two integers a and b such (a/b)² = 2. Later, “simple” meant having an algebraic expression, which involve only addition, multiplication, and nth-roots. For centuries, mathematicians sought the elusive quintic formula, an algebraic expression for the roots of a degree 5 polynomial. One of the greatest triumphs of mathematics was Galois’s discovery that there is no such expression, that in general polynomials are too complex to be subject to easy solution. 

More recently, “simple” has been equated with closed form expressions, which has a rather nebulous definition. Generally, closed form expressions are composed of so-called “elementary” operations: the usual arithmetic operations, radicals, exponentiation and logarithms (and thus trigonometric functions are included). The three-body problem is an iconic example of a system of equations resisting all attempts to find a closed form solution. The problem statement is simple: given the initial state (positions and velocities) of 3 point masses experiencing mutual gravitational attraction, determine their state at a later time. The three-body problem has notoriously consumed mathematicians of the likes of Newton, Lagrange, Euler, Hilbert, Poincare. Their efforts have led to a bountiful harvest in terms of a deeper understanding of analysis and dynamical systems, but the holy grail – the closed form solution – to the three-body problem remains out of reach. Indeed, a closed form solution is probably too much to hope for.

As our mathematical understanding advances, so does the reach of “simple”. Each form of simplicity, whether it meant rational, or algebraic, or closed-form, is a reflection of what mathematicians considered amenable to computation at the time. The Greeks abhorred the irrationality of , because it meant that geometrical computations would fail to be exact. Before calculus, non-algebraic expressions for roots of polynomials was anathema to mathematicians. Without the use of high speed computers, calculating the precise trajectory of three heavenly objects seemed intractable.

Today, none of these problems give mathematicians headaches anymore, mostly thanks to Moore’s Law. Computers barely work up a sweat when dealing with the three-body problem; astrophysicists now routinely simulate the dynamics of entire galactic clusters. Nearly every area of scientific and industrial computing performs massive numerical linear algebra, which can involve finding the roots of polynomials of degree . Today, “simple” means something like “solvable by computers within a reasonable amount of time”.

Fortunately, this has a formal definition in mathematics: it is also known as polynomial-time computability. The formalization of the notion of efficient computation is at heart of computational complexity, the branch of theoretical computer science that studies the difficulty of solving mathematical problems. Just as zoologist attempts to provide a complete classification of life on Earth, so does a computational complexity theorist try to categorize mathematical problems by the computational resources needed to solve them. Perhaps the most important category is the set of problems that are solvable by computers in polynomial time, or . That is, the number of steps that are required to solve a problem in scales as a polynomial in the length of the problem instance you’re trying to solve.

Though they were not aware of it, the Greeks, the predecessors of Galois, mathematicians trying to compute the dynamics of 3 bodies – they were seeking limited forms of polynomial time algorithms for their respective problems. One could say that, before electronic computers, the working notion of polynomial time computation was that involving only plus, times, nth-roots, and perhaps exponentiation and logarithms. Now that we do have computers at our disposal, “simple” now encompasses the much more general notion of . And because of the Church-Turing Thesis, I argue that is probably the most general notion of “simple” (Let’s leave quantum computers out of the picture for a moment). The buck stops with : if your laptop can’t solve it efficiently, neither can any other mathematician (A nod to Kamal Jain’s pithy quote).

Many scientists and engineers continue to seek solutions to their problems that fit the traditional notion of simple; closed form expressions are still regarded as the gold standard for tractable solutions. Other than its historical importance, though, the criteria for closed-formedness is largely arbitrary. What is so special about computation methods that only involve , and ? Why should computing be any more difficult than computing ? Why can’t a polynomial time algorithm count as a solution to a problem?

The age of the closed form expression is over. Modern problems are usually too complex to admit nice, clean formulas. Instead, they require a more general algorithmic approach, and now issues of computational complexity and numerical accuracy take center stage. For solvers of mathematical problems, it is important to understand what can be computed efficiently and accurately. And just as the the discovery of the irrationality of , Galois theory, and the 3-body problem have led to major leaps in mathematics, so will the study of algorithms and complexity.

Unfortunately, it’s only been used so far to draw quantum circuits, but one could hope that one day there will be an Google Quantum Android SDK that will take in the following…

nop q0
nop q2
ZZ q0,q1
ZZ q2,q3
ZZ q1,q2
MeasH q1
MeasH q2

 

and generate the following…

Recently there was news about a supercomputer in Japan taking the crown of the best in the world, for now. However, I believe that this beast of a machine has NOTHING on what is potentially the most powerful computing device, but yet lies dormant.

I ran across this link: http://www.statlect.com/ideas_computational_engine.htm, which puts forth the rather BRILLIANT idea of harnessing Wikipedia or other heavily-trafficked websites as a supercomputer. Simply install a small amount of JavaScript on each page, and take advantage of the user’s likely supply of idle CPU cycles to do something useful for the world, such as finding cures for cancer, or searching for extraterrestrial life, or folding proteins, or solving some difficult mathematical problem. The possibilities are endless!

What I would really like to see – and here I raise a developer’s call-to-arms – is for someone or a group of people to get together and create a distributed crowd sourced computing platform that runs solely on JavaScript. Then a website owner could just drop a small amount of JavaScript (much like they would to install google analytics) into their page, and voila! Every time a visitor navigates to their website, their computers will be periodically send back computational work done in the browser, until they leave the website.

I could see a company like Google or Amazon getting behind such an effort. I can see universities, governments, organized crime wanting to get involved too. Here are some of the possibilities:

- pedestrian science like curing cancer and folding proteins (NIH, universities)
- search for Ramsey numbers
- run simulations or nuclear explosions (governments)
- run molecular dynamics simulations
- factorize large numbers to break encrypted messages (governments, criminals)
- off load your company’s computing costs to unwitting users across the world by disguising it as a protein folding problem (corporations)

I can for see browsers in the future allowing you to throttle how much “free work” you are willing to do for organizations you don’t even know (it’s not really free because of energy costs). Or, perhaps users can charge for the work being done on their computers! This could open up a whole new market, a new type of economics of computing! For example,

In exchange for lending your CPU cycles, the New York Times might give you free access to their content, and the New York Times could be using your computer to run computations for China, who needs to simulate new materials design for their aircraft fleet.

This is great stuff for a technothriller novel. This reminds me of the many sci-fi stories I’ve read that involve aliens/machines using humans/brains as a computational substrate. We’re not quite there, but it’s happening to our personal computing devices. Our laptops, phones, tablets could be enslaved by shadowy organizations in order to come up with the next super mega-virus, or crack RSA, or run massive DDoS attacks (oh wait….).

Here I go again, letting my imagination run amok…

Sarcasm aside, I’ll try my best to make this somewhat un-soporific! Let me start by briefly describing the motivation for this little article: I wanted to analyze the effect of a repeated sequence of quantum circuits on a noisy input. Let’s say you have a quantum circuit Q that you want to run multiple times in order to perform success probability amplification (much as one might want to amplify a BPP algorithm’s success probability from 2/3 to something exponentially close to 1). One way to do that is have k copies of the circuit Q and run them in parallel, and take the majority vote of the answers.

However, I’m not satisfied with this approach. It’s not environmentally friendly – after all, as a theorist living in California, I have a duty to make my theorems as green as possible. Having k parallel copies of Q requires k-times the number of qubits, and that’s quite wasteful. Getting one or two more coherent qubits in a quantum computer is a nightmare for experimentalists all around the world, so wantonly repeating Q a polynomial number of times would be just insult to injury for them. Let’s be environmentally friendly AND nice!

What about running the circuit Q sequentially k times, reusing the same workspace, while storing the answers of each of the k runs separately? This only requires 1 extra qubit per repetition of Q. The catch is that we have to know that the workspace is relatively clean after each run: if there’s junk in the workspace after the last iteration of Q, it’ll mess up the calculations for the next Q.

Long story short – let’s say you were only able to guarantee that the output workspace of Q is somewhat close to being in a pristine, untouched state (namely, being set to all zeros). Then can we still make this sequential repetition of Q accomplish success amplification?

The answer is “maybe”, but regardless, it led me to conceive of the following cute geometric fact:

Theorem: Let be unit vectors in , and that for all , , where denotes the standard inner product on Euclidean space. Then, .

This is closely related to the standard “Triangle Inequality” for Euclidean space, but instead of an inequality about distances, it’s more a triangle inequality of angles between vectors. But indeed, the proof of this relies on the standard triangle inequality for Euclidean space.

Proof: Consider and . Since , we know that the distance between and (the distance between the endpoints of these vectors) is less than , where is the angle between and . Furthermore, we know for all , the distance between the endpoints of and is less than , meaning, by the triangle inequality, the distance between and is at most . Then . QED.

This seems useful enough to keep around, and maybe you’ll find a use for it? If so, don’t forget to drop some change in the “Starving Mathematician” bucket. Or leave a comment, that works too.

Of course, computers have been undeniably crucial to the progress of science in the last half century, but for the most part, computation has been used as a tool for carrying out science, much as one might have a particle accelerator or gel electrophoresis as a tool. Here, the focus is on viewing quantum physics, molecular biology, evolution, as computational phenomena. If one does this, then one can bring out the big guns of theoretical computer science out – theorems and concepts of complexity, computability, algorithms – and potentially make groundbreaking discoveries in the initial science.

The Lens conference was fantastic. For two days, one found an eclectic but exciting mix of computer scientists, mathematicians, biologists, physicists, economists, etc. etc. together in one room, throwing about ideas, educating one another about their respective fields, challenging each other, all peering through the lens of computer science and trying to understand what they saw.

The best part of this conference was that it was about big ideas - the talks weren’t dense with technical proofs and theorems as most conference talks are. On the contrary, the talks talked more about the major questions, the big mysteries, rather than giving answers. This gave me the feeling of being right on the very frontier of science, that all it takes is the right question, some cleverness, and some hard work, and one will find a treasure trove of scientific discoveries.

Let me describe one talk I found to exemplify the conference theme. Leslie Valiant, winner of the 2010 Turing Award, gave a talk on his recent work on “Evolvability.” He starts with the observation that Darwin’s theory of evolution is incomplete. While Darwin’s theory gives a mechanism for how species change over time, it doesn’t offer any way to quantify evolution. To this day, we have no idea of how to predict how species today will evolve in the future, what they will evolve to, or when they will evolve. Can everything we can imagine be in principle evolved?

More specifically, consider protein regulatory circuits that reside within our bodies. We are amazing pieces of machinery whose continued operation hinges (among other things) on the delicate balance of protein expression levels in our cells. The expression levels of a single protein could be dependent on hundreds if not thousands of other proteins, and this dependency can be extremely complicated. One can view such a dependency as a function that takes as input the expression level of (say) ten other proteins, and the output is the expression level of another. This function presumably is a product of evolution. Is there a limit to how complicated this function could be? Can Nature evolve arbitrarily complicated dependencies?

Here’s where the theory of computation comes into play: we know Nature cannot. As a trivial example, suppose the function was: “Express protein A if and only if protein B’s expression level, encoded as a Turing Machine, halts on input 0.” A strange function, perhaps – but we know from computability theory that there is no way Nature could evolve this function!

This suggests that the limits of evolvability don’t stop there. What about functions that are, in some sense, really hard to acquire? Though evolution on Earth has been going on for billions of years, we don’t expect it to have gotten its hands on a function that we intuitively expect would take a trillion trillion years to learn. These ideas are made unvague in the field of computational learning theory, from which Valiant draws inspiration for his evolvability theory. Incidentally, Valiant won his Turing Award partly for his seminal work in computational learning theory (CLT). In CLT, one can show that there are certain functions that are hard to learn, meaning that it either takes a superpolynomial amount of time to learn, or the function requires an exponential number of examples, etc.

Valiant suggests that one can view evolution as a learning algorithm. Consider a species with a particular genome G. Think of G as a function that maps situations to actions, e.g. G(“Tiger heading my way”) = “Run away” or G(“Mushroom in my view”) = “Eat it”. G is a representation of the evolution algorithm’s current hypothesis of the ideal genome H – the genome that maps situations to the ideal action, with respect to the species’ fitness. The evolution algorithm acts on G, updating it, trying to reach this ideal genome H, and it learns of what changes to make to G via experiences derived from the environment. For example, if G(“Mushroom in my view”) = “Eat it”, and the resulting experience is that of death, then the evolution algorithm should try to change G to G(“Mushroom in my view”) = “Get the hell away”.

Though this is a highly stylized view of evolution, it has the benefit of being amenable to the same kind of analyses one uses in CLT. In fact, Valiant reported some interesting theorems showing that certain kinds of boolean functions are not evolvable in polynomial time under a particular evolution algorithm (polynomial in the length of the genome). The take home message is that now one can introduce concepts like polynomial-time computability to evolution, and that these concepts are vital to understanding what can and cannot be evolved. We should, Valiant claims, only expect Nature to evolve genomes that can be evolved using a polynomial number of steps.

Valiant’s theory of “Evolvability” has a markedly different flavor than other approaches to creating a quantitative theory of evolution. Whereas many scientists are trying to derive quantities such as the rate at which species evolve under sexual reproduction, or how fast gene mixing occurs, or what else, Valiant is trying to mathematically quantify the limits of evolution. Although I think there are many shortcomings to his toy model of evolvability, I believe it is a good start to an understand of evolution that complements other approaches. To adopt the language of theoretical computer science, most other evolutionary biologists/physicists are dealing with “upper bounds” – analyzing a particular evolution algorithm (the one we see in Nature), whereas Valiant is dealing with “lower bounds” – showing what can’t be done by all (polytime) evolution algorithms.

Leslie Valiant’s talk was just one of many fascinating talks given, and maybe I’ll write little vignettes about the others soon.

Sunset

I watch despondently
As the lights wink out
One by one
Great tired suns heaving
Their last sighs
Dousing this universe in a deep
Enveloping gloom.

I am the Omega Point
I am Life itself
I am the triumph
of the uncountably infinite multitudes
who have ever roamed space and time
With my coming came they into me
Their voices joined as one, asking deliverance
From the dark recesses whence they came.

I brace myself -
The final bitter cold sets in,
Holding myself tightly to save the last
Vestiges of warmth; but
slowly,
Quietly,
Like an innocent gas leak
Life seeps out
Into the hungry void.

The candles burn low;
I can almost imagine
A terrible snow storm raging outside.
But there is no snow
There is no storm
There is no outside
Of this quickly emptying cell.

God has sent me to witness
The terminal days of this place
Alone.
There is none left
To share my burden, and to
Weep with me.
For all that ever was,
And could have been
Will never be.