A New Kind of Sci- I mean, Solution.

In an act of egomania and narcissism that can only be matched by several Stephen Wolfram tomes on cellular automata, I present the object of my obsession over the past two weeks: A New Kind of Solution, and an Application to Recurrence Relations. In this essay (6 pages), I argue that the notion of “closed form solution” has evolved into something inherently computational in nature, and provide an illustration via the computational complexity of solving recurrence relations. This is from the first part of the essay:

The Evolution of Solution

Throughout the history of mathematics, a central goal of its practioners has been to find simple expressions for solutions to equations. Although there is no rigorous definition of “simple expression”, mathematicians have nonetheless spent enormous amounts of effort seeking solutions to their analytical problems that satisfy their aesthetic of simplicity. This aesthetic has evolved over time. In antiquity, “simple” meant rational: the ancient Greeks conducted a (misguided) search for two integers a and b such (a/b)2 = 2. Later, “simple” meant having an algebraic expression, which involve only addition, multiplication, and nth-roots. For centuries, mathematicians sought the elusive quintic formula, an algebraic expression for the roots of a degree 5 polynomial. One of the greatest triumphs of mathematics was Galois’s discovery that there is no such expression, that in general polynomials are too complex to be subject to easy solution. 

More recently, “simple” has been equated with closed form expressions, which has a rather nebulous definition. Generally, closed form expressions are composed of so-called “elementary” operations: the usual arithmetic operations, radicals, exponentiation and logarithms (and thus trigonometric functions are included). The three-body problem is an iconic example of a system of equations resisting all attempts to find a closed form solution. The problem statement is simple: given the initial state (positions and velocities) of 3 point masses experiencing mutual gravitational attraction, determine their state at a later time. The three-body problem has notoriously consumed mathematicians of the likes of Newton, Lagrange, Euler, Hilbert, Poincare. Their efforts have led to a bountiful harvest in terms of a deeper understanding of analysis and dynamical systems, but the holy grail – the closed form solution – to the three-body problem remains out of reach. Indeed, a closed form solution is probably too much to hope for.

As our mathematical understanding advances, so does the reach of “simple”. Each form of simplicity, whether it meant rational, or algebraic, or closed-form, is a reflection of what mathematicians considered amenable to computation at the time. The Greeks abhorred the irrationality of \sqrt{2}, because it meant that geometrical computations would fail to be exact. Before calculus, non-algebraic expressions for roots of polynomials was anathema to mathematicians. Without the use of high speed computers, calculating the precise trajectory of three heavenly objects seemed intractable.

Today, none of these problems give mathematicians headaches anymore, mostly thanks to Moore’s Law. Computers barely work up a sweat when dealing with the three-body problem; astrophysicists now routinely simulate the dynamics of entire galactic clusters. Nearly every area of scientific and industrial computing performs massive numerical linear algebra, which can involve finding the roots of polynomials of degree 1\times 10^6. Today, “simple” means something like “solvable by computers within a reasonable amount of time”.

Fortunately, this has a formal definition in mathematics: it is also known as polynomial-time computability. The formalization of the notion of efficient computation is at heart of computational complexity, the branch of theoretical computer science that studies the difficulty of solving mathematical problems. Just as zoologist attempts to provide a complete classification of life on Earth, so does a computational complexity theorist try to categorize mathematical problems by the computational resources needed to solve them. Perhaps the most important category is the set of problems that are solvable by computers in polynomial time, or \sf P. That is, the number of steps that are required to solve a problem in \sf P scales as a polynomial in the length of the problem instance you’re trying to solve.

Though they were not aware of it, the Greeks, the predecessors of Galois, mathematicians trying to compute the dynamics of 3 bodies – they were seeking limited forms of polynomial time algorithms for their respective problems. One could say that, before electronic computers, the working notion of polynomial time computation was that involving only plus, times, nth-roots, and perhaps exponentiation and logarithms. Now that we do have computers at our disposal, “simple” now encompasses the much more general notion of \sf P. And because of the Church-Turing Thesis, I argue that \sf P is probably the most general notion of “simple” (Let’s leave quantum computers out of the picture for a moment). The buck stops with \sf P: if your laptop can’t solve it efficiently, neither can any other mathematician (A nod to Kamal Jain’s pithy quote).

Many scientists and engineers continue to seek solutions to their problems that fit the traditional notion of simple; closed form expressions are still regarded as the gold standard for tractable solutions. Other than its historical importance, though, the criteria for closed-formedness is largely arbitrary. What is so special about computation methods that only involve +,\times, and \sqrt[n]{}? Why should computing \int{e^{x^2}dx} be any more difficult than computing \int{e^x dx}? Why can’t a polynomial time algorithm count as a solution to a problem?

The age of the closed form expression is over. Modern problems are usually too complex to admit nice, clean formulas. Instead, they require a more general algorithmic approach, and now issues of computational complexity and numerical accuracy take center stage. For solvers of mathematical problems, it is important to understand what can be computed efficiently and accurately. And just as the the discovery of the irrationality of \sqrt{2}, Galois theory, and the 3-body problem have led to major leaps in mathematics, so will the study of algorithms and complexity.

Advertisement

One thought on “A New Kind of Sci- I mean, Solution.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s