Browse Source

Finished first complete draft of the fibonacci post complete with mathy latex proofs.

pull/5/head
jrtechs 6 years ago
parent
commit
66d0c02f59
4 changed files with 271 additions and 37 deletions
  1. BIN
      blogContent/headerImages/ConstantTimeComplexity.png
  2. +271
    -36
      blogContent/posts/programming/everything-fibonacci.md
  3. BIN
      blogContent/posts/programming/media/fibonacci/FibonacciMatrix.png
  4. +0
    -1
      utils/markdownToHTML.js

BIN
blogContent/headerImages/ConstantTimeComplexity.png View File

Before After
Width: 1284  |  Height: 480  |  Size: 44 KiB

+ 271
- 36
blogContent/posts/programming/everything-fibonacci.md View File

@ -3,14 +3,16 @@ know what the fibonacci sequence is and how to calculate it.
For those who don't know: [Fibonacci](https://en.wikipedia.org/wiki/Fibonacci) For those who don't know: [Fibonacci](https://en.wikipedia.org/wiki/Fibonacci)
is a sequence of numbers starting with 0,1 whose next number is the sum is a sequence of numbers starting with 0,1 whose next number is the sum
of the two previous numbers. After having multiple of my CS classes of the two previous numbers. After having multiple of my CS classes
give lectures and multiple homeworks on the Fibonacci sequence; I decided
that it would be a good idea to write a blog post going over
the 4 main ways of calculating the nth term of the Fibonacci sequence
and proving their time complexities both mathematically and empirically.
gave lectures and multiple homework on the Fibonacci sequence; I decided
that it would be a great idea to write a blog post going over
the 4 main ways of calculating the nth term of the Fibonacci sequence.
In addition to providing python code for calculating the nth perm of the sequence, a proof for their validity
and analysis of their time complexities both mathematically and empirically will
be examined.
# Slow Recursive Definition # Slow Recursive Definition
By the definition of the Fibonacci sequence, it is natural to write it as
By the definition of the Fibonacci sequence, it is the most natural to write it as
a recursive definition. a recursive definition.
```Python ```Python
@ -20,29 +22,38 @@ def fib(n):
return fib(n-1) + fib(n-2) return fib(n-1) + fib(n-2)
``` ```
#### Time Complexity
##Time Complexity
Observing that each call has two recursive calls we can place an upper bound on this Observing that each call has two recursive calls we can place an upper bound on this
function as O(2^n). However, if we solve this recurrence we can compute the exact value
function as $O(2^n)$. However, if we solve this recurrence we can compute the exact value
and place a tight bound for time complexity. and place a tight bound for time complexity.
We can write a recurrence for the number of times fib is called: We can write a recurrence for the number of times fib is called:
```angular2html
T(1) = 1
T(n) = T(n-1) + T(n-2)
a^n = a^{n-1} + a^{n-2}
a^2 = a + 1
a = \frac{1 + sqrt(5)}{2}
T(n) = \frac{1 + sqrt(5)}{2}^n + \frac{1 1 sqrt(5}{2}^n
$$
F(0) = 0\\
F(1) = 1\\
F(n) = F(n-1) + F(n-2)\\
$$
O(1.618^n)
```
Next we replace each instance of F(n) with $a^n$ since we want to solve for the roots since that
will allow us to put a tight asymptotic limit on the growth.
$$
a^n = a^{n-1} + a^{n-2}\\
\frac{a^n}{a^{n-2}} = \frac{a^{n-1} + a^{n-2}}{a^{n-2}}\\
a^2 = a + 1\\
a = \frac{1 + sqrt(5)}{2}\\
$$
From this calculation we can conclude that F(n) $\in \Theta 1.681^n$
#### Measured Performance
Here is a graph of the actual performance that I observed for this algorithm.
## Measured Performance
Here is a graph of the actual performance that I observed from this recursive definition of Fibonacci.
![Recursive Definition](media/fibonacci/RecursiveDefinition.png) ![Recursive Definition](media/fibonacci/RecursiveDefinition.png)
@ -73,20 +84,84 @@ In this code example fibHelper is a method which accumulates the previous two te
The fibIterative is a wrapper method which sets the two initial terms equal to 0 and 1 The fibIterative is a wrapper method which sets the two initial terms equal to 0 and 1
representing the fibonacci sequence. representing the fibonacci sequence.
Proof in latex that fibHelper
## Proof for Fib Helper
**Lemma:** For any n $\epsilon$ N if n $>$ 1 then
$fibHelper(n, a, b) = fibHelper(n - 1, a, b) + fibHelper(n - 2, a, b)$.
**Proof via Induction**
Base Case: n = 2:
$$
LHS = fibHelper(2, a, b)\\
= fibHelper(1, b, a + b) = a + b\\
RHS = fibHelper(2 -1, a, b) + fibHelper(2-2, a, b)\\
= a + b\\
$$
Inductive Step:
Assume proposition is true for all n and show n+1 follows.
$$
RHS=fibHelper(n+1;a,b)\\
= fibHelper(n;b,a+b)\\
=fibHelper(n-1;b,a+b) + fibHelper(n-2;b,a+b)\\
=fibHelper(n;a,b) + fibHelper(n-1;a,b)\\
=LHS\\
$$
$\Box$
## Proof That fibIterative = Fib
proof in latex that fib iterative = fib
**Lemma:** For any n $\in$ N, $fib(n)$ = $fibIterative(n, 0, 1)$
**Proof via Strong Induction**
Base Case: n = 0:
$$
fibIterative(0, 0, 0) = 0\\
= fib(0)
$$
#### Time Complexity
Base Case: n = 1:
$$
fibIterative(1, 0, 0) = 1\\
= fib(1)
$$
proof in latex for time complexity
Inductive Step:
#### Measured Performance
Assume proposition is true for all n and show n+1 follows.
$$
fib(n+1) = fib(n) + fib(n-1)\\
= fibHelper(n, 0, 1) + fibHelper(n+1, 0 ,1) \quad\text{I.H}\\
= fibHelper(n+1, 0, 1) \quad\text{from result in previous proof}\\
$$
$\Box$
## Time Complexity
Suppose that we wish to solve for time complexity in terms of the number of additions needed to be
computed. By observing the algorithm for fibHelper we can see that we perform one addition every time
which we have a recursive call. We can now form a recurrence for time complexity and solve for it.
$$
T(0) = 0\\
T(1) = 0\\
T(n) = 1 + T(n-1)\\
T(n) = n-1\\
$$
From this recurrence we can say that fibHelper $\in \Theta(n)$.
## Measured Performance
Notice how much faster this solution is compared to the original recursive solution for Notice how much faster this solution is compared to the original recursive solution for
Fibonacci. Also, I only measured going out to 500 because beyond that I hit the maximum
number of recursive calls for my installation of Python.
Fibonacci.
![Iterative Performance](media/fibonacci/Iterative.png) ![Iterative Performance](media/fibonacci/Iterative.png)
@ -95,13 +170,23 @@ number of recursive calls for my installation of Python.
# Matrix Solution # Matrix Solution
We can actually get better than linear time for performance while calculating We can actually get better than linear time for performance while calculating
the Fibonacci sequence recursively.
![Inductive Proof of Fibonacci Matrix](media/fibonacci/FibonacciMatrix.png)
the Fibonacci sequence recursively using this fact:
$$
\begin{bmatrix}
1 & 1\\
1 & 0
\end{bmatrix}^n =
\begin{bmatrix}
F_{n+1} & F_n\\
F_n & F{n-1}
\end{bmatrix}^n
$$
Without any other tricks, raising a matrix to a power n times would not get Without any other tricks, raising a matrix to a power n times would not get
us better than linear performance. However, if we use the [Exponentiation by Squaring](https://en.wikipedia.org/wiki/Exponentiation_by_squaring) us better than linear performance. However, if we use the [Exponentiation by Squaring](https://en.wikipedia.org/wiki/Exponentiation_by_squaring)
method, we can expect to see logarithmic time.
method, we can expect to see logarithmic time. Since two spots in the matrix are always equal,
I represented the matrix as an array with only three elements.
```Python ```Python
@ -128,17 +213,112 @@ def fibPower(n):
return power(l, n)[1] return power(l, n)[1]
``` ```
#### Time Complexity
latex proof for 9lg(n) performance
## Time Complexity
For this algorythem lets solve for the time complexity as the number of additions and multiplications.
Since we are always multiplying two 2x2 matrices, that is constant time.
$$
T_{multiply} = 9
$$
Solving for the time complexity of fib power is slightly more complicated.
$$
T_{power}(1) = 0\\
T_{power}(n) = T(\left\lfloor\dfrac{n}{2}\right\rfloor) + T_{multiply}\\
= T(\left\lfloor\dfrac{n}{2}\right\rfloor) + 9\\
= T(\left\lfloor\dfrac{n}{2*2}\right\rfloor) + 9 + 9\\
= T(\left\lfloor\dfrac{n}{2*2*2}\right\rfloor) + 9+ 9 + 9\\
T_{power}(n) = T(\left\lfloor\dfrac{n}{2^k}\right\rfloor) + 9k\\
$$
let $k=k_0$ such that $\left\lfloor\dfrac{n}{2^{k_0}}\right\rfloor = 1$
$$
\left\lfloor\dfrac{n}{2^{k_0}}\right\rfloor = 1 \rightarrow 1 \leq \frac{n}{2^{k_0}} < 2\\
\rightarrow 2^{k_0} \leq n < 2^{k_0 +1}\\
\rightarrow k_0 \leq lg(n) < k_0+1\\
\rightarrow k_0 = \left\lfloor lg(n)\right\rfloor\\
T_{power}(n) = T(1) + 9*\left\lfloor lg(n)\right\rfloor\\
T_{power}(n) = 9*\left\lfloor\ lg(n)\right\rfloor\\
T_{fibPower}(n) = T_{power}(n)\\
$$
## Inductive Proof for Matrix Method
**Lemma:** For any n $\epsilon$ N if n $>$ 0 then
$$
\begin{bmatrix}
1 & 1\\
1 & 0
\end{bmatrix}^n =
\begin{bmatrix}
F_{n+1} & F_n\\
F_n & F{n-1}
\end{bmatrix}^n
$$
Let
$$
A=
\begin{bmatrix}
1 & 1\\
1 & 0
\end{bmatrix}^n
$$
**Base Case:** n = 1
$$
A^1=
\begin{bmatrix}
1 & 1\\
1 & 0
\end{bmatrix}^n =
\begin{bmatrix}
F_{2} & F_2\\
F_2 & F_{0}
\end{bmatrix}^n
$$
**Inductive Step:** Assume proposition is true for n, show n+1 follows
$$
A^{n+1}=
\begin{bmatrix}
1 & 1\\
1 & 0
\end{bmatrix}
\begin{bmatrix}
F_{n+1} & F_n\\
F_n & F{n-1}
\end{bmatrix}^n\\
= \begin{bmatrix}
F_{n+1} + F_n & F_n + F_{n-1}\\
F_{n+1} & F_{n}
\end{bmatrix}\\
= \begin{bmatrix}
F_{n+2} & F_{n+1}\\
F_{n+1} & F_{n}
\end{bmatrix}\\
$$
$\Box$
## Measured Performance
#### Measured Performance
![FibPower Performance](media/fibonacci/FibPower.png) ![FibPower Performance](media/fibonacci/FibPower.png)
As expected by our mathmatical calcuations, the algorthem appears to be running in As expected by our mathmatical calcuations, the algorthem appears to be running in
logarithmic time. logarithmic time.
#### Measured Performance With Large Numbers
## Measured Performance With Large Numbers
![FibPower Performance](media/fibonacci/FibPowerBigPicture.png) ![FibPower Performance](media/fibonacci/FibPowerBigPicture.png)
When calculating the fibonacci term for extremely large numbers dispite having a polynomial When calculating the fibonacci term for extremely large numbers dispite having a polynomial
@ -153,6 +333,61 @@ the 30th term of Fibonacci.
# Closed Form Definition # Closed Form Definition
It is actually possible to calculate Fibonacci in constant time using a closed form definition.
It is actually possible to calculate Fibonacci in constant time using Binet's Formula.
$$
F_n = \frac{(\frac{1+\sqrt{5}}{2})^n-(\frac{1-\sqrt{5}}{2})^n}{\sqrt{5}}
$$
```Python
def fibClosedFormula(n):
p = ((1+ math.sqrt(5))/2)**n
v = ((1-math.sqrt(5))/2)**n
return (p-v)/math.sqrt(5)
```
## Derivation of Formula
Similar to when we were calculating for the time complexity, we want to start by finding the
two roots of the equation.
$$
a^n = a^{n-1} + a^{n-2}\\
\frac{a^n}{a^{n-2}} = \frac{a^{n-1} + a^{n-2}}{a^{n-2}}\\
a^2 = a + 1\\
0 = a^2 - a - 1\\
a = \frac{1 \pm sqrt(5)}{2}\\
$$
Since there are two roots to the equation, the solution of $F_n$ is going to be
a linear combination of the two roots.
$$
F_n = c_1(\frac{1 + \sqrt{5}}{2})^n + c_2(\frac{1 - \sqrt{5}}{2})^n
$$
Fact: $F_1$ = 1
$$
F_1 = 1\\
= c_1(\frac{1 + \sqrt{5}}{2}) + c_2(\frac{1 - \sqrt{5}}{2})\\
= \frac{c_1}{2} + \frac{c_2}{2} + \frac{c_1\sqrt{5}}{2} - \frac{c_2\sqrt{5}}{2}\\
$$
Let $c_1 = \frac{1}{\sqrt{5}}$,
Let $c_2 = \frac{-1}{\sqrt{5}}$
$$
F_n = \frac{1}{\sqrt(5)}((\frac{1+\sqrt{5}}{2})^n-(\frac{1-\sqrt{5}}{2})^n)\\
= \frac{(\frac{1+\sqrt{5}}{2})^n-(\frac{1-\sqrt{5}}{2})^n}{\sqrt{5}}
$$
## Time Complexity
Since we managed to find the closed form of the fibonacci sequence we can expect to see constant performance.
## Measured Performance
latex proof of closed form definition
![FibPower Performance](media/fibonacci/ConstantTimeComplexity.png)

BIN
blogContent/posts/programming/media/fibonacci/FibonacciMatrix.png View File

Before After
Width: 1330  |  Height: 822  |  Size: 61 KiB

+ 0
- 1
utils/markdownToHTML.js View File

@ -19,7 +19,6 @@ module.exports=
} }
resolve(result); resolve(result);
}; };
console.log(markdownContents);
pandoc(markdownContents, args, callback); pandoc(markdownContents, args, callback);
}); });
}, },

Loading…
Cancel
Save