|
@ -3,14 +3,16 @@ know what the fibonacci sequence is and how to calculate it. |
|
|
For those who don't know: [Fibonacci](https://en.wikipedia.org/wiki/Fibonacci) |
|
|
For those who don't know: [Fibonacci](https://en.wikipedia.org/wiki/Fibonacci) |
|
|
is a sequence of numbers starting with 0,1 whose next number is the sum |
|
|
is a sequence of numbers starting with 0,1 whose next number is the sum |
|
|
of the two previous numbers. After having multiple of my CS classes |
|
|
of the two previous numbers. After having multiple of my CS classes |
|
|
give lectures and multiple homeworks on the Fibonacci sequence; I decided |
|
|
|
|
|
that it would be a good idea to write a blog post going over |
|
|
|
|
|
the 4 main ways of calculating the nth term of the Fibonacci sequence |
|
|
|
|
|
and proving their time complexities both mathematically and empirically. |
|
|
|
|
|
|
|
|
gave lectures and multiple homework on the Fibonacci sequence; I decided |
|
|
|
|
|
that it would be a great idea to write a blog post going over |
|
|
|
|
|
the 4 main ways of calculating the nth term of the Fibonacci sequence. |
|
|
|
|
|
In addition to providing python code for calculating the nth perm of the sequence, a proof for their validity |
|
|
|
|
|
and analysis of their time complexities both mathematically and empirically will |
|
|
|
|
|
be examined. |
|
|
|
|
|
|
|
|
# Slow Recursive Definition |
|
|
# Slow Recursive Definition |
|
|
|
|
|
|
|
|
By the definition of the Fibonacci sequence, it is natural to write it as |
|
|
|
|
|
|
|
|
By the definition of the Fibonacci sequence, it is the most natural to write it as |
|
|
a recursive definition. |
|
|
a recursive definition. |
|
|
|
|
|
|
|
|
```Python |
|
|
```Python |
|
@ -20,29 +22,38 @@ def fib(n): |
|
|
return fib(n-1) + fib(n-2) |
|
|
return fib(n-1) + fib(n-2) |
|
|
``` |
|
|
``` |
|
|
|
|
|
|
|
|
#### Time Complexity |
|
|
|
|
|
|
|
|
##Time Complexity |
|
|
|
|
|
|
|
|
Observing that each call has two recursive calls we can place an upper bound on this |
|
|
Observing that each call has two recursive calls we can place an upper bound on this |
|
|
function as O(2^n). However, if we solve this recurrence we can compute the exact value |
|
|
|
|
|
|
|
|
function as $O(2^n)$. However, if we solve this recurrence we can compute the exact value |
|
|
and place a tight bound for time complexity. |
|
|
and place a tight bound for time complexity. |
|
|
|
|
|
|
|
|
We can write a recurrence for the number of times fib is called: |
|
|
We can write a recurrence for the number of times fib is called: |
|
|
```angular2html |
|
|
|
|
|
T(1) = 1 |
|
|
|
|
|
T(n) = T(n-1) + T(n-2) |
|
|
|
|
|
a^n = a^{n-1} + a^{n-2} |
|
|
|
|
|
a^2 = a + 1 |
|
|
|
|
|
a = \frac{1 + sqrt(5)}{2} |
|
|
|
|
|
|
|
|
|
|
|
T(n) = \frac{1 + sqrt(5)}{2}^n + \frac{1 1 sqrt(5}{2}^n |
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
F(0) = 0\\ |
|
|
|
|
|
F(1) = 1\\ |
|
|
|
|
|
F(n) = F(n-1) + F(n-2)\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
O(1.618^n) |
|
|
|
|
|
``` |
|
|
|
|
|
|
|
|
Next we replace each instance of F(n) with $a^n$ since we want to solve for the roots since that |
|
|
|
|
|
will allow us to put a tight asymptotic limit on the growth. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
a^n = a^{n-1} + a^{n-2}\\ |
|
|
|
|
|
\frac{a^n}{a^{n-2}} = \frac{a^{n-1} + a^{n-2}}{a^{n-2}}\\ |
|
|
|
|
|
a^2 = a + 1\\ |
|
|
|
|
|
a = \frac{1 + sqrt(5)}{2}\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
From this calculation we can conclude that F(n) $\in \Theta 1.681^n$ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Measured Performance |
|
|
|
|
|
|
|
|
|
|
|
Here is a graph of the actual performance that I observed for this algorithm. |
|
|
|
|
|
|
|
|
## Measured Performance |
|
|
|
|
|
|
|
|
|
|
|
Here is a graph of the actual performance that I observed from this recursive definition of Fibonacci. |
|
|
|
|
|
|
|
|
![Recursive Definition](media/fibonacci/RecursiveDefinition.png) |
|
|
![Recursive Definition](media/fibonacci/RecursiveDefinition.png) |
|
|
|
|
|
|
|
|
|
|
|
|
|
@ -73,20 +84,84 @@ In this code example fibHelper is a method which accumulates the previous two te |
|
|
The fibIterative is a wrapper method which sets the two initial terms equal to 0 and 1 |
|
|
The fibIterative is a wrapper method which sets the two initial terms equal to 0 and 1 |
|
|
representing the fibonacci sequence. |
|
|
representing the fibonacci sequence. |
|
|
|
|
|
|
|
|
Proof in latex that fibHelper |
|
|
|
|
|
|
|
|
## Proof for Fib Helper |
|
|
|
|
|
**Lemma:** For any n $\epsilon$ N if n $>$ 1 then |
|
|
|
|
|
$fibHelper(n, a, b) = fibHelper(n - 1, a, b) + fibHelper(n - 2, a, b)$. |
|
|
|
|
|
|
|
|
|
|
|
**Proof via Induction** |
|
|
|
|
|
|
|
|
|
|
|
Base Case: n = 2: |
|
|
|
|
|
$$ |
|
|
|
|
|
LHS = fibHelper(2, a, b)\\ |
|
|
|
|
|
= fibHelper(1, b, a + b) = a + b\\ |
|
|
|
|
|
RHS = fibHelper(2 -1, a, b) + fibHelper(2-2, a, b)\\ |
|
|
|
|
|
= a + b\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
Inductive Step: |
|
|
|
|
|
|
|
|
|
|
|
Assume proposition is true for all n and show n+1 follows. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
RHS=fibHelper(n+1;a,b)\\ |
|
|
|
|
|
= fibHelper(n;b,a+b)\\ |
|
|
|
|
|
=fibHelper(n-1;b,a+b) + fibHelper(n-2;b,a+b)\\ |
|
|
|
|
|
=fibHelper(n;a,b) + fibHelper(n-1;a,b)\\ |
|
|
|
|
|
=LHS\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
$\Box$ |
|
|
|
|
|
|
|
|
|
|
|
## Proof That fibIterative = Fib |
|
|
|
|
|
|
|
|
proof in latex that fib iterative = fib |
|
|
|
|
|
|
|
|
**Lemma:** For any n $\in$ N, $fib(n)$ = $fibIterative(n, 0, 1)$ |
|
|
|
|
|
|
|
|
|
|
|
**Proof via Strong Induction** |
|
|
|
|
|
|
|
|
|
|
|
Base Case: n = 0: |
|
|
|
|
|
$$ |
|
|
|
|
|
fibIterative(0, 0, 0) = 0\\ |
|
|
|
|
|
= fib(0) |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
#### Time Complexity |
|
|
|
|
|
|
|
|
Base Case: n = 1: |
|
|
|
|
|
$$ |
|
|
|
|
|
fibIterative(1, 0, 0) = 1\\ |
|
|
|
|
|
= fib(1) |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
proof in latex for time complexity |
|
|
|
|
|
|
|
|
Inductive Step: |
|
|
|
|
|
|
|
|
#### Measured Performance |
|
|
|
|
|
|
|
|
Assume proposition is true for all n and show n+1 follows. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
fib(n+1) = fib(n) + fib(n-1)\\ |
|
|
|
|
|
= fibHelper(n, 0, 1) + fibHelper(n+1, 0 ,1) \quad\text{I.H}\\ |
|
|
|
|
|
= fibHelper(n+1, 0, 1) \quad\text{from result in previous proof}\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
$\Box$ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Time Complexity |
|
|
|
|
|
|
|
|
|
|
|
Suppose that we wish to solve for time complexity in terms of the number of additions needed to be |
|
|
|
|
|
computed. By observing the algorithm for fibHelper we can see that we perform one addition every time |
|
|
|
|
|
which we have a recursive call. We can now form a recurrence for time complexity and solve for it. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
T(0) = 0\\ |
|
|
|
|
|
T(1) = 0\\ |
|
|
|
|
|
T(n) = 1 + T(n-1)\\ |
|
|
|
|
|
T(n) = n-1\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
From this recurrence we can say that fibHelper $\in \Theta(n)$. |
|
|
|
|
|
|
|
|
|
|
|
## Measured Performance |
|
|
|
|
|
|
|
|
Notice how much faster this solution is compared to the original recursive solution for |
|
|
Notice how much faster this solution is compared to the original recursive solution for |
|
|
Fibonacci. Also, I only measured going out to 500 because beyond that I hit the maximum |
|
|
|
|
|
number of recursive calls for my installation of Python. |
|
|
|
|
|
|
|
|
Fibonacci. |
|
|
|
|
|
|
|
|
![Iterative Performance](media/fibonacci/Iterative.png) |
|
|
![Iterative Performance](media/fibonacci/Iterative.png) |
|
|
|
|
|
|
|
@ -95,13 +170,23 @@ number of recursive calls for my installation of Python. |
|
|
# Matrix Solution |
|
|
# Matrix Solution |
|
|
|
|
|
|
|
|
We can actually get better than linear time for performance while calculating |
|
|
We can actually get better than linear time for performance while calculating |
|
|
the Fibonacci sequence recursively. |
|
|
|
|
|
|
|
|
|
|
|
![Inductive Proof of Fibonacci Matrix](media/fibonacci/FibonacciMatrix.png) |
|
|
|
|
|
|
|
|
the Fibonacci sequence recursively using this fact: |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
1 & 1\\ |
|
|
|
|
|
1 & 0 |
|
|
|
|
|
\end{bmatrix}^n = |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
F_{n+1} & F_n\\ |
|
|
|
|
|
F_n & F{n-1} |
|
|
|
|
|
\end{bmatrix}^n |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
Without any other tricks, raising a matrix to a power n times would not get |
|
|
Without any other tricks, raising a matrix to a power n times would not get |
|
|
us better than linear performance. However, if we use the [Exponentiation by Squaring](https://en.wikipedia.org/wiki/Exponentiation_by_squaring) |
|
|
us better than linear performance. However, if we use the [Exponentiation by Squaring](https://en.wikipedia.org/wiki/Exponentiation_by_squaring) |
|
|
method, we can expect to see logarithmic time. |
|
|
|
|
|
|
|
|
method, we can expect to see logarithmic time. Since two spots in the matrix are always equal, |
|
|
|
|
|
I represented the matrix as an array with only three elements. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```Python |
|
|
```Python |
|
@ -128,17 +213,112 @@ def fibPower(n): |
|
|
return power(l, n)[1] |
|
|
return power(l, n)[1] |
|
|
``` |
|
|
``` |
|
|
|
|
|
|
|
|
#### Time Complexity |
|
|
|
|
|
|
|
|
|
|
|
latex proof for 9lg(n) performance |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Time Complexity |
|
|
|
|
|
|
|
|
|
|
|
For this algorythem lets solve for the time complexity as the number of additions and multiplications. |
|
|
|
|
|
|
|
|
|
|
|
Since we are always multiplying two 2x2 matrices, that is constant time. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
T_{multiply} = 9 |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
Solving for the time complexity of fib power is slightly more complicated. |
|
|
|
|
|
$$ |
|
|
|
|
|
T_{power}(1) = 0\\ |
|
|
|
|
|
T_{power}(n) = T(\left\lfloor\dfrac{n}{2}\right\rfloor) + T_{multiply}\\ |
|
|
|
|
|
= T(\left\lfloor\dfrac{n}{2}\right\rfloor) + 9\\ |
|
|
|
|
|
= T(\left\lfloor\dfrac{n}{2*2}\right\rfloor) + 9 + 9\\ |
|
|
|
|
|
= T(\left\lfloor\dfrac{n}{2*2*2}\right\rfloor) + 9+ 9 + 9\\ |
|
|
|
|
|
T_{power}(n) = T(\left\lfloor\dfrac{n}{2^k}\right\rfloor) + 9k\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
let $k=k_0$ such that $\left\lfloor\dfrac{n}{2^{k_0}}\right\rfloor = 1$ |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
\left\lfloor\dfrac{n}{2^{k_0}}\right\rfloor = 1 \rightarrow 1 \leq \frac{n}{2^{k_0}} < 2\\ |
|
|
|
|
|
\rightarrow 2^{k_0} \leq n < 2^{k_0 +1}\\ |
|
|
|
|
|
\rightarrow k_0 \leq lg(n) < k_0+1\\ |
|
|
|
|
|
\rightarrow k_0 = \left\lfloor lg(n)\right\rfloor\\ |
|
|
|
|
|
T_{power}(n) = T(1) + 9*\left\lfloor lg(n)\right\rfloor\\ |
|
|
|
|
|
T_{power}(n) = 9*\left\lfloor\ lg(n)\right\rfloor\\ |
|
|
|
|
|
T_{fibPower}(n) = T_{power}(n)\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Inductive Proof for Matrix Method |
|
|
|
|
|
|
|
|
|
|
|
**Lemma:** For any n $\epsilon$ N if n $>$ 0 then |
|
|
|
|
|
$$ |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
1 & 1\\ |
|
|
|
|
|
1 & 0 |
|
|
|
|
|
\end{bmatrix}^n = |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
F_{n+1} & F_n\\ |
|
|
|
|
|
F_n & F{n-1} |
|
|
|
|
|
\end{bmatrix}^n |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
Let |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
A= |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
1 & 1\\ |
|
|
|
|
|
1 & 0 |
|
|
|
|
|
\end{bmatrix}^n |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
**Base Case:** n = 1 |
|
|
|
|
|
$$ |
|
|
|
|
|
A^1= |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
1 & 1\\ |
|
|
|
|
|
1 & 0 |
|
|
|
|
|
\end{bmatrix}^n = |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
F_{2} & F_2\\ |
|
|
|
|
|
F_2 & F_{0} |
|
|
|
|
|
\end{bmatrix}^n |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
**Inductive Step:** Assume proposition is true for n, show n+1 follows |
|
|
|
|
|
$$ |
|
|
|
|
|
A^{n+1}= |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
1 & 1\\ |
|
|
|
|
|
1 & 0 |
|
|
|
|
|
\end{bmatrix} |
|
|
|
|
|
\begin{bmatrix} |
|
|
|
|
|
F_{n+1} & F_n\\ |
|
|
|
|
|
F_n & F{n-1} |
|
|
|
|
|
\end{bmatrix}^n\\ |
|
|
|
|
|
= \begin{bmatrix} |
|
|
|
|
|
F_{n+1} + F_n & F_n + F_{n-1}\\ |
|
|
|
|
|
F_{n+1} & F_{n} |
|
|
|
|
|
\end{bmatrix}\\ |
|
|
|
|
|
= \begin{bmatrix} |
|
|
|
|
|
F_{n+2} & F_{n+1}\\ |
|
|
|
|
|
F_{n+1} & F_{n} |
|
|
|
|
|
\end{bmatrix}\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
$\Box$ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Measured Performance |
|
|
|
|
|
|
|
|
#### Measured Performance |
|
|
|
|
|
![FibPower Performance](media/fibonacci/FibPower.png) |
|
|
![FibPower Performance](media/fibonacci/FibPower.png) |
|
|
|
|
|
|
|
|
As expected by our mathmatical calcuations, the algorthem appears to be running in |
|
|
As expected by our mathmatical calcuations, the algorthem appears to be running in |
|
|
logarithmic time. |
|
|
logarithmic time. |
|
|
|
|
|
|
|
|
#### Measured Performance With Large Numbers |
|
|
|
|
|
|
|
|
## Measured Performance With Large Numbers |
|
|
|
|
|
|
|
|
![FibPower Performance](media/fibonacci/FibPowerBigPicture.png) |
|
|
![FibPower Performance](media/fibonacci/FibPowerBigPicture.png) |
|
|
|
|
|
|
|
|
When calculating the fibonacci term for extremely large numbers dispite having a polynomial |
|
|
When calculating the fibonacci term for extremely large numbers dispite having a polynomial |
|
@ -153,6 +333,61 @@ the 30th term of Fibonacci. |
|
|
|
|
|
|
|
|
# Closed Form Definition |
|
|
# Closed Form Definition |
|
|
|
|
|
|
|
|
It is actually possible to calculate Fibonacci in constant time using a closed form definition. |
|
|
|
|
|
|
|
|
It is actually possible to calculate Fibonacci in constant time using Binet's Formula. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
F_n = \frac{(\frac{1+\sqrt{5}}{2})^n-(\frac{1-\sqrt{5}}{2})^n}{\sqrt{5}} |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
```Python |
|
|
|
|
|
def fibClosedFormula(n): |
|
|
|
|
|
p = ((1+ math.sqrt(5))/2)**n |
|
|
|
|
|
v = ((1-math.sqrt(5))/2)**n |
|
|
|
|
|
return (p-v)/math.sqrt(5) |
|
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
|
|
## Derivation of Formula |
|
|
|
|
|
|
|
|
|
|
|
Similar to when we were calculating for the time complexity, we want to start by finding the |
|
|
|
|
|
two roots of the equation. |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
a^n = a^{n-1} + a^{n-2}\\ |
|
|
|
|
|
\frac{a^n}{a^{n-2}} = \frac{a^{n-1} + a^{n-2}}{a^{n-2}}\\ |
|
|
|
|
|
a^2 = a + 1\\ |
|
|
|
|
|
0 = a^2 - a - 1\\ |
|
|
|
|
|
a = \frac{1 \pm sqrt(5)}{2}\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
Since there are two roots to the equation, the solution of $F_n$ is going to be |
|
|
|
|
|
a linear combination of the two roots. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
F_n = c_1(\frac{1 + \sqrt{5}}{2})^n + c_2(\frac{1 - \sqrt{5}}{2})^n |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
Fact: $F_1$ = 1 |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
F_1 = 1\\ |
|
|
|
|
|
= c_1(\frac{1 + \sqrt{5}}{2}) + c_2(\frac{1 - \sqrt{5}}{2})\\ |
|
|
|
|
|
= \frac{c_1}{2} + \frac{c_2}{2} + \frac{c_1\sqrt{5}}{2} - \frac{c_2\sqrt{5}}{2}\\ |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
Let $c_1 = \frac{1}{\sqrt{5}}$, |
|
|
|
|
|
Let $c_2 = \frac{-1}{\sqrt{5}}$ |
|
|
|
|
|
|
|
|
|
|
|
$$ |
|
|
|
|
|
F_n = \frac{1}{\sqrt(5)}((\frac{1+\sqrt{5}}{2})^n-(\frac{1-\sqrt{5}}{2})^n)\\ |
|
|
|
|
|
= \frac{(\frac{1+\sqrt{5}}{2})^n-(\frac{1-\sqrt{5}}{2})^n}{\sqrt{5}} |
|
|
|
|
|
$$ |
|
|
|
|
|
|
|
|
|
|
|
## Time Complexity |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Since we managed to find the closed form of the fibonacci sequence we can expect to see constant performance. |
|
|
|
|
|
|
|
|
|
|
|
## Measured Performance |
|
|
|
|
|
|
|
|
latex proof of closed form definition |
|
|
|
|
|
|
|
|
![FibPower Performance](media/fibonacci/ConstantTimeComplexity.png) |