jrtechs-NodeJSBlog/blogContent/posts/programming/cs-theory-exam-2-review.md

This a very high level review post that I am making for myself and other people reviewing CS Theory.
If you want to lean more about content in this blog post I recommend cracking open a text book-- I know, gross.
This hastily thrown together post will cover how to solve typical problems relating to topics covered by my second CS Theory exam.

# Myhill-Nerode Theorem

## Definition
L is regular if and only if it has a finite index. The index is the maximum number of elements that are pairwise distinguishable.
Two strings are said to be pairwise distinguishable if you can append something to both of the strings and it makes one string
accepted by the language and the other string rejected. 
The size of an index set X equals the number of equivalence classes it has and the minimum number of states required to represent it
using a DFA. Each element in the language is accepted by only one equivalence class.

## Problem Approach

Prove that language L is regular.

1) Define a set X which is infinite in size - this doesn't have to be in the language.

2) Make a general argument that show that each element in X is pairwise distinguishable.
Pick any two elements x, y in X and show that if you append z to them one is accepted by the language and 
the other is not in the language.

## Example

Prove the following language is non-regular:

$$
L={ww^r | w \in {0,1}^*}
$$

answer:

1)

$$
X = {(01)^i | i \geq 0}
$$

Pick any 2 elements of X and show pairwise distinguishable

$$
x = (01)^i, y = (01)^j  | i \neq j
$$

suppose we pick
$$
z = (10)^i\\
xz \in L\\
yz \notin L
$$


# DFA minimization algorithm

Types of Problems:
- Prove DFA is minimal
- Minimize the DFA

The argument for DFA minimization comes from the Myhill-Nerode theorem. Given
a DFA, if you can form a set of strings which represent each state and they are all 
pairwise distinguishable, then the DFA is minimal with that many states.


## Prove DFA is minimal

For these types of problems you construct a table and show that each state is pairwise distinguishable.
To show pairwise distinguishably you have to show that there exists a string where if appended to one element 
makes it accepted by the language but pushes the other string out of the language.

### Example

Prove the following DFA is minimal.

![DFA Example](media/CSTHEORY/DFAMinimalProof.png)

Find a set of strings which represent the minimal path to each state in the DFA.

$$
X = \{\epsilon, b, bb, ba\}
$$

Show that each state is pairwise distinguishable.

![DFA Example](media/CSTHEORY/DFAMinimalTable.png)


## Minimize the DFA

To use the concept of being indistinguishable to minimize a DFA, you can use a table to keep track which
states are distinguishable from each other. The states which are not indistinguishable can 
be combined. To solve one of these problems you start by creating a table which compares each of the 
states in the DFA. You then go through and mark the states which are indistinguishable -- start with 
the ones with different accepting statuses. Then you continue marking off states where if you transition with
a symbol on the DFA you are distinguishable and the other state is non-distinguishable according to the table.

### Example

Minify the Following DFA:

![DFA Example](media/CSTHEORY/DFAMinification.png)

After marking the states with different accepting criteria as being distinguishable you get this table:

![Half Complete Table](media/CSTHEORY/MinificationTable.svg)


After looping through all pairs and marking them on the table if there exists symbol which results in one state 
to be distinguishable and one to be indistinguishable you get this table:

![Min TableTable](media/CSTHEORY/MinTable2.svg)

According to the table you are able to combine {D, A, B}, {C, F}, and {E, G}.

Minimal DFA:

![Min DFA](media/CSTHEORY/MinimalDFA.svg)


# Pumping lemma for regular languages

The pumping lemma cannot prove that a language is regular, however, you can use it
to show that some languages are non-regular. This theory gets at the idea that if
a regular language is long enough/infinite, it will have a state somewhere which is
repeated on the path that accepts the string.

The accepted strings can be divided into three parts:

- Symbols leading up to the loop
- Symbols which complete a loop and come back to start of loop
- Symbols at the end of the string

![Min DFA](media/CSTHEORY/PumpingLemmaTheory.svg)



To Show that a language L is not regular using pumping lemma:

- Use a proof by contradiction
- Assume L is regular
- Choose a representative string S which is just barely in the language and is represented in terms of p. 
- Express S = xyz such that |xy| < p and y > 0
- Show that you can pump y some amount of times such that it is not in the language.
- This contradicts the pumping lemma.
- The assumption that L is regular is wrong.
- L must not be regular.


## Example

Show that the following language is non-regular.

$$
{0^n1^n | n \geq 0}
$$

Proof by contradiction

Assume that L is regular.

Let p be the pumping length associated with L

$$
S = o^p1^p
$$

S is valid since

$$
|s| \geq p, S \in L
$$

For any valid decomposition

S = xyz

such that |xy| <= p and |y| > 0

Consider:

$$
xy^2z
$$

By the pumping lemma this should be in the language but it is not. Therefore our assumption that the
language is regular is false. 

![Min DFA](media/CSTHEORY/PumpingLemmaExample.svg)

# Context-free grammars, closure properties for CFLs

The context-free grammars are a super-set of the regular languages. This means that CFG's can represent 
some non-regular languages and every regular language. Contest-free Languages are defined by Context-Free Grammars and accepted using
Push-down Automata machines.

Context Free Grammars are Represented using:

- **Terminals** = Set of symbols in that language
- **Variables** = Set of symbols representing categories
- **Start Symbol** = Variable which you start with- written on top
- **Substitution Rules** = Set of rules that recursively define the language.

## Example 1

Grammar G:

$$
A \rightarrow 0A1 \\
A \rightarrow B \\
B \rightarrow \# \\
$$


This grammar describes the following language:

$$
L = \{0^k\#1^k | k \geq 0\}
$$

## Example 2

Give CFG for non-Palindromes

$$
S \rightarrow aXb | bXa | aSa | bSb | ab | ba \\
X \rightarrow aX | bX | a | b \\
$$


In this example, the S rule recursively applies itself until something that is not a palindrome is added.
Once you exit the S state, you can finish by appending anything to the middle of the string. 


## Example 3

Give CFG for the following language:

$$
\{a^ib^jc^kd^l | i+k = j + l\}
$$

$$
S \rightarrow aSd | XYZ \\
X \rightarrow aXb | \epsilon\\
Y \rightarrow bYc | \epsilon\\
Z \rightarrow cZd | \epsilon
$$

## Closure

CFLs are closed under union, concatenation, and Kleene star.
- Union: Create a new starting variable which goes to either branch.
-Kleene Star: We can repeatedly concat the derivations of the string. However, we also need to make sure that epsilon occurs in the string. 
- Concatenation: From start variable we force the concatenation of two variables representing each state sub CFG

# Parse trees, ambiguity

Parse Trees are simply graphical means to illustrate a deviation of a string from a grammar.
The root of the tree will be the start variable, interior nodes are other variables. Leaf nodes are 
terminal symbols. 

A CFG is said to be ambiguous if there is at least one string with two or more distinct derivations.
Leftmost and rightmost derivations are not ambiguous. 

To remove ambiguity try to force order or break CFG into cases.


# Chomsky Normal Form

Useful form for CFGs since they allow you to easily identify if a string is in a language.

Form:

$$
A \rightarrow BC\\
A \rightarrow a\\
S \rightarrow \epsilon
$$

Convert CFG to CNF (Chomsky Normal Form).   

- Add new start variable   

- Remove Epsilon rules (multi-step process)   

- Remove unit rules    

- Convert to A -> BC, A -> a 

## Example

Convert the following CFG to a CNF

$$
S \rightarrow ASA | aB\\
A \rightarrow B | S\\
B \rightarrow b | \epsilon
$$

Step 1: Add new start variable.

$$
S_0 \rightarrow S\\
S \rightarrow ASA | aB\\
A \rightarrow B | S\\
B \rightarrow b | \epsilon
$$

Step 2: Remove epsilon rules.

$$
S_0 \rightarrow S\\
S \rightarrow ASA | aB | a\\
A \rightarrow B | S | \epsilon\\
B \rightarrow b
$$


$$
S_0 \rightarrow S\\
S \rightarrow ASA | aB | SA | AS | S | a\\
A \rightarrow B | S\\
B \rightarrow b
$$

Step 3: Remove unit rules

(Remove A -> B)

$$
S_0 \rightarrow S\\
S \rightarrow ASA | aB | SA | AS | a\\
A \rightarrow b | S\\
B \rightarrow b
$$

(Remove A -> S)

$$
S_0 \rightarrow S\\
S \rightarrow ASA | aB | SA | AS | a\\
A \rightarrow b | ASA | aB | SA | AS | a\\
B \rightarrow b
$$

(Remove S0-> S)

$$
S_0 \rightarrow ASA | aB | SA | AS | a\\
S \rightarrow ASA | aB | SA | AS | a\\
A \rightarrow b | ASA | aB | SA | AS | a\\
B \rightarrow b
$$

# Push-Down automata

These are NFAs but they also have a stack. This allows us to solve any problem which
can be represented with a CFG.

The stack has it's own alphabet. The dollar symbol typically represents the empty stack.

With each transition you can examine the stack, push to the stack and move to a new state.

## Example


$$
L = \{a^n\#b^n\}
$$

![](media/CSTHEORY/PDAExample.svg)


# Construction to convert CFG to a PDA

Basic idea: use stack to hold progressive derivations of a string using rules of grammar.



## Example

Convert the following CFG to a PDA:

$$
S \rightarrow aTb | b \\
T \rightarrow Ta | \epsilon
$$
![](media/CSTHEORY/PDAConstruction.svg)