#import "@preview/ilm:1.4.1": * #import "@preview/ctheorems:1.1.3": * #import "@preview/finite:0.5.0": automaton #show: thmrules #show: ilm.with( title: [CS3231 - The Theory of Computing], author: "Yadunand Prem", // bibliography: bibliography("refs.bib"), table-of-contents: none, // figure-index: (enabled: true), // table-index: (enabled: true), // listing-index: (enabled: true), ) #set math.equation(numbering: none) #set text(lang: "en", font: ("SF Pro Display")) #show raw: set text(font: "SF Mono") #let theorem = thmbox("theorem", "Theorem") #let definition = thmbox("definition", "Definition", inset: (x: 1.2em, top: 1em)) #let proof = thmproof("proof", "Proof") #let example = thmplain("example", "Example") #let numeqn(eq) = math.equation(block: true, numbering: "(1)", eq) #let dh = math.hat(math.delta) #let es = math.epsilon.alt = Chapter 1 == Deductive Proofs Sequence of statements, whose truth leads us from some initial statement, the _hypothesis_ to _conclusion_. Each step in the proof must follow by some accepted logical principle, either from facts or some of the previous statements in the deductive proof. #theorem[If $x >=4$ then $2^x >= x^2$] #proof[ ] #theorem[If $x$ is the sum of the squares of 4 positive integers, then $2^x >= x^2$] #proof[ + $x = a^2 + b^2 + c^2 + d^2$ (Given) + $a >= 1, b >= 1, c >= 1, d >= 1$ (Given) + $a^2 >= 1, b^2 >= 1, c^2 >= 1, d^2 >= 1$ ((2) and properties of arithmetic) + $x >= 4$ ((1), (3), and properties of arithmetic + $2^x >=x^2$ ((4) and Theorem 1.3) #qedhere ] == Proof by Contradiction Another way to prove statements of the form "if H then C" is to prove "H and not C implies falsehood". We can start by assuming both hypothesis $H$ and negation of conclusion $C$. Complete proof by showing that something known to be false follows logically from $H$ and $"not" C$ == Proofs about sets === Equivalences To prove the equality of to sets, $E$ and $F$, $E=F$, we need to prove the following. + Proof that if $x$ is in $E$, then $x$ is in $F$. + Proof that if $x$ is in $F$, then $x$ is in $E$. #theorem[$R union (S inter T) = (R union S) inter (R union T)$] #proof[The two set expressions involved are $E = R union (S inter T)$ and $F = (R union S) inter (R union T)$. In the _if_ part, we assume element $x$ is in $E$ and show it is in $F$. + $x$ is in $R union (S inter T)$ (given) + $x$ is in $R$ or $x$ is in $S inter T$ (defn of union) + $x$ is in $R$ or $x$ is in both $S$ and $T$ (defn of intersection) + $x$ is in $R union S$ (defn of union) + $x$ is in $R union T$ (defn of union) + $x$ is in $(R union S) inter (R union T)$ (4, 5, defn of intersection) In the _only if_ part, we assume element $x$ is in $F$ and show it is in $E$. + $x$ is in $(R union S) inter (R union T)$ (Given) + $x$ is in $(R union S)$ (defn of intersection) + $x$ is in $(R union T)$ (defn of intersection) + $x$ is in $R$ or $x$ is in both $S$ and $T$ (2, 3, reasoning about unions) + $x$ is in $R$ or $x$ is in $S inter T$ (defn of intersection) + $x$ is in $R union (S inter T)$ (defn of union) ] == Inductive Proofs Suppose we are given statement $S(n)$ about an integer $n$ to prove. We need to prove 2 things. + The _basis_, where we show $S(i)$ for a particular integer $i$. Usually $i = 0$ or $i = 1$. + The _inductive_ step, where we assume $n >=i$, where $i$ is the basis integer, and we show that "if $S(n)$ then $S(n+1)$" These 2 parts should convince us that $S(n)$ is true for every integer $n$ that is equal to or greater than basis integer $i$. = Central Concept of Automata Theory == Alphabet Alphabet is a finite, nonempty set of symbols. We use the symbol $Sigma$ for alphabet. Common alphabets include + $Sigma = {0, 1}$ the binary alphabet + $Sigma = {a, b, ..., z}$ the set of all lowercase letters == String String is a finite sequence of symbols chosen from some alphabet. $01101$ is a string from the binary alphabet. *Empty String* is the string with 0 occurances of symbols, denoted by $es$. This string is a string that may be chosen from any alphabet whatsoever *Length of String* is denoted by $|w|$ , where $w$ is a string *Powers of an Alphabet* If $Sigma$ is an alphabet, we can express the set of all strings of a certain length. We define $Sigma^k$ to be the set of strings of length $k$, whose embols is in $Sigma$. Note that $Sigma^0 = { es }$ The set of all strings over an alphabet $Sigma$ is denoted $Sigma^*$. For instance, ${0, 1}^* = {es, 0, 1, 00, 01, 10, 11, ...}$ The set of nonempty strings is denoted by $Sigma^+$ The set of nonempty strings is denoted by $Sigma^+$ - $Sigma^+ = Sigma^1 union Sigma^2 union ...$ - $Sigma^* = Sigma^+ union {es}$ *Concat of Strings* Let $x$ and $y$ be strings. $x y$ denotes concatenation of $x$ and $y$. If $x$ is a string composed of $i$ symbols $x = a_1a_2...a_i$ and $y$ is the string composed of $j$ symbols $y = b_1b_2...b_j$ then $x y$ is the string of length $i + j: x y = a_1...a_i b_1...b_j$. == Languages A set of strings all of which are chosen from some $Sigma^*$, where $Sigma$ is a particular alphabet, is called a language. If $Sigma$ is an alphabet, and $L subset.eq Sigma^*$, then $L$ is a language over $Sigma$. Language over $Sigma$ need not include strings with all the symbols of $Sigma$, so once we have established than $L$ is a language over $Sigma$, we also know it is a language over any alphabet that is a superset of $Sigma$. Example languages: - $Sigma^*$ is a language for any alphabet $Sigma$. - $emptyset$,the empty language, is a language over any alphabet - ${es}$, the language consisting of only empty string. Only constraint on what can be a language is that all alphabets are finite. Languages can have an infinite number of strings, but are restricted to consist of strings drawn from one fixed, finite alphabet. = Deterministic Finite Automata DFA consists of + Finite set of states, often denoted $Q$ + Finite set of input symbols,often denoted $Sigma$ + A transition function that takes as arguments a state, and an input symbol, and returns a state. Commonly denoted by $delta$ + A start state, one of the states in $Q$ + A set of final or accepting states $F$. $F subset Q$ In proofs we often talk about DFA in "5 tuple" notation: $ A = (Q, Sigma, delta, q_0, F) $ where $A$ is the name of the DFA. == DFA Processes Strings The Language of a DFA: The set of all strings that result in a sequence of state transitions from the _start_ state to an _accepting_ state. We define an _extended transition function_ to describe what happens when we start in any state, and follow a _sequence_ of inputs, denoted as $hat(delta)$. The extended transition function is a function that takes state $q$ and a string $w$ and returns state $p$ - the state automaton reaches when starting in state $q$ and processing the sequence of input $w$. We define $hat(delta)$ by induction on the length of input string, as follows: #definition[][*Extended Transition Function* *Basis: * $hat(delta)(q, es) = q$. That is if we are in state $q$ and read no inputs, we are still in state q. * Induction: * Suppose $w$ is a string of the form $x a$, that is $a$ is the last symbol of $w$ and $x$ is the string consisting of all but the last symbol. Then $ hat(delta)(q, w) = delta(hat(delta)(q, x), a) $ ] == Language of DFA The language of a DFA $A = (Q, Sigma, delta, q_0, F)$, denoted by $L(A)$ is defined by $ L(A) = {w | hat(delta)(q_0, w) in F} $ That is, the language of $A$ is the set of strings $w$ that take the start state $q_0$ to one of the accepting states. If $L$ is $L(A)$ for some DFA A, then $L$ is a *regular language* #theorem[For any state $q$ and string $x$ and $y$, $hat(delta)(q, x y) = hat(delta)(hat(delta)(q, x), y)$] #proof[By inducting on $|y|$ Base case: $(y = es)$: $hat(delta)(q, x es) = hat(delta)(q, x)$ and $hat(delta)(hat(delta)(q, x), es) = hat(delta)(q, x)$ Inductive Step: Assume the statement holds for some $y = w in Sigma^*$, i.e. $hat(delta)(q, x w) = hat(delta)(hat(delta)(q, x), w)$ Let $y = w a, a in Sigma$, we have + $hat(delta)(q, x w a) = delta(hat(delta)(q, x w) a)$ (defn of $hat(delta)$) + $= delta(hat(delta)(hat(delta)(q, x), w), a)$ (Apply IH) + $= hat(delta)(hat(delta)(q, x), w a)$ (defn of $hat(delta)$) So the statement holds true for $w a$. ] #theorem[For any state $q$, string $x$ and symbol $a$, $hat(delta)(q, a x) = hat(delta)(delta(q, a), x)$] #proof[ + Let $x$ = $a$ and $y$ = $x$. Then, $hat(delta)(q, a x) = hat(delta)(hat(delta)(q, a), x)$. + $= hat(delta)(delta(q, a), x)$ (by defn of $hat(delta)$) ] = Nondeterministic Finite Automata NFA has a set of finite states, finite input symbols, 1 start and a set of accepting states. NFA's transition function takes a state and input symbols but returns a *set* of 0, 1, or more states. #definition("Nondeterministic Finite Automata")[ $ A = (Q, Sigma, delta, q_0, F) $, where + $Q$ is a finite set of states + $Sigma$ is a finite set of symbols + $q_0 in Q$, is the start state + $F subset Q$, set of final states + $delta$, the transition function, takes in a state in $Q$ and an input symbol in $Sigma$ and returns a subset of $Q$. ] == Extended Transition function function $dh$ takes a state $q$, and a string of input symbols $w$, and returns the set of states that the NFA is in if it starts in state $q$ and processes string $w$. #definition[Extended Transition Function for NFA][ *Basis:* $dh(q, es) = {q}$. That is, without reading any input symbols, we are only in the state we began in. *Induction:* Suppose $w$ is of the form $w = x a$, where $a$ is the final symbol of $w$ and $x$ is the rest of $w$. Also suppose that $dh(q, x) = {p_1, ..., p_k}$. Let $ union.big^k_(i=1) delta(p_i, a) = {r_1, r_2, ..., r_m} $ Then $dh(q, w) = {r_1, r_2, ..., r_m}$ ] == Language of NFA NFA accepts string $w$ if it is possible to make any sequence of choices of next state, while reading characters of $w$, and go from start state to any accepting state. If $A = (Q, Sigma, delta, q_0, F)$ is an NFA, then $ L(A) = {w | dh(q_0, w) inter F != emptyset $ That is, $L(A)$ is the set of strings $w$ in $Sigma^*$ such that $dh(q_0, w)$ contains at least 1 accepting state #example[ #automaton(( q0: (q0: "0, 1", q1: 0), q1: (q2: 1), q2: (), )) Prove formally that this NFA accepts language $L = {w | w "ends in 01" }$ ] #proof[ The following 3 statements characterisze the 3 states: + $dh(q_0, w)$ contains $q_0$ for every $w$ + $dh(q_0, w)$ contains $q_1$ if and only if $w$ ends in 0. + $dh(q_0, w)$ contains $q_2$ if and only if $w$ ends in 01. We prove by induction on $|w|$. *Basis: * If $|w| = 0, "then" w = es$. - Statement (1) says that $dh(q_0, es)$ contains $q_0$, by defn of $dh$. - Statement (2), we know that $es$ does not end in 0, and $dh(q_0, es)$ does not contain $q_1$ by defn of $dh$ - Statement (3), same as statement 2. *Induction: * Assume $w = x a$, where $a$ is a symbol either in $0$ or $1$. We assume statements 1-3 hold for $x$, and we need to prove them for $w$, that is, we assume $|w| = n + 1, |x| = n$. + $dh(q_0, x)$ contains $q_0$. Since there are transitions from 0/1 from $q_0$ to itself, it follows that $dh(q_0, w)$ also contains $q_0$, so statement 1 is proved for $w$ + (If) Assume $w$ ends in 0, i.e. $a = 0$. By statement (1) applied to $x$, we know that $q_0 in dh(q_0, x)$. Since there are transitions from $q_0$ to $q_1$ on input 0, we know that $q_1 in dh(q_0, w)$.\ (Only-if) Assume $q_1 in dh(q_0, w)$. Only way to get to $q_1$ is if $w = x 0$. + (If) Assume $w$ ends in 01. If $w = x a$, then $a = 1$ and $x$ ends in 0. By statement 2 applied to $x$, we know that $q_1 in dh(q_0, x)$. Since there is a transition from $q_1$ to $q_2$ on input 1, we conclude that $q_2 in dh(q_0, w)$ \ (Only-if) Suppose $q_2 in dh(q_0, w)$. Only way to get to $q_2$ is for $w$ to be of the form $x 1$, where $q_1 in dh(q_0, w)$. By (2) applied to $x$, we know that $x$ ends in 0. Thus, $w$ ends in $01$. ] == Equivalence of DFA and NFA We prove this using subset construction. We start with NFA $N = (Q_N, Sigma, delta_N, Q_0, F_N)$. The goal is the description of a DFA $D = (Q_D, Sigma, delta_D, {q_0}, F_D)$ such that $L(D) = L(N)$. The input alphabets are the same, and the start of $D$ is the set containing only the start state of $N$. - $Q_D$ is the set of subsets of $Q_N$, that is $Q_D$ is the power set of $Q_N$. If $Q_N$ has $n$ states, $Q_D$ has $2^n$ states. - $F_D$ is the set of subsets $S$ of $Q_N$ such that $S inter F_N != emptyset$. $F_D$ is all sets of $N$'s states that include at least 1 accepting state of $N$. - For each set $S subset.eq Q_N$, and for each input symbol $a$ in $Sigma$, $ delta_D (S, a) = union.big_(p in S) delta_N (p, a) $ #theorem[If $D = (Q_d, Sigma, delta_D, {q_0}, F_D)$ is the DFA constructed from NFA $N = (Q_N, Sigma, delta_N, q_0, F_N)$ by subset construction, then $L(D) = L(N)$.] #proof[ #set math.equation(numbering: "(1)") We prove by induction on $|w|$ that $ dh_D ({q_0}, w) = dh_N (q_0, w) $ Notice that each of the $dh$ function returns a set of states from $Q_N$, but $dh_D$ interprets this set as one of the states of $Q_D$, while $dh_N$ interprets this set as a subset of $Q_N$. *Basis: *Let $|w| = 0, w = es$. By basis definition of $dh$ for DFA and NFA, both $dh_D ({q_0}, es)$ and $dh_N (q_0, es)$ are ${q_0}$ *Induction: * Let $w$ be of length $n+1$, assume statement for length $n$. $w = x a$, where $a$ is final symbol of $w$. By inductive hypothesis, $dh_D ({q_0}, x) = dh_N (q_0, x) = {p_1,..., p_k}$ @etf-nfa tells us that $ dh_N(q_0, w) = union.big^k_(i=1) delta_N (p_i, a) $ and subset construction tells us $ delta_D ({p_1, ..., p_k}, a) = union.big^k_(i=1) delta_N (p_i, a) $. We can use this to construct $ dh_D ({q_0}, w) = delta_D (dh_D ({q_0}, x), a) = delta_D ({p_1, ..., p_k}, a) = union.big^k_(i=1) delta_N (p_i, a) $ Thus, (2) and (4) demonstrate that $dh_D ({q_0}, w) = dh_N (q_0, w)$. When we observe that $D$ and $N$ both accept $w$ if and only if $dh_D ({q_0}, w)$ or $dh_N (q_0, w)$, respectively, contain a state $F_N$, we have a completed proof that $L(D) = L(N)$ ] #theorem[Language $L$ is accepted by some DFA if and only if $L$ is accepted by some NFA] #proof[ (If) The if part is subset construction and @subset-constr (Only if) Convert a DFA into an identical NFA. Let $D = (Q, Sigma, delta_D, q_0, F)$ be a DFA. Define $N = (Q, Sigma, delta_N, q_0, F)$ to be the equivalent NFA, where $delta_N$ is defined by "if $delta_D (q, a) = p$, then $delta_N (q, a) = {p}$ We can induct on $|w|$, that $ dh_N (q_0, w) = {dh_D (q_0, w)} $ *Basis: * $ (w = es)$ $ dh_N (q_0, es) = {q_0} = {dh_D (q_0, w)} $ *Inductive Step*: Let $w$ be of length $n+1$, $w = x a$, where $a$ is final symbol of $w$. $ dh_N (q_0, x a) &= union.big_(p in dh_N (q_0, x)) delta_N (p, a) && ("defn of" dh_N) \ &= union.big_(p in dh_d ({q_0}, x)) {delta_D (p, a)} && ("I.H")\ &= {dh_D (q_0, x a)} &&("def of "dh_D) $ ] = Reference - Alphabet - Finite Non empty set of symbols, denoted by $Sigma$ - Powers of Alphabet - $Sigma^2 = {00, 01, 10, 11}$ - $Sigma^0 = {epsilon.alt}$ - $Sigma^ lt.eq 2 = Sigma^0 union Sigma^1 union Sigma^2$ - String - Sequence of Symbols from $Sigma$, denoted by $epsilon.alt$ - Concat - $x = 00, y = 10, x dot y = x y = 0010$ - Substring - $a b$ is a substring of $b a b a a$, $b b$ is not. - Subsequence - $b b a$ is not subseq of $b a b a a $, $a b b$ is - Length of String - Number of symbols in a string, denoted by $|w|$ - Language - Set of strings over an alphabet - $L = {00, 11, 01, 110}$ - $L = emptyset$ - $L_1 dot L_2 = L_1L_2 = {x y : x in L_1, y in L_2}$ - $L^* = {x_1x_2...x_n : x_1, x_2, ... x_n in L, n in NN}$ - $L^+ = {x_1x_2...x_n : x_1, x_2, ... x_n in L, n gt.eq 1}$ === Proof Structure ==== Set A = Set B 1. Show $A subset.eq B$ - Take *arbitrary* element $x in A$ - Use definition of A to show $x in B$ - Therefore, $A subset.eq B$ 2. Show $B subset.eq A$ - Take *arbitrary* element $x in B$ - Use definition of B to show $x in A$ - Therefore, $B subset.eq A$ 3. Since $A subset.eq B$ and $B subset.eq A$, $A = B$