$$
\begin{aligned}
- i For a clear commit history, I rebase my repository regularly. Therefore, if you have forked the repository before, consider re-forking it again. Licensed under the MIT License. Describe your own real-world example that requires sorting. Describe one that requires finding the shortest distance between two points. Other than speed, what other measures of efficiency might you need to consider in a real-world setting? Memory efficiency and coding efficiency. Select a data structure that you have seen, and discuss its strengths and limitations. arrays: How are the shortest-path and traveling-salesperson problems given above similar?How are they different? Suggest a real-world problem in which only the best solution will do. Then come up with one in which \"approximately\" the best solution is good enough. Describe a real-world problem in which sometimes the entire input is available before you need to solve the problem, but other times the input is not entirely available in advance and arrives over time. In team work: Give an example of an application that requires algorithmic content at the application level, and discuss the function of the algorithms involved. Suppose we are comparing implementations of insertion sort and merge sort on the same machine. For inputs of size $n$ , insertion sort runs in $8n^2$ steps, while merge sort runs in $64n\\lg n$ steps. For which values of $n$ does insertion sort beat merge sort? $$ \\begin{aligned} 8n^2 < 64n \\lg n \\cr 0 < n < 8 \\lg n \\cr 1< 2^n < n^8 \\cr 2 \\leq n \\leq 43 \\cr \\end{aligned} $$ What is the smallest value of n such that an algorithm whose running time is $100n^2$ runs faster than an algorithm whose running time is $2^n$ on the same machine? $$ \\begin{aligned} 100n^2 < 2^n \\cr n \\ge 15.\\cr \\end{aligned} $$ For each function $f(n)$ and time $t$ in the following table, determine the largest size $n$ of a problem that can be solved in time $t$, assuming that the algorithm to solve the problem takes $f(n)$ microseconds. Using Figure 2.2 as a model, illustrate the operation of $\\text{INSERTION-SORT}$ on the array $A = \\langle 31, 41, 59, 26, 41, 58 \\rangle$. as shown in the figure above, the array A change as follow $$ \\begin{aligned} A = \\langle 31, 41, 59, 26, 41, 58 \\rangle\\cr A = \\langle 31, 41, 59, 26, 41, 58 \\rangle\\cr A = \\langle 26, 31, 41, 59, 41, 58 \\rangle\\cr A = \\langle 26, 31, 41, 41, 59, 58 \\rangle\\cr A = \\langle 26, 31, 41, 41, 58, 59 \\rangle\\cr A = \\langle 26, 31, 41, 41, 58, 59 \\rangle\\cr \\end{aligned} $$ Consider the procedure SUM-ARRAY on the facing page. It computes the sum of the n numbers in array $A[1:n]$. State a loop invariant for this procedure, and use its initialization, maintenance, and termination properties to show that the SUMARRAY procedure returns the sum of the numbers in $A[1:n]$. Rewrite the INSERTION-SORT procedure to sort into monotonically decreasing instead of monotonically increasing order. Consider the searching problem: Input: A sequence of n numbers $\\langle a1 , a2,..., an \\rangle$stored in array A[1:n] and a value x. Output: An index i such that x equals $A[i]$ or the special value NIL if x does not appear in $A$. Write pseudocode for linear search, which scans through the array from beginning to end, looking for x. Using a loop invariant, prove that your algorithm is correct. Make sure that your loop invariant fulfills the three necessary properties. Consider the problem of adding two n-bit binary integers $a$ and $b$, stored in two $n$-element arrays $A[0:n-1]$ and $B[0:n-1]$, where each element is either $0$ or $1$. $a = \\sum_{i=0}^{n-1} A[i] \\cdot 2^i$, and $b = \\sum_{i=0}^{n-1} B[i] \\cdot 2^i$. The sum $c = a + b$ of the two integers should be stored in binary form in an ($n+1$)-element array $C[0:n]$, where $c = \\sum_{i=0}^{n} C[i] \\cdot 2^i$. Write a procedure ADD-BINARY-INTEGERS that takes as input arrays $A$ and $B$, along with the length n, and returns array C holding the sum. Express the function $n^3/1000+100n^2-100n+3$ in terms of $\\Theta$-notaion. $\\Theta (n^3)$ Consider sorting $n$ numbers stored in array $A[1:n]$ by first finding the smallest element of $A[1:n]$ and exchanging it with the element in $A[1]$. Then find the smallest element of $A[2:n]$, and exchange it with $A[2]$. Then find the smallest element of $A[3:n]$, and exchange it with $A[3]$. Continue in this manner for the first $n-1$ elements of $A$. Write pseudocode for this algorithm, which is known as selection sort. What loop invariant does this algorithm maintain? Why does it need to run for only the first $n-1$ elements, rather than for all $n$ elements? Give the worst-case running time of selection sort in $\\Theta$-notation. Is the best-case running time any better? Consider linear search again (see Exercise 2.1-4). How many elements of the input array need to be checked on the average, assuming that the element being searched for is equally likely to be any element in the array? How about in the worst case? Using $\\Theta$-notation, give the average-case and worst-case running times of linear search. Justify your answers. How can you modify any sorting algorithm to have a good best-case running time modidy combine with insertion sort ,such as check whether each element in array is smaller(bigger) than the righthand element, since it produce a best - case running time of $\\Theta(n)$, the same as travel throught the array, which is a neccessary step for sorting. Using Figure 2.4 as a model, illustrate the operation of merge sort on an array initially containing the sequence $\\langle 3, 41, 52, 26, 38, 57, 9, 49 \\rangle$. $$ [3|41|52|26|38|57|9|49]$$ $$\\downarrow$$ $$[3|41|52|26] \\quad [38|57|9|49]$$ $$\\downarrow$$ $$[3|41] \\quad [52|26] \\quad [38|57] \\quad [9|49]$$ $$\\downarrow$$ $$[3] \\quad [41] \\quad [52] \\quad [26] \\quad [38] \\quad [57] \\quad [9] \\quad [49]$$ $$\\downarrow$$ $$[3|41] \\quad [26|52] \\quad [38|57] \\quad [9|49]$$ $$\\downarrow$$ $$[3|26|41|52] \\quad [9|38|49|57]$$ $$\\downarrow$$ $$[3|9|26|38|41|49|52|57]$$ The test in line 1 of the MERGE-SORT procedure reads \"if $p \\geq r$\" rather than \"if $p \\neq r$.\" If MERGE-SORT is called with $p > r$, then the subarray $A[p:r]$ is empty. Argue that as long as the initial call of MERGE-SORT(A, 1, n) has $n \\geq 1$, the test \"if $p \\neq r$\" suffices to ensure that no recursive call has $p > r$. $$ \\begin{aligned} if \\quad p \\leq r\\cr if \\quad p < r\\cr q = \\lfloor (p+r)/2 \\rfloor\\cr q \\geq p\\cr q+1 \\leq r\\cr MERGE-SORT(A,p,q)\\cr MERGE-SORT(A,q+1,r)\\cr \\text{else return}\\cr \\text{no recursive call has p>r}\\cr p \\leq r \\text{ holds in new recursive call}\\cr \\end{aligned} $$ $$ \\begin{aligned} MERGE-SORT(A,1,n)\\cr n \\geq 1\\cr p \\leq r holds\\cr \\end{aligned} $$ \"if $p \\neq r$\" suffices to ensure that no recursive call has $p > r$. State a loop invariant for the while loop of lines 12\u201318 of the MERGE procedure. Show how to use it, along with the while loops of lines 20\u201323 and 24\u201327, to prove that the MERGE procedure is correct. loop invariant: at the start of each iteration, $A[p:k-1]$ consist of the smallest elements of union of $L$ and $R$ at sort order. when the while loop of lines 12-18 finish $A[p:k-1]$ consist of the smallest elements of union of $L$ and $R$ at sort order, either while loop of lines 20\u201323 or 24\u201327 will be run, $L$ or $R$ has the biggest rest elements in sort order.when the second while loop finish, they are added to the end of A, and A consist of all elements at sort order. Use mathematical induction to show that when $n \\geq 2$ is an exact power of 2, the solution of the recurrence $$ \\begin{aligned} T(n) = \\begin{cases} 2 & \\text{if } n = 2, \\cr 2T(n/2) + n & \\text{if } n > 2 \\end{cases} \\end{aligned} $$ is $T(n) = n \\lg n$. $$ \\begin{aligned} T(n) & = T(2^t)\\cr & = 2T(2^{t-1}) + 2^t\\cr & = 4T(2^{t-2}) + 2\\cdot{2^t}\\cr & = 2^{t-1}T(2) + 2^t\\cdot{\\lg 2^t }\\cr & = n\\lg n \\end{aligned} $$ You can also think of insertion sort as a recursive algorithm. In order to sort $A[1:n]$, recursively sort the subarray $A[1:n-1]$ and then insert $A[n]$ into the sorted subarray $A[1:n-1]$. Write pseudocode for this recursive version of insertion sort. Give a recurrence for its worst-case running time. Referring back to the searching problem (see Exercise 2.1-4), observe that if the subarray being searched is already sorted, the searching algorithm can check the midpoint of the subarray against v and eliminate half of the subarray from further consideration. The binary search algorithm repeats this procedure, halving the size of the remaining portion of the subarray each time. Write pseudocode, either iterative or recursive, for binary search. Argue that the worst-case running time of binary search is $\\Theta (\\lg n)$. iterative: recursive: the worst-case is no x in $A$, BINARY SEARCH will end al condition l>r which occurs when A[l:r] is empty. since BINARY SEARCH halving the size of A[l:r] each time, it cost $\\Theta (\\lg n)$ to make A[l:r] empty The while loop of lines 5-7 of the INSERTION-SORT procedure in Section 2.1 uses a linear search to scan (backward) through the sorted subarray $A[1:j-1]$. What if insertion sort used a binary search (see Exercise 2.3-6) instead of a linear search? Would that improve the overall worst-case running time of insertion sort to $\\Theta(n \\lg n)$ It can save search time, but can not save movement time. At worst -case, it cost $j-1 = \\Theta(j)$ time to move elements bigger than $A[j]$ in $A[1:j-1]$.The same as the original INSERTION-SORT Therefore, that improve the overall worst-case running time of insertion sort to $\\Theta(n\\lg n)$ Dscribe Describe an algorithm that, given a set $S$ of $n$ integers and another integer $x$, determines whether $S$ contains two elements that sum to exactly $x$. Your algorithm should take $\\Theta (n \\lg n)$ time in the worst case. The time complexity of the algorithm is $\\Theta (n \\lg n)+\\Theta (n)$ Although merge sort runs in $\\Theta(n \\lg n)$ worst-case time and insertion sort runs in $\\Theta (n^2)$ worst-case time, the constant factors in insertion sort can make it faster in practice for small problem sizes on many machines. Thus it makes sense to coarsen the leaves of the recursion by using insertion sort within merge sort when subproblems become sufficiently small. Consider a modification to merge sort in which $n/k$ sublists of length $k$ are sorted using insertion sort and then merged using the standard merging mechanism, where $k$ is a value to be determined. a. Show that insertion sort can sort the $n/k$ sublists, each of length k, in $\\Theta (nk)$ worst-case time. b. Show how to merge the sublists in $\\Theta (n \\lg (n/k))$ worst-case time. c. Given that the modified alogotithm runs in $\\Theta (nk+n \\lg (n/k))$ worst-case time, what is the largest value of k as a function of n for which the modified algorithm has the same running time as standard merge sort, in terms of $\\Theta$-notation? d. How should you choose k in practice? Each sublists of length can be sorted in $\\Theta (k^2)$, there are n/k sublists, so in n $n/k \\cdot \\Theta(k^2) = \\Theta (nk)$ sorst-case time. Each level of merging need to merge $n$ elements, and there are $\\lg (n/k)$levels(since we merge each two sublists in one in each level). So the worse-case time to merge the sublists is $\\Theta (n \\lg (n/k))$. $$ \\begin{aligned} & if \\quad \\Theta (nk+n \\lg (n/k)) = \\Theta (n \\lg n)\\cr & \\exist \\quad c_{1} , c_{2} , n_{0} \\quad satisfies:\\cr & c_{1}(n \\lg n) \\leq nk+n \\lg (n/k) \\leq c_{2}(n \\lg n) \\quad for \\quad n > n_{0}\\cr &(c_{1}-1)(\\lg n )\\leq k - \\lg k \\leq (c_{2}-1)(\\lg n ) \\quad for \\quad n > n_{0}\\cr & since \\quad lg k \\llless k, \\quad \\text{we have } k = \\Theta (\\lg n)\\cr \\end{aligned} $$ Choose k be the largest size of the sublists that insertion-sort is faster than merge-sort. Bubblesort is a popular, but inefficient, sorting algorithm. It works by repeatedly swapping adjacent elements that are out of order. The procedure BUBBLESORT sorts array $A[1:n]$. a. Let $A'$ denote the array $A$ after BUBBLESORT> (A, n) is executed. To prove that it terminates and that $$ A'[1] \\leq A'[2] \\leq \\cdots \\leq A'[n] $$ In order to show that BUBBLESORT actually sorts, what else do you need to prove? The next two parts prove inequality (2.5). b. State precisely a loop invariant for the for loop in lines 2-4, and prove that this loop invariant holds. Your proof should use the structure of the loop-nvariant proof presented in this chapter. c. Using the termination condition of the loop invariant proved in part (b), state a loop invariant for the for loop in lines 1-4 that allows you to prove inequal ity (2.5). Your proof should use the structure of the loop-invariant proof presented in this chapter. d. What is the worst-case running time of BUBBLESORT? How does it compare with the running time of INSERTION-SORT? $A'$ consist of all the elements of $A$ loop invariant: At the start of each iteration of $for$ loop in lines 2-4, $A[j:n]$ consist of the elements originally in $A[j:n]$ before entering the loop but possibly in a different order, and $A[j]$ is the smallest one of them. Initialization: $j=n$, $A[n]$ consist of the elements originally in A$[n]$ and it is the smallest. Maintenance: During the iteration, if$A[j-1]<A[j]$, we swap them. So A[j-1] will be the smallest and A[j-1:n] consist of the elements originally in $A[j-1:n]$. Decresementing j preserves the loop invariant. Termination: the loop termination when $j = i$, $A[i:n]$ consist of the elements originally in $A[i:n]$ before entering the loop but possibly in a different order, and $A[i]$ is the smallest one of them. Loop invariant: At the start of each iteration of the for loop of lines 1-4, the subarray $A[1..i \u2212 1]$ consists of the $i - 1$ smallest elements in $A[1..n]$ in sorted order. $A[i..n]$ consists of the $n - i + 1$ remaining elements in $A[1..n]$. Initialization: Initially the subarray $A[1..i \u2212 1]$ is empty and trivially this is the smallest element of the subarray. Maintenance: From part (b), after the execution of the inner loop, $A[i]$ will be the smallest element of the subarray $A[i..n]$. And in the beginning of the outer loop, $A[1..i \u2212 1]$ consists of elements that are smaller than the elements of $A[i..n]$, in sorted order. So, after the execution of the outer loop, subarray $A[1..i]$ will consists of elements that are smaller than the elements of $A[i + 1..n]$, in sorted order. Termination: The loop terminates when $i = A.length$. At that point the array $A[1..n]$ will consists of all elements in sorted order. The $i$th iteration of the for loop of lines 1-4 will cause $n \u2212 i$ iterations of the for loop of lines 2-4, each with constant time execution, so the worst-case running time of bubble sort is $\\Theta(n^2)$ which is same as the worst-case running time of insertion sort. You are given the coefficents $a_0; a_1; a_2; \\dots; a_n$ of a polynomial $$ \\begin{aligned} P(x) & = \\sum_{k=0}^n a_kx^k \\cr &=a_0+a_1x+a_2x^2+\\cdots +a_{n-1}x^{n-1}+a_nx^n, \\end{aligned} $$ and you want to evaluate this polynomial for a given value of x. Horner's rule says to evalusate the polynomial according to this > parenthesization: $$ P(x) = a_0 + x ( a_1 + x ( a_2 + \\cdots + x ( a_{n-1} + x a_n) > \\cdots )). $$ The procedure HORNER implements Horner**\u2019s rule to > evaluate $P(x)$, > givem the coefficients $a_0,a_1,a_2,\\dots,> a_n$ in an array $A[0:n]$ and > the value of $x$. a. In terms of $\\Theta$-notation, what is the tunning time of this procedure? b. Write pseudocode to implement the naive polynomial-evaluation algorithm that computes each term of the polynomial from scratch. What is the running time of this algorithm? How does it compare with HORNER? c. Consider the following loop invariant for the procedure HORNER: At the start of each iteration of the for loop of lines 2-3, $$ p = \\sum _{k=0}^{n-(i+1)} A[k+i+1] \\cdot x^k $$ Interpret a summation with no terms at equaling 0. Following the structure of th loop-invariant proof presented in this chapter, use this loop invariant to show that, at termination, $p = \\sum _{k=0} ^{n} A[k] \\cdot x^k$ $\\Theta(n)$ pseudocode: $\\text{running time:} \\Theta (n^2)$, HORNER beat it asymptotically. Initialization: $i=n, p=\\sum_{k=0}^{n-(n+1)} A[k+n+1] \\cdot{x^k} = 0$, loop invariant holds. Maintenance: in the end of each iteration: $$ \\begin{aligned} p & = A[i] + x \\cdot{\\sum_{k=0}^{n-(i+1)} A[k+i+1] \\cdot {x^k}} \\cr & = A[i] + \\sum_{k=0}^{n-(i+1)} A[k+i+1] \\cdot{x^{k+1}}\\cr & = A[i] \\cdot{x^0} + \\sum_{k=1}^{n-i} A[k+i] \\cdot{x^{k}}\\cr & = \\sum_{k=0}^{n-i} A[k+i] \\cdot{x^k}\\cr \\end{aligned} $$ Decrementing i preserves the loop invariant. Termination: The loop terminates at $i=\u22121$. If we substitute, $$ p = \\sum_{k=0}^{n-(i+1)} A[k+i+1] \\cdot{x^k} = \\sum_{k=0}^{n} A[k] \\cdot{x^k} $$ Let $A[1:n]$ be an array of n distinct numbers. If $i < j$ and $A[i] > A[j]$, then the pair $(i,j)$ is called an inversion of A. a. List the five inversions of the array $\\langle 2,3,8,6,1 \\rangle$. b. What array with elements from the set $\\langle 1,2,\\dots ,n \\rangle$. has the most inversions? How many does it have? c. What is the relationship between the running time of insertion sort and the number of inversions in the input array? Justify your answer. d. Give an algorithm that determines the number of inversions in any permutation. on n elementa in $\\Theta (n \\lg n)$ worse-case time. (Hint: Modify merge sort.) a. (1,5),(2,5),(3,4),(3,5),(4,5) b. $\\langle n,n-1,\\dots ,1\\rangle \\text{ totally has } {n \\choose 2} = n(n-1)/2 \\text{ inversions}$ c. The running time of insertsion sort is a constant times the number of inversions, since each time exchanging elements in insertion sort reduces Inversions num by 1. d. Modify the lower-bound argument for insertion sort to handle input sizes that are not necessarily a multiple of 3. At least $\\lfloor \\frac n 3 \\rfloor \\geq \\frac 3n-1$ elements have to pass through at least $\\lfloor \\frac n 3 \\rfloor \\geq \\frac 3n-1$ the time taken by INSERTION-SORT in the worst case is at least proportional to $(\\frac 3n-1)(\\frac 3n-1)=\\Omega(n^2)$ Using reasoning similar to what we used for insertion sort, analyze the running time of the selection sort algorithm from Exercise 2.2-2. Inner loop in selection sort iterates $(n-i+1)$ times, for i = 1 to n-1, so the running time of selection sort is $\\sum_{i=1}^{n-1} (n-i+1)=(n+2)(n-1)/2=\\Theta(n^2)$ Suppose that $\\alpha$ is a fraction in the range $0 < \\alpha < 1$. Show how to generalize the lower-bound argument for insertion sort to consider an input in which the $\\alpha n$ largest values start in the first $\\alpha n$ positions. What additional restriction do you need to put on $\\alpha$? What value of $\\alpha$ maximizes the number of times that the $\\alpha n$ largest values must pass through each of the middle $(1-2 \\alpha)$ array positions? At least $\\alpha n$ values have to pass through at least $(1-2 \\alpha)n$ times. insertion sort is at least proportional to $(\\alpha n)(1-2 \\alpha)n = \\alpha(1-2\\alpha)n^2 = \\Omega(n^2)$ if $\\alpha(1-2\\alpha)=\\Omega(1)$ $\\max(\\alpha(1-2\\alpha))=\\frac 1 8$ when $\\alpha = \\frac 1 4$ Let $f(n)$ and $g(n)$ be the asympotically nonnegative functions. Using the basic definition of $\\Theta$-notation, prove that max ${f(n),g(n)} = \\Theta (f(n) + g(n))$. $$ \\begin{aligned} \\text{Let } n_0 \\text{ be a value that makes f(n) greater than 0 for n >}n_0\\cr \\text{Then for }n>n_0\\cr \\frac 1 2 (f(n)+g(n)) \\leq \\max(f(n),g(n)) \\leq 1(f(n)+g(n))\\cr \\max(f(n),g(n))=\\Theta(f(n)+g(n))\\cr \\end{aligned} $$ Tips: I will omit details like \"$\\text{Let } n_0 \\text{ be a value that makes f(n) greater than 0 for n >}n_0$\"\"$\\exist c,n_0$\" and \"for $n > n_0$\" for convenience from here. Explain why the statement, \"The running time of algorithm A is at least $O(n^2)$,\"if meaningless. $A = O(n^2)$ means $\\exist n_0,c:A \\leq cn^2$ for $n>n_0$. $A$ is at least $O(n)$ means $A \\geq \\text{a function that }\\leq cn^2$ You can say A can be any function or no function. Both are meaningless. Is $2^{n+1}=O(2^n)$? Is $2^{2n}=O(2^n)$? Prove Theorem 3.1. $$ \\begin{aligned} & f(n) = O(g(n)) \\quad and \\quad f(n)=\\Omega(g(n))\\cr & \\implies c_1 g(n)\\leq f(n) \\leq c_2g(n)\\cr & \\implies f(n) = \\Theta(g(n))\\cr & f(n) = \\Theta(g(n))\\cr & \\implies c_1 g(n)\\leq f(n) \\leq c_2g(n)\\cr & \\implies f(n) = O(g(n)) \\quad and \\quad f(n)=\\Omega(g(n))\\cr \\end{aligned} $$ We have $f(n) = \\Theta(g(n))$ if and only if $f(n) = \\Omega(g(n))$ and $f(n) = O(g(n))$ Prove that the running time of an algorithm is $\\Theta(g(n))$ if and only if its worst-case running time $O(g(n))$ and its best-casse running time is $\\Omega (g(n))$. $$ \\begin{aligned} A=\\Theta(g(n)) & \\implies c_1 g(n) \\leq A \\leq c_2 g(n) \\text{ for all cases}\\cr & \\implies \\text{worst-case running time}= O(n)\\cr & \\And \\text{best-case running time}=\\Omega(g(n))\\cr \\text{worst-case running time}= O(n) & \\implies A \\leq c_2 g(n) (1)\\cr \\text{best-case running time}=\\Omega(g(n)) & \\implies c_1 g(n) \\leq A (2)\\cr (1) \\And (2) & \\implies A=\\Theta(g(n))\\cr \\end{aligned} $$ Prove that $o(g(n)) \\cap \\omega (g(n))$ is empty set. $f(n)c(g(n)) \\implies NIL$"},{"location":"Chap03/3.2/#32-7","title":"3.2-7","text":" We can extend our notation to the case of two parameters n and m that can go to $\\infty$ independently at different rates. For a given function $g(n,m)$, we denote by $O(g(n,m))$ the set of functions $$ \\begin{aligned} O(g(n, m)) =\\lbrace f(n, m): & \\text{ there exist positive constants } c, n_0, \\text{ and } m_0 \\cr & \\text{ such that } 0 \\leq f(n, m) \\leq cg(n, m)\\cr & \\text{ for all } n \\geq n_0 \\text{ or } m \\geq m_0.\\rbrace\\cr \\end{aligned} $$ Give corresponding definitionss for $\\Omega(g(n,m))$ and $\\Theta(g(n,m))$. $$ \\begin{aligned} \\Theta(g(n, m)) =\\lbrace f(n, m): & \\text{ there exist positive constants } c_1,c_2, n_0, \\text{ and } m_0 \\cr & \\text{ such that } 0 \\leq c_1g(n,m)\\leq f(n, m) \\leq c_2g(n, m)\\cr & \\text{ for all } n \\geq n_0 \\text{ or } m \\geq m_0.\\rbrace\\cr \\Omega(g(n, m)) =\\lbrace f(n, m): & \\text{ there exist positive constants } c, n_0, \\text{ and } m_0 \\cr & \\text{ such that } 0 \\leq cg(n,m)\\leq f(n, m)\\cr & \\text{ for all } n \\geq n_0 \\text{ or } m \\geq m_0.\\rbrace\\cr \\end{aligned} $$ Show that if $f(n)$ and $g(n)$ are monotonically increasing functions, then so are the functions $f(n)+g(n)$ and $f(g(n))$, and if $f(n)$ and $g(n)$ are in addition nonnegative, then $f(n)\\cdot g(n)$ $$ \\begin{aligned} \\forall n_1 \\geq n_2: & f(n_1) \\geq f(n_2) \\And g(n_1) \\geq g(n_2)\\cr \\implies & f(n_1)+g(n_1) \\geq f(n_2)+g(n_2)\\cr \\implies & f(g(n_1) \\geq g(n_2)) \\geq f(g(n_2))\\cr & if \\quad f(n) \\geq 0 \\And g(n) \\geq 0\\cr \\implies & f(g(n_1)\\geq g(n_2)\\geq 0)\\geq f(g(n_2)) \\end{aligned} $$ Prove that $\\lfloor \\alpha n \\rfloor +\\lceil (1-\\alpha )n \\rceil = n$ for any integer n and real number $\\alpha$ in the range $0 \\leq \\alpha \\leq 1$ $$ \\begin{aligned} \\lfloor \\alpha n \\rfloor +\\lceil (1-\\alpha )n \\rceil & = \\lfloor \\alpha n \\rfloor +\\lceil -\\alpha n\\rceil + n\\cr &=\\lfloor \\alpha n \\rfloor - \\lfloor \\alpha n \\rfloor +n\\cr &=n \\end{aligned} $$ Use equation (3.14) or other means to show that $(n+o(n))^k = \\Theta(n^k)$ for any real constant k. Conclude that $\\lceil n \\rceil ^k$ and $\\lfloor n \\rfloor ^k = \\Theta(n^k)$. $$ \\begin{aligned} n^k\\leq(n+o(n))^k=n^k(1+\\frac {o(n)}n)^k \\leq n^k e^{\\frac {o(n)}n} \\leq en^k\\cr \\implies (n+o(n))^k = \\Theta(n^k)\\cr \\lceil n \\rceil ^k= (n+o(n))^k=\\Theta(n^k)\\cr \\lfloor n \\rfloor ^k = (n+o(n))^k=\\Theta(n^k)\\cr \\end{aligned} $$ Prove the following a. Equation(3.21). b. Equations(3.26)-(3.28). c. $\\lg(\\Theta(n))=\\Theta(\\lg n)$ $$ \\begin{aligned} n! = \\sqrt {2\\pi n}(\\frac n e)^n(1+\\Theta(\\frac{1}{n}))\\cr \\forall c>0,\\lim_{n \\to \\infin} \\frac {n!=\\frac{\\sqrt {2\\pi n}} {e^n}(1+\\Theta(\\frac 1 n))n^n} {cn^n} = \\lim_{n \\to \\infin} \\frac{\\sqrt {2\\pi n}} {e^n}(1+\\Theta(\\frac 1 n)) = 0\\cr \\implies n!=o(n^n)\\text{ (Equation(3.21))}\\cr \\lim_{n \\to \\infin} \\frac {n!=\\frac{\\sqrt {2\\pi n}} {e^n}(1+\\Theta(\\frac 1 n))n^n} {2^n} = \\lim_{n \\to \\infin }\\sqrt{2\\pi n}(1+\\Theta(\\frac 1 n))(\\frac{n}{2e})^n = \\infin \\cr \\implies n! =\\omega(n) \\text{ (Equation(3.27))}\\cr \\lg (n!)=n\\lg n+ \\frac{1}{2}\\lg(2\\pi n)+n\\lg e+\\lg(1+\\Theta(\\frac{1}{n})) = \\Theta(n\\lg n)\\text{ (Equation(3.28))}\\cr \\end{aligned} $$ $$ \\begin{aligned} \\forall f(n) \\in \\Theta(n)\\cr c_3\\lg n\\leq \\lg {c_1 n} \\leq \\lg(f(n))\\leq \\lg{c_2 n} \\leq c_4\\lg n\\cr \\implies \\lg(\\Theta(n))=\\Theta(\\lg n) \\end{aligned} $$ Is the function $\\lceil \\lg n \\rceil !$ polynomially bounded? Is the function $\\lceil \\lg \\lg n \\rceil !$ polynomially bounded? Polynomially bounded:$f(n) \\leq cn^k \\implies \\lg(f(n))=ck\\lg(n)\\implies \\lg(f(n))=O(\\lg n)$ $$ \\begin{aligned} \\lg(\\lceil \\lg n\\rceil !)& =\\Theta(\\lceil \\lg n\\rceil \\lg(\\lceil \\lg n\\rceil))\\cr &=\\Theta(\\lg n \\lg(\\lg n))\\cr &\\implies \\lceil \\lg n\\rceil !\\text{ is not polynomially bouded}\\cr \\lg \\lceil \\lg \\lg n \\rceil !&=\\Theta(\\lceil \\lg \\lg n\\rceil \\lg\\lceil\\lg\\lg n\\rceil)\\cr & =O(\\lg^2\\lg n)\\cr & =O(\\lg n)\\cr & \\implies\\lceil\\lg\\lg n\\rceil \\text{ is Polynomially bounded} \\end{aligned} $$ Which is asymptotically larger: $\\lg(\\lg^{\\ast}n)$ or $\\lg^{\\ast}(\\lg n)$? $$ \\begin{aligned} n=2^k\\cr \\lim_{n\\to\\infin}\\frac{\\lg(\\lg^{\\ast}n)}{\\lg^{\\ast}(\\lg n)}& = \\lim_{k\\to\\infin}\\frac{\\lg(\\lg^{\\ast} 2^k)}{\\lg^{\\ast}(\\lg 2^k)}\\cr &= \\lim_{k\\to\\infin} \\frac{\\lg(1+\\lg^{\\ast}k)}{\\lg^{\\ast}k}\\cr & =\\lim_{t\\to \\infin}\\frac{\\lg(1+t)}{t}\\cr & = 0 \\end{aligned} $$ $\\lg^{\\ast}(\\lg n)$ is larger. Show that the golden ratio $\\phi$ and its conjugate $\\hat{\\phi}$ both satisfy the equation $x^2 = x + 1$ $$ \\begin{aligned} ax^2+bx+c=0\\implies x=\\frac{-b\\pm\\sqrt{b^2-4ac}}{2a}\\cr x^2-x-1=0 \\implies x=\\frac{1 \\pm\\sqrt{5}}{2}=\\phi \\And \\hat\\phi \\end{aligned} $$ prove by induction that the i-th Fibonacci num satisfies the equation $$ F_i = (\\phi^i - \\hat{\\phi}^i)/\\sqrt{5} $$ where $\\phi$ is the golden ratio and $\\hat{\\phi}$ is its conjugate. For inductive case: $$ \\begin{aligned} & \\text{truth: }\\phi\\hat\\phi=-1,\\phi+\\hat\\phi=1\\cr \\text{assume }:\\cr F_{i-1} &= (\\phi^{i-1} - \\hat{\\phi}^{i-1})/\\sqrt{5}\\cr F_{i} &= (\\phi^{i} - \\hat{\\phi}^{i})/\\sqrt{5}\\cr \\text{then:}\\cr F_{i-1}+F_i & =(\\phi^{i-1} - \\hat{\\phi}^{i-1})/\\sqrt{5}+(\\phi+\\hat\\phi)(\\phi^i - \\hat{\\phi}^i)/\\sqrt{5}\\cr & = (\\phi^{i+1}-\\hat\\phi^{i+1})/\\sqrt 5+((1+\\phi\\hat\\phi)\\phi^{i-1}-(1+\\phi\\hat\\phi)\\hat\\phi^{i-1})/\\sqrt 5\\cr & =F_{i+1}\\cr \\end{aligned} $$ For base case: $$ \\begin{aligned} (\\phi^{1} - \\hat{\\phi}^{1})/\\sqrt{5}=1=F_1\\cr (\\phi^{2} - \\hat{\\phi}^{2})/\\sqrt{5}=(\\phi^{1}+1 - (\\hat{\\phi}^{1}+1))/\\sqrt{5}=1=F_2\\cr \\end{aligned} $$ Tips: I will omit details like\"assume $F_i$ $F_{i-1}$\" and proof of base case from now on for convenience. Show that $k\\lg k = \\Theta(n)$ implies $k = \\Theta(n/\\lg n)$. $$ \\begin{aligned} & k\\lg k=\\Theta(n)\\cr \\implies & n=\\Theta(k\\lg k)\\cr \\implies & \\lg n = \\Theta(\\lg k + \\lg\\lg k)=\\Theta(\\lg k)\\cr \\implies & n/\\lg n = \\Theta(k\\lg k)/\\Theta(\\lg k)=\\Theta (k)\\cr \\implies & k = \\Theta(n/\\lg n) \\end{aligned} $$ Let $$ P(n) = \\sum_{i=0}^d a_i n^i, $$ where $a_d > 0$, be a degree-d polynomial in n, and let k be a constant. Use the definitions of the asymptotic notations to prove the following properties. a. if $k \\geq d$, then $p(n)=O(n^k)$ b. if $k \\leq d$, then $p(n)=\\Omega(n^k)$ c. if $k =d$, then $p(n)=\\Theta(n^k)$ d. if $k > d$, then $p(n)=o(n^k)$ e. if $k < d$, then $p(n)=\\omega(n^k)$ To prove $p(n)=O(n^k)$, we only need to prove: $$ \\begin{aligned} \\sum_{i=0}^{d}a_in^i \\leq cn^k (\\text{ for }n \\geq n_0)\\cr \\end{aligned} $$ Which is implied by $$ \\begin{aligned} c \\geq \\sum_{i=0}^da_in^{i-k} (\\text{ for }n \\geq n_0)\\cr \\end{aligned} $$ since $\\sum_{i=0}^{d}a_i \\geq \\sum_{i=0}^da_in^{i-k} (\\text{ for }n \\geq 1)$ $\\text{choose } c = \\sum_{i=0}^{d}a_i$ and $n_0 = 1$ satisfies. We only need to prove: $$ \\sum_{i=0}^{d}a_in^i \\geq cn^k $$ which is implied by: $$ \\begin{aligned} c \\leq \\sum_{i=0}^da_in^{i-k}\\cr c=a_d \\leq \\sum_{i=0}^da_in^{i-k}\\cr n_0=1 \\end{aligned} $$ We only need to prove: $$ c_1n^d\\leq \\sum_{i=0}^{d}a_in^i \\leq c_2n^{d} $$ which is implied by: $$ \\begin{aligned} c_1 \\leq \\sum_{i=0}^da_in^{i-d} \\leq c_2\\cr c_1=a_d \\leq \\sum_{i=0}^da_in^{i-d} \\leq \\sum_{i=0}^{d} a_i=c_2\\cr n_0=1 \\end{aligned} $$ We only need to prove: $$ \\begin{aligned} \\forall c >0,\\exist n_0 \\text{ satisfies: }\\cr \\sum_{i=0}^{d} a_i n^i < cn^k (\\text{ for } n > n_0)\\cr \\impliedby c > \\sum_{i=0}^{d} a_in^{i-k}\\cr \\text{since }\\sum_{i=0}^{d} a_in^{d-k} \\geq \\sum_{i=0}^{d} a_in^{i-k}\\cr n_0=\\sqrt[d-k]{\\frac {c} {\\sum_{i=0}^d a_i}} \\end{aligned} $$ We only need to prove: $$ \\begin{aligned} \\forall c >0,\\exist n_0 \\text{ satisfies: }\\cr \\sum_{i=0}^{d} a_in^i > cn^k (\\text{ for }n>n_0)\\cr \\impliedby c < \\sum_{i=0}^{d} a_in^{i-k}\\cr \\text{since } a_d n^{d-k} \\leq \\sum_{i=0}^{d} a_in^{i-k}\\cr n_0=\\sqrt[d-k]{\\frac {c} {a_d}} \\end{aligned} $$ Indicate, for each pair of expressions $(A, B)$ in the table below whether A is $O, o,\\Omega,\\omega$, or $\\Theta$ of B. Assume that $k \\geq 1,\\epsilon >,$ and $c>1$ are constants. Write your answer in the foem of the table with \"yes\" or \"no\" written in each box. a. Rank the following fuctions vy oefer of growth. That is, find an arrangement $g_1,g_2,\\dots,g_{30}$ of the functions satisfying $g_1=\\Omega(g_2),g_2=\\Omega(g_3),\\dots ,g_{29}=\\Omega(g_{30})$. Partion your list into equivalence classes such that functions $f(n)$ and $g(n)$ belong to the same class if and only if $f(n)$ = $\\Theta g(n)$ $$ \\begin{array}{} & \\lg(\\lg^{\\ast}n) & 2^{\\lg^{\\ast}n} & (\\sqrt {2})^{\\lg n} & n^2 & n! & (\\lg n)!\\cr & (3/2)^n & n^3 & \\lg^2n & \\lg(n!) & 2^{2^n} & n^{1/\\lg n}\\cr & \\ln\\ln n & \\lg^{\\ast}n & n\\cdot 2^n & n^{\\lg\\lg n} & \\ln n & 1\\cr & 2^{\\lg n} & (\\lg n)^{\\lg n} & e^n & 4^{\\lg n} & (n+1)! & \\sqrt{\\lg n}\\cr & \\lg^{\\ast}(\\lg n) & 2^{\\sqrt{2 \\lg n}}& n & 2^n& n\\lg n & 2^{2^{n+1}}\\cr \\end{array} $$ b. Give an example of a single nonnegative function $f(n)$ such that for all functions $g_i(n)$ in part (a), $f(n)$ is neither $O(g_i(n))$ nor $\\Omega (g_i(n))$. $$ \\begin{aligned} 2^{2^{n+1}}\\cr 2^{2^n}\\cr (n+1)!\\cr n!\\cr e^n\\cr n\\cdot 2^n\\cr 2^n\\cr (3/2)^n\\cr n^{\\lg\\lg n} \\And (\\lg n)^{\\lg n}\\cr (\\lg n)!\\cr n^3\\cr n^2 \\And 4^{\\lg n} \\cr \\lg(n!) \\And n\\lg n\\cr 2^{\\lg n} \\And n\\cr (\\sqrt {2})^{\\lg n}\\cr 2^{\\sqrt{2\\lg n}}\\cr \\lg^2 n\\cr \\ln n\\cr \\sqrt{\\lg n}\\cr \\ln\\ln n\\cr 2^{\\lg^{\\ast}n}\\cr \\lg^{\\ast}n \\And \\lg^{\\ast}(\\lg n)\\cr \\lg(\\lg^{\\ast}n) \\cr n^{1/\\lg n}\\And 1\\cr \\end{aligned} $$ $|\\sin{n}|\\cdot 2^{2^{2^{n+1}}}$ Let $f(n)$ and $g(n)$ be asymptotically positive functions. Prove or disprove each of the following conjectures. a. $f(n)=O(g(n))$ implies $g(n)=O(f(n))$. b. $f(n)+g(n)=\\Theta(\\min(f(n),g(n)))$. c. $f(n)=O(g(n))$ implies $\\lg(f(n))=O(\\lg(g(n)))$, where $\\lg g(n)\\geq 1$ and $f(n)\\geq 1$ for all sufficiently large $n$. d. $f(n)=O(g(n))$ implies $2^{f(n)} = O(2^{g(n)})$. e. $f(n)=O((f(n))^2)$. f. $f(n)=O(g(n))$ implies $g(n)=\\Omega(f(n))$. g. $f(n)=\\Theta(f(n/2))$. h. $f(n)+o(f(n))=\\Theta(f(n))$. Disprove: $n=O(n^2)$ but $n^2\\neq O(n)$ Disprove:$n+n^2\\neq \\Theta(\\min(n,n^2))$ prove: $$ \\begin{aligned} 0\\leq f(n)\\leq cg(n) & \\implies \\lg(f(n))\\leq \\lg c+\\lg(g(n))\\cr \\text{to prove: }&\\lg(f(n)) \\leq d\\lg(g(n))\\cr \\text{only need to prove: }& \\lg c + \\lg(g(n)) \\leq d\\lg(g(n))\\cr \\text{choose } & \\quad d = 1+\\lg c \\text{ satisfies} \\end{aligned} $$ prove: $$ \\begin{aligned} 0\\leq f(n)\\leq cg(n) \\implies 2^{f(n)}\\leq 2^{cg(n)}=2^c\\cdot c^{g(n) } \\end{aligned} $$ disprove: $\\frac{1}{n}\\neq O(\\frac{1}{n^2})$ prove: $0\\leq f(n)\\leq cg(n) \\implies g(n)=\\Omega (f(n))$ disprove:$2^n\\neq\\Theta(2^{\\frac{n}{2}})$ prove: $f(n)\\leq f(n)+o(f(n))\\leq 2f(n)$ since $0\\leq o(f(n))< f(n)$ Let $f(n)$ and $g(n)$ be asymptotically positive functions. Prove the following identities: a. $\\Theta(\\Theta(f(n)))=\\Theta(f(n))$. b. $\\Theta(f(n))+O(f(n))=\\Theta(f(n))$. c. $\\Theta(f(n))+\\Theta(g(n)) =\\Theta(f(n)+g(n))$. d. $\\Theta(f(n))\\cdot \\Theta(g(n))=\\Theta(f(n)\\cdot g(n))$. e. Argue that for any real constants $a_1,b_1>0$ and integer constants $k_1,k_2$, the following asymptotic bound holds: $(a_1 n)^{k_1}\\lg^{k_2}(a_2 n)=\\Theta(n^{k_1}\\lg^{k_2}n)$. $\\star$ f. Prove that for $S\\subseteq \\Z$, we have $$ \\sum_{k\\in S}\\Theta(f(k))=\\Theta(\\sum_{k\\in S}f(k)), $$ assuming that both sums converge. $\\star$ g. Show that for $S \\subseteq \\Z$, the following asymptotic bound does not necessarily hold, even assuming that both products converge, by giving a counterexample: $$ \\prod_{k\\in S}\\Theta(f(k))=\\Theta(\\prod_{k\\in S}f(k)). $$ $$ \\begin{aligned} g(n)=\\Theta(f(n))\\cr \\implies c_1 f(n)\\leq g(n) \\leq c_2 f(n)\\cr h(n)=\\Theta(g(n))\\cr \\implies c_1 d_1 f(n)\\leq d_1 g(n)\\leq h(n)\\leq d_2g(n)\\leq c_2 d_2 f(n)\\cr \\implies h(n)=\\Theta(f(n))\\cr \\implies\\Theta(\\Theta(f(n)))=\\Theta(f(n))\\cr \\end{aligned} $$ $$ \\begin{aligned} \\forall g(n)=\\Theta(f(n)) \\And h(n)=\\Theta(f(n))\\cr \\implies c_2 f(n) \\leq g(n)\\leq c_1 f(n) \\And 0 \\leq h(n)\\leq df(n)\\cr \\implies c_1f(n)\\leq g(n)+h(n)\\leq (c_1+d)f(n)\\cr \\implies g(n)+h(n)=\\Theta(f(n))\\cr \\implies \\Theta(f(n))+O(f(n))=\\Theta(f(n))\\cr \\end{aligned} $$ $$ \\begin{aligned} \\forall A(n)=\\Theta(f(n))\\And B(n)=\\Theta(g(n))\\cr \\implies c_1 f(n)\\leq A(n)\\leq c_2 f(n)\\And d_1 f(n)\\leq B(n) \\leq d_2 f(n)\\cr \\implies \\min(c_1,d_1)(f(n)+g(n))\\leq A(n)+B(n)\\leq \\max(c_2+d_2)(f(n)+g(n))\\cr \\implies \\Theta(f(n))+\\Theta(g(n)) =\\Theta(f(n)+g(n)) \\end{aligned} $$ $$ \\begin{aligned} \\forall A(n)=\\Theta(f(n))\\And B(n)=\\Theta(g(n))\\cr \\implies c_1 f(n)\\leq A(n)\\leq c_2 f(n)\\And d_1 f(n)\\leq B(n) \\leq d_2 f(n)\\cr \\implies c_1 d_1f(n)\\cdot g(n)\\leq A(n)\\cdot B(n)\\leq c_2 d_2f(n)\\cdot g(n)\\cr \\implies \\Theta(f(n))\\cdot \\Theta(g(n)) =\\Theta(f(n)\\cdot g(n)) \\end{aligned} $$ $$ \\begin{aligned} \\text{we only need to prove: }c_1 n^{k_1}\\lg^{k_2}n\\leq (a_1 n)^{k_1}\\lg^{k_2}(a_2 n)\\leq c_2 n^{k_1}\\lg^{k_2}n\\cr \\text{which is implied by: }d_1 lg^{k_2}n\\leq\\lg^{k_2}(a_2 n)\\leq d_2\\lg^{k_2}n\\cr \\text{which is implied by: }e_1\\lg n\\leq \\lg a+\\lg n \\leq e_2 \\lg n\\cr \\text{which is implied by: }e_1\\leq \\lg a/\\lg n +1 \\leq e_2\\cr \\text{choose }e_1=1/2 \\quad e_2=2 \\quad n_0=\\lg max(a^2,2^{\\lg a}) \\text{ satisfies}\\cr \\end{aligned} $$ $$ \\begin{aligned} g(k)=\\Theta(f(k))\\cr \\implies \\sum_{k\\in S} c_kf(k)\\leq \\sum_{k\\in S} g(k)\\leq \\sum_{k\\in S} d_kf(k)\\cr \\implies \\min( c_k) \\sum_{k\\in S} f(k)\\leq \\sum_{k\\in S} g(k)\\leq \\max(d_k)\\sum_{k\\in S} f(k)\\cr \\implies \\sum_{k\\in S}\\Theta(f(k))=\\Theta(\\sum_{k\\in S}f(k))\\cr \\end{aligned} $$ $\\prod_{k\\in S} \\frac 1 2=\\prod_{k\\in S} \\Theta(\\frac{1}{2^n})=\\Theta(0)\\neq \\Theta(\\prod_{k\\in S} 1)=\\Theta(1)$ Some authors define $\\Omega$-notation in a slightly different way than this textbook does. We'll use the nomenclature $\\stackrel{\\infty}{\\Omega}$ (read \"omega infinity\") for this alternative definition. We say that f(n) = $\\stackrel{\\infty}{\\Omega}(g(n))$if there exists a positive constant $c$ such that $f(n) \\geq cg(n) \\geq 0$ for infinitely many integers n. a. Show that for any two asymptotically nonnegative functions $f(n)$ and $g(n)$, we have $f(n)=O(g(n))$ or $f(n)=\\stackrel{\\infty}{\\Omega}(g(n))$ (or both). b. Show that there exist two asymptotically nonnegative functions $f(n)$ and $g(n)$ for which neither $f(n) = O(g(n))$ nor $f(n) = \\Omega(g(n))$ holds. c. Describe the potential advantages and disadvantages of using $\\stackrel{\\infty}{\\Omega}$-notation instead of $\\Omega$-notation to characterize the running times of programs. d. Some authors also define $O$ in a slightly different manner. We\u2019ll use $O'$ for the alternative definition: $f(n)=O'(g(n))$ if and only if $|f(n)|=O(g(n))$. What happens to each direction of the \"if and only if\" in Theorem 3.1 on page 56 if we substitute $O'$ for $O$ but still use $\\Omega$? e Some authors define $\\tilde{O}$(read \"soft-oh\") to mean O with logarithmic factors ignored: $$ \\begin{aligned} \\tilde{O}(g(n))=\\lbrace f(n) : & \\text{ there exist positive constants }c, k, \\text{ and } n_0 \\text{ such that }\\cr & 0\\leq f(n)\\leq cg(n)\\lg^k(n) \\text{ for all } n\\geq n_0\\rbrace .\\cr \\end{aligned} $$ Define $\\tilde{\\Omega}$ and $\\tilde{\\Theta}$ in a similar manner. Prove the corresponding analog to Theorem 3.1. $$ \\begin{aligned} \\text{if } f(n)\\neq O(g(n))\\cr \\text{there must be infinite integers that }f(n) \\geq cg(n)\\geq 0\\cr \\text{otherwise choose the last } n = \\text{ that } f(n)\\geq cg(n) \\text{ as } n_0\\cr \\text{for all }n\\geq n_0,f(n)\\leq cg(n)\\implies f(n)=O(g(n))\\cr \\end{aligned} $$ $f(n)=n|\\sin n|,g(n)=1$ $$ \\begin{aligned} \\forall f(n) , g(n):\\cr f(n)=\\Theta(g(n)) \\implies f(n)=\\Omega(g(n)) \\And f(n)=O'(g(n))\\cr \\text{but the conversion is not true} \\end{aligned} $$ $$ \\begin{aligned} \\tilde\\Omega(g(n))=\\lbrace f(n): & \\text{ there exist positive constants }c,k,\\text{ and } n_0 \\text{ such that}\\cr & 0\\leq f(n) \\leq cg(n)\\lg^{k}(n) \\text{ for all }n \\geq n_0\\rbrace .\\cr \\tilde\\Theta(g(n))=\\lbrace f(n): & \\text{ there exist positive constants }c_1,c_2,k,\\text{ and } n_0 \\text{ such that}\\cr & 0\\leq c_1g(n)\\lg^{k}(n)\\leq f(n) \\leq c_2g(n)\\lg^{k}(n) \\text{ for all }n \\geq n_0\\rbrace .\\cr \\text{according to there definity: } & f(n)=\\tilde\\Theta(g(n))\\iff f(n)=\\tilde O(g(n)) \\And f(n)=\\tilde\\Omega (g(n))\\cr \\end{aligned} $$ We can apply the iteration operator * used in the $\\lg^{\\ast}$ function to any monotonically increasing function $f(n)$ over the real. For a given constant $c \\in \\R$, we define the iterated dunction $f_{c}^{\\ast}$ by $f_{c}^{\\ast}=\\min \\lbrace i\\geq0:f^{(i)}\\leq c\\rbrace$, which need not be well defined in all cases. In other words, the quantity $f_{c}^{\\ast}(n)$ is the minimum number of iterated applications of the function $f$ required to reduce its argument down to $c$ or less. For each of the functions $f(n)$ and constants $c$ in the table below, give as tight a bound as possible on $f_{c}^{\\ast}(n)$. If there is no $i$ such that $f^{(i)} \\leq c$, write \"undefined\" as your answer. Note: You may wish to read Section 4.5 before attempting some of these exercises. Generalize MATRIX-MULTIPLY-RECURSIVE to multiple $n\\times n$ matrices for which $n$ is not necessarily an exact power of 2. Give a recurrence describing its running time. Argue that it runs in $\\Theta(n^3)$ time in the worst case. running time = $\\Theta(n^3)+O(n^2)+O(1)+O(n)=\\Theta(n^3)$ How quickly can you multiply a $kn\\times n$ matrix (kn rows and n columns) by an $n\\times kn$ matrix, where $k\\geq 1$ , using MATRIX-MULTIPLY-RECURSIVE as a subtoutine? Answer the same question for multiplying an $n\\times kn$ matrix by a $kn\\times n$ matrix. Which is asymptotically faster, and by how much? Suppose that instead of partitioning matrices by index calculation in MATRIX-MULTIPLY-RECURSIVE, you copy the appropriate elements of A,B,and C into separate $n/2\\times n/2$ submarices $A_{11},A_{12},A_{21},A_{22};B_{11},B_{12},B_{21},B_{22}$; and $C_{11},C_{12},C_{21},C_{22}$, respectively. After the recursive calls, you copy the results from $C_{11},C_{12},C_{21}$, and $C_{22}$ back into the appropriate places in $C$. How does recurrence (4.9) change, and what is its solution? $T(n)=8T(n/2)+\\Theta(n^2)$ According to the master theorem, $T(n)=\\Theta(n^3)$ Write pseudocode for a divide-and-conquer algorithm MATRIX-ADD-RECURSIVE that sums two $n\\times n$ matrices $A$ and $B$ by partitioning each of them into four $n/2\\times n/2$ submatrices and then recursively summing corresponding pairs of submatrices. Assume that matrix partitioning uses $\\Theta(1)$-time index calculations. Write a recurrence for the worst-case running time of MATRIX-ADD-RECURSIVEE, and solve your recurrence. What happens if you use $\\Theta(n^2)$-time copying to implement the partitioning instead of index calculations? $T(n)=4T(n/2)+\\Theta(1)$ According to master theorem, $T(n)=\\Theta(n^2)$ if use copying,$T(n)=4T(n/2)+\\Theta(n^2)\\implies T(n)=\\Theta(n^2\\lg n)$ Note: You may wish to read Section 4.5 before attempting some of these exercises. Use Strassen\u2019s algorithm to compute the matrix product $$ \\begin{pmatrix} 1 & 3\\cr 7 & 5 \\end{pmatrix} \\begin{pmatrix} 6 & 8\\cr 4 & 2 \\end{pmatrix} $$ Show your work. $$ \\begin{aligned} S_1 & = B_{12} - B_{22} = 8 - 2 = 6\\cr S_2 & = A_{11} + A_{12} = 1 + 3 = 4\\cr S_3 & = A_{21} + A_{22} = 7 + 5 = 12\\cr S_4 & = B_{21} - B_{11} = 4 - 6 = -2\\cr S_5 & = A_{11} + A_{22} = 1 + 5 = 6\\cr S_6 & = B_{11} + B_{22} = 6 + 2 = 8\\cr S_7 & = A_{12} - A_{22} = 3 - 5 = -2\\cr S_8 & = B_{21} + B_{22} = 4 + 2 = 6\\cr S_9 & = A_{11} - A_{21} = 1 - 7 = -6\\cr S_{10} & = B_{11} + B_{12} = 6 + 8 = 14\\cr P_1 & = A_{11} \\cdot S_1 = 1 * 6 = 6\\cr P_2 & = S_2 \\cdot B_{22}=4 * 2 = 8\\cr P_3 & = S_3 \\cdot B_{11} = 12 * 6 = 72\\cr P_4 & = A_{22} \\cdot S_{4} = 5 * (-2) = -10\\cr P_5 & = S_5 \\cdot S_6 = 6 * 8 = 48\\cr P_6 & = S_7 \\cdot S_8 = -2 * 6 = -12\\cr P_7 & = S_9 \\cdot S_10 = -6 * 14 = -84\\cr C_{11} & = C_{11} + P_5 + P_4 - P_2 + P_6 = 48 + (-10) - 8 + (-12) = 18\\cr C_{12} & = C_{12} + P_1 + P_2 = 6 + 8 = 14\\cr C_{21} & = C_{21} + P_3 + P_4 = 72 + (-10) = 62\\cr C_{22} & =C_{22} + P_5 + P_1 - P_3 - P_7 = 48 + 6 - 72 - (-84) = 66\\cr C & = \\begin{pmatrix} 18 & 14\\cr 62 & 66\\cr \\end{pmatrix} \\end{aligned} $$ Write pseudocode for Strassen\u2019s algorithm. What is the largest $k$ such that if you can multiply $3\\times 3$ matrices using $k$ multiplications (not assuming commutativity of multiplication), then you can multiply $n\\times n$ matrices in $o(n^{\\lg 7})$ time? What is the running time of this algorithm? $$ \\begin{aligned} T(n)=kT(n/3)+O(1)=o(n^{\\lg 7})\\cr \\implies T(n)=\\Theta(n^{\\log_{3} k})=o(n^{\\lg 7})\\cr \\implies \\log_{3} k < \\lg 7\\cr \\implies k < 21.8\\cr \\implies \\max(k) = 21\\cr \\text{running time }= n^{\\log_{3} 21}\\cr \\end{aligned} $$ V. Pan discovered a way of multiplying $68\\times 68$ matrices using 132,464 multiplications, a way of multiplying $70\\times 70$ matrices using 143,640 multiplications, and a way of multiplying $72\\times 72$ matrices using 155,424 multiplications. Which method yields the best asymptotic running time when used in a divide-and-conquer matrix-multiplication algorithm? How does it compare with Strassen\u2019s algorithm? $n^{\\log_{68}132464}=n^{2.79513};n^{\\log_{70} 143640}=n^{2.79512};n^{\\log_{72} 155424}=n^{2.79515};n^{\\lg 7}=n^{2.80735}$ the second method is the best; better than Strassen's algorithm. Show how to multiply the complex numbers $a+bi$ and $c+di$ using only three multiplications of real numbers. The algorithm should take $a,b,c$, and $d$ as input and produce the real component $ac-bd$ and the imaginary component $ad+bc$ separately. $$ \\begin{aligned} S_1=a(c-d)=ac-ad\\cr S_2=b(c+d)=bc+bd\\cr S_3=d(a-b)=ad-bd\\cr S_1+S_3=ac-bd\\cr S_2+S_3=ad+bc\\cr \\end{aligned} $$ Suppose that you have a $\\Theta(n^{\\alpha})$-time algorithm for squaring $n \\times n$ matrices, where $\\alpha \\geq 2$. Show how to use that algorithm to multiply two different $n \\times n$ matrices in $\\Theta(n^{\\alpha})$ time. $$ \\begin{aligned} A\\cdot B=\\frac{(A+B)^{2}-(A-B)^2}{4} \\end{aligned} $$ Use the substitution method to show that each of the following recurrences defined on the reals has the asymptotic solution specified: a. $T(n)=T(n-1)+n$ has solution $T(n) = O(n^2)$ b. $T(n)=T(n/2)+\\Theta(1)$ has solution $T(n)=O(\\lg n)$ c. $T(n)=2T(n/2)+n$ has solution $T(n)=O(n\\lg n)$. d. $T(n)=2T(n/2+17)+n$ has solution $T(n) = O(n\\lg n)$. e. $T(n)=2T(n/3)+\\Theta(n)$ has solution $T(n)=\\Theta(n)$ f. $T(n) = 4T(n/2)+\\Theta(n)$ has solution $T(n)=\\Theta(n^2)$. a. $$ \\begin{aligned} \\text{ inductive case: } &\\cr & \\text{assume }T(n')\\leq cn'^2 \\text{ for all } n_0 \\leq n'\\leq n\\cr T(n)& =T(n-1)+n\\cr & \\leq c(n-1)^2+n\\cr & = cn^2+(1-2c)n+c\\cr & \\leq cn^2\\cr & \\text{while the last step holds for }c > \\frac{1}{2}\\cr \\text{base case: } &\\cr T(n) & \\leq c \\leq cn \\text{ for all }n \\leq n_0 \\text{ is true if and only if choose a large enough } n_0\\cr \\end{aligned} $$ b. assume $T(n) \\leq c\\lg n$ $$ \\begin{aligned} T(n)&=T(n/2)+c'\\cr &\\leq c\\lg n -c\\lg 2 +c'\\cr &\\leq c\\lg n\\cr \\end{aligned} $$ the last step holds for $c>c'/\\lg 2$. c. assume $T(n)\\leq cn\\lg n$ $$ \\begin{aligned} T(n) & = 2T(n/2)+n\\cr & \\leq 2c(n/2)\\lg (n/2) +n\\cr & = cn\\lg n + (1-c\\lg 2)n\\cr & \\leq cn\\lg n \\end{aligned} $$ the last step holds for $c>1/\\lg 2$. d. assume $T(n)\\leq c(n-a)\\lg (n-a)$ $$ \\begin{aligned} T(n)&=2T(n/2+17)+n\\cr & \\leq 2(c(n/2+17-a))\\lg(n/2-a+17)+n\\cr & = c(n-2a+34)\\lg(n -a +34)+(1-c\\lg 2)n +c(-2a+34)\\lg 2\\cr & \\leq c(n-a)\\lg (n-a)\\cr \\end{aligned} $$ the last step holds for $n\\geq 34$ e. assume $c_{1}n\\leq T(n)\\leq c_{2}n$ $$ \\begin{aligned} T(n) & =2T(n/3)+\\Theta(n)\\cr & \\geq 2(c_{1}n/3)+c_{3}n\\cr & =(2c_{1}/3+c_{3})n\\cr & \\geq c_{1}n\\cr \\end{aligned} $$ the last step holds for $c_{1}\\leq 3c_{3}$ $c_{3}$ is the lowwer bound constant in $\\Theta(n)$. $$ \\begin{aligned} T(n) & =2T(n/3)+c_{4}n\\cr & \\leq (2c_{2}/3+c_{4})n\\cr & \\leq c_{2}n\\cr \\end{aligned} $$ the last step holds for$c_{2}\\geq 3c_{4}$ $c_{4}$ is the upper bound constant in $\\Theta(n)$. f. assume $c_{1}n^2 \\leq T(n)\\leq c_{2}n^2-c_{0}n$ $$ \\begin{aligned} T(n) & =4T(n/2)+\\Theta(n)\\cr & \\geq c_{1}n^2+c_{3}n\\cr & \\geq c_{1}n^2\\cr T(n) & =4T(n/2)+\\Theta(n)\\cr & \\leq c_{2}n^2-2c_{0}n+c_{4}n\\cr & \\leq c_{2}n^2-c_{0}n\\cr \\end{aligned} $$ the last step holds for $c_{0}\\geq c_{4}$ $c_{3}$ is the lowwer bound constant in $\\Theta(n)$ and $c_{4}$ is the upper bound constant in $\\Theta(n)$. The solution to the recurrence $T(n) = 4T(n/2) + n$ turns out to be $T(n) = \\Theta(n^2)$.Show that a substitution proof with the assumption $T(n) \\leq cn^2$ fails. Then show how to subtract a lower-order term to make a substitution proof work. assume $T(n)\\leq cn^2$ $$ \\begin{aligned} T(n) & =4T(n/2)+n\\cr & \\leq cn^2+n\\cr \\end{aligned} $$ It can not further imply $T(n)\\leq cn^2+n$ assume $T(n)\\leq c_{1}n^2-c_{2}n$ $$ \\begin{aligned} T(n) & =4T(n/2)+n\\cr & \\leq c_{1}n^2-2c_{2}n+n\\cr & \\leq c_{1}n^2-c_{2}n\\cr \\end{aligned} $$ the last step holds for $c_{2}\\geq 1$ The recurrence $T(n)=2T(n-1)+1$ has the solution T(n) = O(2^n). Show that a subtitution proof fails with the assumption $T(n)\\leq c2^n$, where $c > 0$ is constant. Then show how to subtract a lower-order term to make a substitution proof work. assume $T(n)\\leq c2^{n}$ $$ \\begin{aligned} T(n) & =2T(n-1)+1\\cr & \\leq 2\\cdot c2^{n-1}+1\\cr & =c2^{n}+1\\cr \\end{aligned} $$ It can not further imply $T(n)\\leq c2^{n}$ assume $T(n)\\leq c2^{n}-d$ $$ \\begin{aligned} T(n) & =2T(n-1)+1\\cr & \\leq 2\\cdot c2^{n-1}-2d+1\\cr & =c2^{n}-d\\cr \\end{aligned} $$ the last step holds for $d\\geq 1$ For each of the following recurrences, sketch its recursion tree, and guess a good asymptotic upper bound on its solution. Then use the substitution method to verify your answer. a. $T(n)=T(n/2)+n^3$. b. $T(n) = 4T(n/3)+n$. c. $T(n) = 4T(n/2)+n$. d. $T(n) = 3T(n-1)+1$. a. guess $T(n)=O(n^3)$,assume $T(n)\\leq cn^3$ $$ \\begin{aligned} T(n) & = T(n/2)+n^3\\cr & \\leq \\frac{cn^3}{8}+n^3\\cr & \\leq (1+\\frac{c}{8})n\\cr & \\leq cn^3\\cr \\end{aligned} $$ the last step holds for $c\\geq 7/8$ b. guess $T(n)=O(n^{\\log_{3} 4})$, assume $T(n)\\leq cn^{\\log_3 4}-dn$ $$ \\begin{aligned} T(n) & =4T(n/3)+n\\cr & \\leq cn^{\\log_{3} 4} -\\frac{4dn}{3}+ n\\cr & \\leq cn^{\\log_{3} 4}\\cr \\end{aligned} $$ the last step holds for $d\\geq 4/3$ c. guess $T(n)=O(n^2)$, assume $T(n)\\leq cn^2-dn$ $$ \\begin{aligned} T(n) & =4T(n/2)+n\\cr & \\leq cn^2-2dn+n\\cr & \\leq cn^2\\cr \\end{aligned} $$ the last step holds for $d\\geq 1/2$ d. guess $T(n)=O(3^n)$, assume $T(n)\\leq c3^n-d$ $$ \\begin{aligned} T(n) & =3T(n-1)+1\\cr & \\leq 3\\cdot 3^{n-1}-3d +1\\cr & \\leq 3^n\\cr \\end{aligned} $$ the last step holds for $d\\geq 1/3$ Use the substitution method to prove that recurrence (4.15) has the asymptotic lower bound $L(n)=\\Omega(n)$. Conclude that $L(n)=\\Theta(n)$. assume$L(n)\\geq cn$ $$ \\begin{aligned} L(n) & =L(n/3)+L(2n/3)\\cr & \\geq c(n/3)+c(n2/3)\\cr & = cn\\cr \\end{aligned} $$ since the textbook has proved that $L(n)=O(n)$, and now $L(n)=\\Omega(n)$ is proved, we can conclude that $L(n)=\\Theta(n)$. Use the substitution method to prove that recurrence (4.14) has the solution $T(n) = \\Omega(n\\lg n)$. Conclude that $T(n)=\\Theta(n\\lg n)$. guess $T(n)\\leq cn\\lg n$ $$ \\begin{aligned} T(n) & =T(n/3)+T(2n/3)+\\Theta(n)\\cr & \\leq c(n/3)\\lg(n/3)+c(2n/3)\\lg(2n/3)+dn\\cr & = cn\\lg{n}-\\frac{cn}{3}(3\\lg{3}-2-3d/c)\\cr & \\leq cn\\lg{n}\\cr \\end{aligned} $$ the last step holds for $c\\geq 3d/2$. Use a recursion tree to justify a good guess for the solution to the recurrence $T(n)=T(\\alpha n)+T((1-\\alpha)n)+\\Theta(n)$, where $\\alpha$ is a constant in the range $0 < \\alpha 1$. assume $\\alpha<1/2$, since otherwise let $\\beta=1-\\alpha =$ and solve it for $\\beta$. let $L(n)$ donates leaves number, then $L(n)=L(\\alpha n)+L((1-\\alpha)n)\\implies L(n)=\\Theta(n)$ the smallest depth of leaves is $\\log_{1/\\alpha}n$, while the largest depth is $\\log_{\\alpha}n$, total cost over all nodes at depth $i$, for $i=0,1,\\dots,\\log_{1/\\alpha}n-1$ is $\\Theta(n)$ $$ \\begin{aligned} & \\sum_{i=0}^{\\log_{1/\\alpha}n-1}c_{1}n+d_{1}n\\leq T(n)\\leq \\sum_{i=0}^{\\log_{\\alpha}n-1}c_{2}n+d_{2}n\\cr \\implies & c_{1}n\\log_{1/\\alpha}n+d_{1}n \\leq T(n) \\leq c_{2}n\\log_{\\alpha}n+d_{2}n\\cr \\implies & T(n)=\\Theta(n\\lg n)\\cr \\end{aligned} $$ Use the master method to give tight asymptotic bounds for the following recurrences. a. $T(n)=2T(n/4)+1$. b. $T(n)=2T(n/4)+\\sqrt{n}$. c. $T(n)=2T(n/4)+\\sqrt{n}\\lg^2 n$. d. $T(n)=2T(n/4)+n$. e. $T(n)=2T(n/4)+n^2$. a. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=1=O(n^{\\log_{b}a-1/2})\\cr T(n)=\\Theta(n^{1/2})\\cr \\end{aligned} $$ b. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n^{1/2}=\\Theta(n^{\\log_{b}a})\\cr T(n)=\\Theta(n^{1/2}\\lg n)\\cr \\end{aligned} $$ c. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n^{1/2}\\lg^{2}n=\\Theta(n^{\\log_{b}a}\\lg^{2}n)\\cr T(n)=\\Theta(n^{1/2}\\lg^{3} n)\\cr \\end{aligned} $$ d. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n=\\Omega(n^{\\log_{b}a+1/2})\\cr \\text{regularity condition: }af(n/b)=n/2\\leq (1/2)n\\cr T(n)=\\Theta(n)\\cr \\end{aligned} $$ e. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n^2=\\Omega(n^{\\log_{b}a+3/2})\\cr \\text{regularity condition: }af(n/b)=n/8\\leq (1/8)n\\cr T(n)=\\Theta(n^2)\\cr \\end{aligned} $$ Professor Caesar wants to develop a matrix-multiplication algorithm that is asymptotically faster than Strassen\u2019s algorithm. His algorithm will use the divide-and-conquer method, dividing each matrix into $n/4\\times n/4$ submatrices, and the divide and combine steps together will take $\\Theta(n^2)$ time. Suppose that the professor\u2019s algorithm creates a recursive subproblems of size $n/4$. What is the largest integer value of $a$ for which his algorithm could possibly run asymptotically faster than Strassen\u2019s? $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{4}a} < n^{\\lg 7}\\cr \\implies a < 49\\cr \\end{aligned} $$ Use the master method to show that the solution to the binary-search recurrence $T(n) = T(n/2) + \\Theta(1)$ is $T(n) = \\Theta(\\lg n)$. (See Exercise 2.3-6 for a description of binary search.) $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr f(n)=1=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T(n)=\\Theta(\\lg n)\\cr \\end{aligned} $$ Consider the function $f(n) = \\lg n$. Argue that although $f(n/2) < f(n)$, the regularity condition $af(n/b)\\leq cf(n)$ with $a=1$ and $b=2$ does not hold for any constant $c<1$. Argue further that for any $\\epsilon > 0$, the condition in case 3 that $f(n) = \\Omega(n^{\\log_{b}a+\\epsilon})$ does not hold. $$ \\begin{aligned} &\\forall c<1\\cr &f(n/2)\\leq cf(n)\\cr \\iff & \\lg n-1\\leq c\\lg n\\cr \\iff & \\lg n\\leq 1/(1-c)\\cr \\iff & n\\leq 2^{1/(1-c)}\\cr \\end{aligned} $$ so the regularity condition does not hold for $n\\geq 2^{1/(1-c)}$ $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr \\forall \\epsilon>0\\cr cn^{\\epsilon}\\leq \\lg n \\text{ does not hold for }n\\to \\infty\\cr \\end{aligned} $$ so $f(n) = \\Omega(n^{\\log_{b}a+\\epsilon})$ does not hold. Show that for suitable constants $a,b$, and $\\epsilon$, the function $f(n)=2^{\\lceil\\lg n\\rceil}$ satisfies all the conditions in case 3 of the master theorem except the regularity condition. $$ \\begin{aligned} a=1.1,b=1.2,n^{\\log_{b}a}=n^{\\log_{1.2}1.1}\\cr f(n)\\geq 2^{\\lg n}=n\\geq n^{\\log_{1.2}1.1+\\log_{1.2}1.001}\\cr \\implies \\exist \\epsilon>0 \\text{ such that }f(n)=\\Omega(n^{\\log_{b}a+\\epsilon})\\cr \\end{aligned} $$ $f(n)$ satisfies all the conditions in case 3 of the master theorem except the regularity condition. $1.1f(\\frac{1.3\\cdot 2^k}{1.2})=1.1\\cdot 2^{k+1}\\leq cf(1.3\\cdot 2^k)=c\\cdot 2^{k+1}$ dose not hold for any c<1. Show that $\\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(\\log_{b} n-j)^k=\\Omega(\\log_b^{k+1}n)$. $$ \\begin{aligned} \\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(\\log_{b} n-j)^k & \\geq \\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(\\lfloor \\log_{b}n\\rfloor n-j)^k\\cr & = \\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(j)^k\\cr & \\geq \\sum_{j=0}^{\\log_{b}n-1}(j)^k\\cr & = \\Theta((\\log_{b}n-1)^{k+1})\\cr & = \\Omega(\\log_b^{k+1}n)\\cr \\end{aligned} $$ Show that case 3 of the master theorem is overstated (which is also why case 3 of Lemma 4.3 does not require that $f(n)=\\Omega(n^{\\log_b a+\\epsilon})$) in the sense that the regularity condition $af(n/b)\\leq cf(n)$ for some constant $c<1$ implies that there exists a constant $\\epsilon > 0$ sunch that $f(n)=\\Omega(n^{\\log_b a+\\epsilon})$. $$ \\begin{aligned} & af(n/b) \\leq cf(n)\\cr \\implies & f(n)\\geq \\frac{a}{c}f(n/b)\\cr \\implies & f(n) \\geq (\\frac{a}{c})^i f(n/{b^i})\\cr & \\text{let } n=b^i\\cr \\implies & f(n)\\geq (\\frac{a}{c})^{\\log_{b}n}f(1)\\cr \\implies & f(n) \\geq n^{\\log_{b}\\frac{a}{c}}\\cr & \\because c<0,\\therefore \\frac {a}{c}=a+\\epsilon \\text{ for some constant }\\epsilon >0\\cr \\implies & f(n)=\\Omega(n^{\\log_{b}a+\\epsilon})\\cr \\end{aligned} $$ For $f(n)=\\Theta(n^{\\log_b a}/\\lg n)$, prove that the summation in equation (4.19) has solution $g(n)=\\Theta(n^{\\log_b{a}}\\lg\\lg n)$. Conclude that a master recurrence $T(n)$ using $f(n)$ as its driving function has solution $T(n) = \\Theta(n^{\\log_b a}\\lg\\lg n)$. $$ \\begin{aligned} & \\text{let }n=b^i\\cr g(n) = & \\sum_{j=0}^{i}a^jf(n/b^j)\\cr = & \\sum_{j=0}^{i}a^jf(b^{i-j})\\cr = & \\sum_{j=0}^{i}a^j\\Theta(a^{i-j}/(i-j))\\cr = & \\sum_{j=0}^{i}a^i\\Theta(1/j)\\cr = & a^i\\Theta(\\lg i)\\cr = & \\Theta(a^{\\log_{b}n} \\lg\\log_{b}n)\\cr = & \\Theta(n^{\\log_{b}a} \\lg\\lg n)\\cr g(bn) = & \\Theta((bn)^{\\log_{b}a} \\lg\\lg bn)=(n^{\\log_{b}a} \\lg\\lg n)\\cr \\implies g(n)=& (n^{\\log_{b}a} \\lg\\lg n)\\text{ even if }n \\neq b^i\\cr \\end{aligned} $$ We can concluded that $T(n)=g(n)+n^{\\log_{b}a}=n^{\\log_{b}a} \\lg\\lg n$. Consider an Akra-Bazzi recurrence $T(n)$ on the reals as given in recurrence (4.22), and define $T'(n)$ as $$ \\begin{aligned} T'(n)=cf(n)+\\sum_{i=1}^k a_iT'(n/b_i), \\end{aligned} $$ where $c > 0$ is constant. Prove that whatever the implicit initial conditions for T(n) might be, there exist initial conditions for $T'(n)$ such that $T'(n) = cT(n)$ for all $n>0$. Conclude that we can drop the asymptotics on a driving function in any Akra-Bazzi recurrence without affecting its asymptotic solution. assume $T'(n)=cT(n)$ $$ \\begin{aligned} T'(n) & =cf(n)+\\sum_{i=1}^k a_iT'(n/b_i)\\cr & =cf(n)+\\sum_{i=1}^k a_icT(n/b_i)\\cr & =c(f(n)+\\sum_{i=1}^k a_iT(n/b_i))\\cr & =cT(n)\\cr \\end{aligned} $$ $T'(n)=T(n)$ holds only if the base case(initial conditions) that $T'(n)=T(n)$ for all $n<n_{}0$ is true Show that $f(n)=n^2$ satisfies the polynomial-growth condition but that $f(n)=2^n$ does not. $$ \\begin{aligned} & \\forall \\phi \\forall \\psi:1 \\leq \\psi \\leq \\phi\\cr & n^2/d\\leq (\\psi n)^2\\leq dn^2\\cr \\iff & \\max(1/\\psi^{2})\\leq d \\wedge \\max(\\psi^{2}) \\leq d\\cr \\iff & \\phi^{2}\\leq d\\qquad(1)\\cr & 2^n/d\\leq 2^{\\psi n}\\leq d2^n\\cr \\iff & 2^{n-\\psi n}\\leq d\\wedge 2^{\\psi n-n}\\leq d\\qquad (2)\\cr \\end{aligned} $$ $d=\\phi^{2}$ satisfies $(1)$, since $2^{(\\psi-1)n}\\to\\infty$ therefore no constant $d$ satisfies $(2)$ Let f(n) be a function that satisfies the polynomial-growth condition. Prove that $f(n)$ is asymptotically positive, that is, there exists a constant $n_0\\geq 0$ such that $f(n)\\geq 0$ for all $n\\geq n_0$. $$ \\begin{aligned} \\exist \\hat{n} \\text{ that }\\forall n > \\hat{n}:\\cr f(n)/d \\leq f(\\psi n) \\leq df(n)\\cr \\implies 0\\leq (d^2-1) f(n)\\cr \\because d>1\\cr \\therefore f(n)>0\\cr \\end{aligned} $$ Give an example of a function $f(n)$ that does not satisfy the polynomial-growth condition but for which $f(\\Theta(n)) = \\Theta(f(n))$. Use the Akra-Bazzi method to solve the following recurrences. a. $T(n) = T(n/2)+T(n/3)+T(n/6)+n\\lg n$. b. $T(n) = 3T(n/3)+8T(n/4)+n^2/\\lg n$. c. $T(n) = (2/3)T(n/3)+(1/3)T(2n/3)+\\lg n$. d. $T(n) = (1/3)T(n/3) + 1/n$. e. $T(n) = 3T(n/3) + 3T(2n/3)+n^2$. a. $$ \\begin{aligned} & (\\frac{1}{2})^p+(\\frac{1}{3})^p+(\\frac{1}{6})^p=1\\cr \\implies & p=1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{x\\lg x}{x^{2}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{\\ln x}{x\\ln 2}dx))\\cr & = \\Theta(n^1(1+\\left[\\frac{\\ln^{2} x}{2\\ln 2}\\right]_{1}^{n}))\\cr & = \\Theta(n(1+\\frac{\\ln^{2} n}{2\\ln 2}))\\cr & =\\Theta(n\\lg^{2} n) \\end{aligned} $$ b. $$ \\begin{aligned} & \\frac{3}{3^p}+\\frac{8}{4^p}=1\\cr & \\frac{3}{3^{1.5}}+\\frac{8}{4^{1.5}}=3^{-(1/2)}+1\\cr & \\frac{3}{3^2}+\\frac{8}{4^2}=\\frac{5}{6}\\cr \\implies & 1.5<p<2\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{x^2/\\lg x}{x^{p+1}}dx))\\cr & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{1}{x^{p-1}\\lg x}dx))\\cr & =\\Omega(n^p(1+\\lg n\\int_{1}^{n}\\frac{1}{x^{p-1}}dx))\\cr & =\\Omega(n^p(1+n^{2-p}\\lg n ))\\cr & =\\Omega(n^2/\\lg n)\\cr T(n) & =O(n^p(1+\\lg 1(\\text{regarded as constant})\\int_{1}^{\\sqrt{n}}\\frac{1}{x^{p-1}}dx+\\lg \\sqrt{n}\\int_{\\sqrt{n}}^{n}\\frac{1}{x^{p-1}}dx))\\cr & = O(n^p(1+\\Theta(n^{(p-1)/2})+\\Theta(n^{p-1}/\\lg n)))\\cr & =O(n^2/\\lg n)\\cr \\end{aligned} $$ c. $$ \\begin{aligned} & \\frac{2}{3}(\\frac{1}{3})^p+\\frac{1}{3}(\\frac{2}{3})^p=1\\cr \\implies & \\frac{2+2^p}{3^{p+1}}=1\\cr \\implies & p=0\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(1+\\int_{1}^{n}\\frac{\\lg x}{x}dx)\\cr & =\\Theta(\\lg^{2}n) \\end{aligned} $$ d. $$ \\begin{aligned} & \\frac{1}{3}(\\frac{1}{3})^p=1\\cr \\implies & p=-1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^{-1}(1+\\int_{1}^{n}\\frac{1}{x}dx))\\cr & =\\Theta(n^{-1}(1+\\left[lg n\\right]_{1}^{n}))\\cr & =\\Theta(n^{-1}lg n)\\cr \\end{aligned} $$ e. $T(n) = 3T(n/3) + 3T(2n/3)+n^2$. $$ \\begin{aligned} & 3(\\frac{1}{3})^p + 3(\\frac{2}{3})^p=1\\cr \\implies & p=2\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^{2}(1+\\int_{1}^{n}\\frac{x^2}{x^3}dx))\\cr & =\\Theta(n^2\\lg n) \\end{aligned} $$ Use the Akra-Bazzi method to prove the continuous master theorem. $$ \\begin{aligned} & T(n)=aT(n/b)+f(n)\\cr & \\frac{a}{b^p}=1\\cr \\implies & p=\\log_{b}a\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^{\\log_{b}a}(1+\\int_{1}^{n}\\frac{f(x)}{x^{\\log_{b}a+1}}dx))\\cr & G(n)=\\int_{1}^{n}\\frac{f(x)}{x^{\\log_{b}a+1}}dx\\cr \\text{case 1:} & \\cr & f(n)=O(n^{\\log_{b}a-\\epsilon})(\\epsilon>0)\\cr G(n) & = O(\\int_{1}^{n}\\frac{1}{x^{1+\\epsilon}}dx)\\cr & = O(x^{-\\epsilon})\\cr T(n) & =\\Theta(n^{\\log_{b}a}(1+G(n)))\\cr & = \\Theta(n^{\\log_{b}a})\\cr \\text{case 2:} & \\cr & f(n)=\\Theta(n^{\\log_{b}a}\\lg^{k}n)(k\\geq 0)\\cr G(n) & = \\Theta(\\int_{1}^{n}\\frac{lg^{k}n}{x^{1}}dx)\\cr & = \\Theta(\\lg^{k+1}n)\\cr T(n) & =\\Theta(n^{\\log_{b}a}(1+G(n)))\\cr & = \\Theta(n^{\\log_{b}a}\\lg^{k+1}n)\\cr \\text{case 3:} & \\cr T(n) & =\\Omega(f(n))\\cr & f(n)=\\Omega(n^{\\log_{b}a+\\epsilon})(\\epsilon>0)\\cr &\\text{if } af(n/b)<cf(n)\\text{ for some }c<1\\cr G(n) & = O(f(n)\\int_{1}^{n}\\frac{1}{x^{\\log_{b}a+1}}dx)\\cr & = O(\\frac{f(n)}{x^{\\log_{b}a}})\\cr T(n) & =\\Theta(n^{\\log_{b}a}(1+G(n)))\\cr & =O(f(n))\\cr T(n)& =\\Theta(f(n)) \\end{aligned} $$ Give asymptotically tight upper and lower bounds for $T(n)$ in each of the following algorithmic recurrences. Justify your answers. a. $T(n) = 2T(n/2)+n^3$. b. $T(n) = T(8n/11)+n$. c. $T(n) = 16T(n/4)+n^2$. d. $T(n)=4T(n/2)+n^2\\lg n$. e. $T(n) = 8T(n/3)+n^2$. f. $T(n)=7T(n/2)+n^2\\lg n$. g. $T(n)=2T(n/4)+\\sqrt{n}$. h. $T(n)=T(n-2)+n^2$. $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{2}2}=n\\cr f(n)=n^3=\\Omega(n^{2+\\log_{b}a})\\cr 2f(n/2)=n^3/4\\leq \\frac{1}{4}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n^3)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{11/8}1}=1\\cr f(n)=n=\\Omega(n^{1+\\log_{b}a})\\cr f(8n/11)=8n/11\\leq \\frac{8}{11}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{4}{16}}=n^{2}\\cr f(n)=n^2=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T(n)=\\Theta(n^2\\lg^{} n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{2}{4}}=n^{2}\\cr f(n)=n^2\\lg^{} n=\\Theta(n^{\\log_{b}a}\\lg^{}n)\\cr T(n)=\\Theta(n^2\\lg^{2} n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{3}{8}}\\cr f(n)=n^2=\\Omega(n^{\\log_{b}a})\\cr f8(n/3)=\\frac{8n^2}{9}\\leq \\frac{8}{9}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n^2)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{2}{7}}\\cr f(n)=n^2\\lg^{} n=O(n^{\\log_{b}a-\\log_2{(7/4)}})\\cr T(n)=\\Theta(n^{\\log_{2}{7}})\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{4}{2}}=n^{1/2}\\cr f(n)=n^{1/2}=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T(n)=\\Theta(n^{1/2}\\lg^{} n)\\cr \\end{aligned} $$ $$ \\begin{aligned} T(n)=\\frac{1}{2}\\sum_{i=1}^{n}i^2=\\Theta(n^3)\\cr \\end{aligned} $$ Throughout this book, we assume that parameter passing during procedure calls takes constant time, even if an N -element array is being passed. This assumption is valid in most systems because a pointer to the array is passed, not the array itself. This problem examines the implications of three parameter-passing strategies: Arrays are passed by pointer. Time $= \\Theta(1)$. Arrays are passed by copying. Time $=\\Theta(N)$, where $N$ is the size of the array. Arrays are passed by copying only the subrange that might be accessed by the called procedure. Time $=\\Theta(n)$ if the subarray contains $n$ elements. Consider the following three algorithms: a. The recursive binary-search algorithm for finding a number in a sorted array(see Exercise 2.3-6). b. The MERGE-SORT procedure from Section 2.3.1. c. The MATRIX-MULTIPLY-RECURSIVE procedure from Section 4.1. Give nine recurrences $T_{a1}(N,n),T_{a2}(N,n),\\dots ,T_{c3}(T,n)$ for the worst-case running times of each of the three algorithms above when arrays and matrices are passed using each of the three parameter-passing strategies above. Solve your recurrences, giving tight asymptotic bounds. a1 $$ \\begin{aligned} T_{a1}(N,n)=T(N,\\frac{n}{2})+\\Theta(1)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr f(n)=\\Theta(1)=\\Theta(n^{\\log_{b}a})\\cr T_{a1}(N,n)=\\Theta(\\lg n)\\cr \\end{aligned} $$ a2 $$ \\begin{aligned} T_{a2}(N,n)=T(N,\\frac{n}{2})+\\Theta(N)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr T_{a2}(N,n)=\\Theta(N\\lg n)\\cr \\end{aligned} $$ a3 $$ \\begin{aligned} T_{a3}(N,n)=T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr f(n)=\\Theta(n)=\\Omega(n^{1+\\log_{b}a})\\cr af(\\frac{n}{b})=f(\\frac{n}{2})\\leq \\frac{1}{2}f(n)\\cr T_{a3}(N,n)=\\Theta(f(n))=\\Theta(n)\\cr \\end{aligned} $$ b1 $$ \\begin{aligned} T_{b1}(N,n)=2T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}2}=n\\cr f(n)=\\Theta(n)=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T_{b1}(N,n)=\\Theta(n \\lg n)\\cr \\end{aligned} $$ b2 $$ \\begin{aligned} T_{b2}(N,n)=2T(N,\\frac{n}{2})+\\Theta(N+n)=2T(N,\\frac{n}{2})+\\Theta(N)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=n\\cr T_{b2}(N,n)=\\Theta(n+N\\lg n)=\\Theta(N\\lg n)\\cr \\end{aligned} $$ b3 $$ \\begin{aligned} T_{b3}(N,n)=2T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}2}=n\\cr f(n)=\\Theta(n)=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T_{b3}(N,n)=\\Theta(n \\lg n)\\cr \\end{aligned} $$ c1 $$ \\begin{aligned} T_{c1}(N,n)=8T(N,\\frac{n}{2})+\\Theta(1)\\cr n^{\\log_{b}a}=n^{\\log_{2}8}=n^3\\cr f(n)=\\Theta(1)=O(n^{\\log_{b}a-3})\\cr T_{c1}(N,n)=\\Theta(n^3)\\cr \\end{aligned} $$ c2 $$ \\begin{aligned} T_{c2}(N,n)=8T(N,\\frac{n}{2})+\\Theta(N)\\cr n^{\\log_{b}a}=n^{\\log_{2}8}=n^3\\cr T_{c2}(N,n)=\\Theta(n^3+N\\lg N)\\cr \\end{aligned} $$ c3 $$ \\begin{aligned} T_{c3}(N,n)=8T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}8}=n^3\\cr f(n)=\\Theta(n)=O(n^{\\log_{b}a-2})\\cr T_{c3}(N,n)=\\Theta(n^3)\\cr \\end{aligned} $$ Sometimes, a little algebraic manipulation can make an unknown recurrence similar to one you have seen before. Let\u2019s solve the recurrence $T(n)=2T(\\sqrt{n})+\\Theta(\\lg n)$ (4.25) by using the change-of-variables method. a. Define $m=\\lg n$ and $S(m)=T(2^m)$. Rewrite recurrence (4.25) in terms of m and $S(m)$. b. Solve your recurrence for $S(m)$. c. Use your solution for $S(m)$ to conclude that $T(n)=\\Theta(\\lg n\\lg\\lg n)$. d. Sketch the recursion tree for recurrence (4.25), and use it to explain intuitively why the solution is $T(n)=\\Theta(\\lg n \\lg\\lg n)$. Solve the following recurrences by changing variables: e. $T(n)=2T(\\sqrt{n})+\\Theta(1)$. f. $T(n)=3T(\\sqrt[3]{n})+\\Theta(n)$. $S(m)=T(2^m)=2T(\\sqrt{2^m})+\\Theta(m)=2T(2^{m/2})+\\Theta(m)=2S(m/2)+\\Theta(m)$ $$ \\begin{aligned} m^{\\log_{b}a}=m\\cr f(m)=m=\\Theta(m^{\\log_{b}a})\\cr S(m)=\\Theta(m\\lg m) \\end{aligned} $$ $T(n)=S(\\lg n)=\\Theta(\\lg n\\lg\\lg n)$ depth:$\\lg\\lg n +1$, leaves: $2^{\\lg\\lg n}$ The total cost over all nodes at depth $i$,for $i=0,1,\\dots,\\lg\\lg n-1$ is $2^i\\cdot \\frac{1}{2^i}\\Theta(\\lg n)=\\Theta(\\lg n)$ $$ \\begin{aligned} T(n) & =2^{\\lg\\lg n}\\Theta(1)+\\sum_{i=0}^{\\lg\\lg n-1}\\Theta(\\lg n)\\cr & =\\Theta(\\lg n)+\\lg\\lg n\\Theta(\\lg n)\\cr & = \\Theta(\\lg n\\lg\\lg n)\\cr \\end{aligned} $$ depth:$\\lg\\lg n +1$, leaves: $2^{\\lg\\lg n}$ The total cost over all nodes at depth $i$,for $i=0,1,\\dots,\\lg\\lg n-1$ is $2^i\\cdot \\Theta(1)=\\Theta(2^i)$ $$ \\begin{aligned} T(n) & =2^{\\lg\\lg n}\\Theta(1)+\\sum_{i=0}^{\\lg\\lg n-1}2^i\\Theta(1)\\cr & =\\Theta(\\lg n)+(2^{\\lg\\lg n}-1)\\Theta(1)\\cr & = \\Theta(\\lg n)\\cr \\end{aligned} $$ $S(m)=T(3^m)=3T(3^{m/3})+\\Theta(3^m)=3S(m/3)+\\Theta(3^m)$. $$ \\begin{aligned} m^{\\log_{b}a}=m^{\\log_{3}3}=m\\cr f(m)=3^m=\\Omega(m^{2+\\log_{b}a})\\cr 3f(m/3)=3\\cdot 3^{m/3}\\leq (1/3)f(m)\\cr S(m)=\\Theta(f(m))=\\Theta(3^m)\\cr T(n)=S(\\log_{3}n)=\\Theta(n)\\cr \\end{aligned} $$ Give asymptotically tight upper and lower bounds for T(n) in each of the following recurrences. Justify your answers. a. $T(n)=5T(n/3)+n\\lg n$. b. $T(n)=3T(n/3)+n/\\lg n$. c. $T(n)= 8T(n/2)+n^3\\sqrt{n}$. d. $T(n)=2T(n/2-2)+n/2$. e. $T(n)=2T(n/2)+n/\\lg n$. f. $T(n)=T(n/2)+T(n/4)+T(n/8)+n$. g. $T(n)=T(n-2)+1/n$. h. $T(n)=T(n-1)+\\lg n$. i. $T(n)=T(n-2)+1/\\lg n$. j. $T(n)=\\sqrt{n}T(\\sqrt{n})+n$. $$ \\begin{aligned} n^{\\log_{b}{a}} = n^{\\log_{3}{5}}\\cr f(n) = n\\lg{n} = O(n^{\\log_{b}{a}-\\log_{3}{(5/4)}})\\cr T(n)=\\Theta({n^{\\log_{b}{a}}})=\\Theta(n^{\\log_{3}{5}})\\cr \\end{aligned} $$ $$ \\begin{aligned} & 3(\\frac{1}{3})^p=1\\cr \\implies & p = 1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{x/\\lg x}{x^{2}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{1}{x^{}\\lg x}dx))\\cr & = \\Theta(n^1(1+\\left[\\lg\\lg x\\right]_{1}^{n}))\\cr & = \\Theta (n\\lg\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}} = n^{\\log_{2}{8}}=n^{3}\\cr f(n) = n^{7/2} = \\Omega(n^{\\log_{b}{a}+1/2})\\cr af(n/b)=2^{-1/2}(n^{7/2})\\leq 2^{-1/2}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n^{7/2})\\cr \\end{aligned} $$ $$ \\begin{aligned} f(n)=n/2 \\text{ satisfies polynomial-growth condition}\\cr T(n)=2T(n/2)+n/2\\cr n^{\\log_{b}{a}} = n^{\\log_{2}{2}}=n^{}\\cr f(n) = n^{}/2 = \\Theta(n^{\\log_{b}{a}}\\lg^{0}n)\\cr T(n)=\\Theta(n^{\\log_{b}{a}}\\lg n)=\\Theta(n\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} & 2(\\frac{1}{2})^p=1\\cr \\implies & p = 1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{x/\\lg x}{x^{2}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{1}{x^{}\\lg x}dx))\\cr & = \\Theta(n^1(1+\\left[\\lg\\lg x\\right]_{1}^{n}))\\cr & = \\Theta(n\\lg\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} & (\\frac{1}{2})^{p}+(\\frac{1}{4})^{p}+(\\frac{1}{8})^{p}=1\\cr & 0.5<p<1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^p(1+\\int_{1}^{n}\\frac{x}{x^{p+1}}dx))\\cr & = \\Theta(n^p(1+\\int_{1}^{n}\\frac{1}{x^{p}}dx))\\cr & = \\Theta(n^p(1+\\left[x^{1-p}\\right]_{1}^{n}))\\cr & = \\Theta(n)\\cr \\end{aligned} $$ $$ \\begin{aligned} T(n) & = \\frac{1}{2}\\sum_{i=1}^{n}(\\frac{1}{i})\\cr & \\leq \\frac{1}{2}+1/2\\int_{1}^{n-1}\\frac{1}{x}\\cr & = \\Theta(\\frac{1}{2}+\\frac{1}{2}\\lg (n-1))\\cr & = \\Theta(\\lg n)\\cr \\end{aligned} $$ $T(n)=T(n-1)+\\lg n$ $$ \\begin{aligned} T(n) & = \\sum_{i=1}^{n}\\lg i\\cr & = \\lg(n!)\\cr & = \\Theta(n\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} T(n) & =1+\\sum_{i=2}^{n}\\frac{1}{\\lg i}\\cr & = \\Omega(\\frac{n^2}{\\sum_{i=2}^{n}\\lg n})\\cr & = \\Omega(\\frac{n}{\\lg n})\\cr T(n) & =O(\\sum_{i=2}^{\\sqrt{n}-1}\\frac{1}{\\lg 2}+\\sum_{i=\\sqrt{n}}^{n}\\frac{1}{\\lg \\sqrt{n}})\\cr & = O(\\sqrt{n}+\\frac{n}{\\lg n})\\cr T(n) & = \\Theta(n)\\cr \\end{aligned} $$ depth: $\\lg\\lg n+1$, leaves: $n$ total cost of over all nodes at depth $i$, for $i=0,1,...,\\lg \\lg n-1$, is n $$ \\begin{aligned} T(n)=\\Theta(n\\lg\\lg n)\\cr \\end{aligned} $$ This problem develops properties of the Fibonacci numbers, which are defined by recurrence (3.31) on page 69. We\u2019ll explore the technique of generating functions to solve the Fibonacci recurrence. Define the generating function (or formal power series) $\\mathcal{F}$ as $$ \\begin{aligned} \\mathcal{F}(z) & = \\sum_{i=0}^{\\infty}F_iz^i\\cr & = 0+z+z^2+2z^3+3z^4+5z^5+8z^6+13z^7+21z^8+\\cdots,\\cr \\end{aligned} $$ where $F_i$ is the $i$th Fibonaccci nunber. a. Show that $\\mathcal{F}(z)=z+z\\mathcal{F}(z)+z^2\\mathcal{F}(z)$. b. Show that $$ \\begin{aligned} \\mathcal{F}(z) & = \\frac{z}{1-z-z^2}\\cr & = \\frac{z}{(1-\\phi z)(1-\\hat\\phi z)}\\cr & = \\frac{1}{\\sqrt{5}}(\\frac{1}{1-\\phi z}-\\frac{1}{1-\\hat\\phi z})\\cr \\end{aligned} $$ where $\\phi$ is the golden ratio, and $\\hat{\\phi}$ is its conjugatr(see page 69). c. Show that $$ \\mathcal{F}(z)=\\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\phi^{i}-\\hat{\\phi}^i)z^i. $$ You may use without proof the generating-function version of equation (A.7) on page 1142, $\\sum_{k=0}^{\\infty} x^k=1/(1-x)$. Because this equation involves a generating function, $x$ is formal variable, not a real-valued variable, so that you don\u2019t have to worry about convergence of the summation or about the requirement in equation (A.7) that $|x|<1$, which doesn\u2019t make sense here. d. Use part (c) to prove that $F_i=\\phi^i/\\sqrt{5}$ for $i>0$, rounded to the nearest integer. (Hint: Observe that $|\\hat\\phi<1|$.) e Prove that $F_{i+2}\\geq \\phi^{i}$ for $i\\geq 0$. $$ \\begin{aligned} z+z\\mathcal{F}(z)+z^2\\mathcal{F}(z) & =z+z\\sum_{i=0}^{\\infty}F_iz^i + z^{2}\\sum_{i=0}^{\\infty}F_iz^i\\cr & = z+\\sum_{i=1}^{\\infty}F_{i-1}z^i + \\sum_{i=2}^{\\infty}F_{i-2}z^i\\cr & = z+F_{0}z^1 + \\sum_{i=2}^{\\infty}(F_{i-2}+F_{i-1})z^i\\cr & = z+0 + \\sum_{i=2}^{\\infty}F_{i}z^i\\cr & =\\sum_{i=0}^{\\infty}F_{i}z^i\\cr & =\\mathcal{F}(z)\\cr \\end{aligned} $$ $$ \\begin{aligned} &\\mathcal{F}(z)=z+z\\mathcal{F}(z)+z^2\\mathcal{F}(z)\\cr \\implies & \\mathcal{F}(z) = \\frac{z}{1-z-z^2}\\cr & \\phi\\hat{\\phi}=-1;\\phi + \\hat{\\phi}=1;\\phi - \\hat{\\phi}=\\sqrt{5}\\cr \\implies & \\mathcal{F}(z) = \\frac{z}{1-(\\phi + \\hat{\\phi})z-(-\\phi\\hat{\\phi})z^2}\\cr \\implies & \\mathcal{F}(z) = \\frac{z}{(1-\\phi z)(1-\\hat\\phi z)}\\cr \\implies & \\mathcal{F}(z) = \\frac{1}{\\sqrt{5}}\\frac{(\\phi - \\hat{\\phi}) z}{(1-\\phi z)(1-\\hat\\phi z)}\\cr \\implies & \\mathcal{F}(z) = \\frac{1}{\\sqrt{5}}(\\frac{1}{1-\\phi z}-\\frac{1}{1-\\hat\\phi z})\\cr \\end{aligned} $$ $$ \\begin{aligned} \\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\phi^{i}-\\hat{\\phi}^i)z^i & = \\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\phi z)^i - \\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\hat{\\phi}z)^i\\cr & = \\frac{1}{\\sqrt{5}}(\\frac{1}{1-\\phi z}-\\frac{1}{1-\\hat{\\phi} z})\\cr & = \\mathcal{F}(z)\\cr \\end{aligned} $$ $$ \\begin{aligned} F_{i} & =\\frac{1}{\\sqrt{5}}(\\phi^{i}-\\hat{\\phi}^i)\\cr & = \\frac{\\phi^{i}}{\\sqrt{5}}-\\frac{\\hat{\\phi}^i}{\\sqrt{5}}\\cr & \\because |\\hat{\\phi}^i|<1 \\therefore |\\hat{\\phi}^i/\\sqrt{5}|<0.5\\cr \\implies & F_{i}=round(\\frac{\\phi^{i}}{\\sqrt{5}})\\cr \\end{aligned} $$ $$ \\begin{aligned} F_{2} & =1\\geq\\phi^{0}=1\\cr F_{3} & =2\\geq\\phi^{1}=\\frac{1+\\sqrt{5}}{2}\\cr F_{i+2} & = F_{i+1}+F_{i}\\cr & \\geq \\phi^{i-1}+\\phi^{i-2}\\cr & =\\phi^{i-2}(\\phi+1)\\cr & =\\phi^{i-2}(\\phi^2)\\cr & = \\phi^{i}\\cr \\end{aligned} $$ Professor Diogenes has $n$ supposedly identical integrated-circuit chips that in principle are capable of testing each other. The professor\u2019s test jig accommodates two chips at a time. When the jig is loaded, each chip tests the other and reports whether it is good or bad. A good chip always reports accurately whether the other chip is good or bad, but the professor cannot trust the answer of a bad chip. Thus, the four possible outcomes of a test are as follows: a. Show that if at least $n/2$ chips are bad, the professor cannot necessarily determine which chips are good using any strategy based on this kind of pairwise test. Assume that the bad chips can conspire to fool the professor. Now you will design an algorithm to identify which chips are good and which are bad, assuming that more than $n/2$ of the chips are good. First, you will determine how to identify one good chip. b. Show that $\\lfloor n/2 \\rfloor$ pairwise tests are sufficient to reduce the problem to one of nearly half the size. That is, show how to use $\\lfloor n/2 \\rfloor$ pairwise tests to obtain a set with at most $\\lceil n/2 \\rceil$ chips that still has the property that more than half of the chips are good. c. Show how to apply the solution to part (b) recursively to identify one good chip. Give and solve the recurrence that describes the number of tests needed to identify one good chip. You have now determined how to identify one good chip. d. Show how to identify all the good chips with an additional $\\Theta(n)$ pairwise tests. If the professor want to determine which chip is good, he must get case 1 that A and B both say the other is good, and ensure there is no bad chip left. So he need to use case 2,3,4 to drop bad chips, since bad chips conspire to fool the professor, each time will drop a bad chip together a good one. If bad chips number is more than good chips, the professor cannot necessarily determine which chips are good. already apply the solution recursively in part(b), $$ \\begin{aligned} T(n)=n/2+T(n/2)\\cr T(n)=\\Theta(n)\\cr \\end{aligned} $$ use the good chip to test other chips$\\Theta(n)$. An $m\\times n$ array $A$ of real numbers is a Monge array if for all $i,j,k$, and $l$ such that $1\\leq i <k \\leq m$ and $i \\leq j < l \\leq n$, we have $A[i,j]+A[k,l]\\leq A[i,l]+A[k,j]$. In other words, whenever we pick two rows and two columns of a Monge array and consider the four elements at the intersections of the rows and the columns, the sum of the upper-left and lower-right elements is less than or equal to the sum of the lower-left and upper-right elements. For example, the following array is Monge: $$ \\begin{array}{} 10 & 17 & 13 & 28 & 23\\cr 17 & 22 & 16 & 29 & 23\\cr 24 & 28 & 22 & 34 & 24\\cr 11 & 13 & 6 & 17 & 7\\cr 45 & 44 & 32 & 37 & 23\\cr 36 & 33 & 19 & 21 & 6\\cr 75 & 66 & 51 & 53 & 34\\cr \\end{array} $$ a. Prove that an array is Monge if and only if for all $i=1,2,\\dots,m-1$ and $j=1,2,\\dots,n-1$, we have $$ A[i,j]+A[i+1,j+1]\\leq A[i,j+1]+A[i+1,j]. $$ (Hint: For the \"if\" part, use induction separately on rows and columns.) b. The following array is not Monge. Change one element in order to make it Monge. (Hint: Use part (a).) $$ \\begin{array}{} 37 & 23 & 22 & 32\\cr 21 & 6 & 7 & 10\\cr 53 & 34 & 30 & 31\\cr 32 & 13 & 9 & 6\\cr 43 & 21 & 15 & 8\\cr \\end{array} $$ c. Let $f(i)$ be the index of the column containing the leftmost minimum elementof row $i$. Prove that $f(1)\\leq f(2)\\leq \\cdots \\leq f(m)$ for any $m\\times n$ Monge array. d. Here is a description of a divide-and-conquer algorithm that computes the left most minimum element in each row of an $m\\times n$ Monge array A: Construct a submatrix $A'$ of $A$ consisting of the even-numbered rows of $A$. Recursively determine the leftmost minimum for each row of $A'$. Then compute the leftmost minimum in the odd-numbered rows of $A'$. Explain how to compute the leftmost minimum in the odd-numbered rows of $A$ (given that the leftmost minimum of the even-numbered rows is known) in $O(m+n) time. e. Write the recurrence for the running time of the algorithm in part (d). Show that its solution is $O(m+n\\log m)$. $$ \\begin{aligned} \\text{assume }A[i,j]+A[i+k,j+l]\\leq A[i,j+l]+A[i+k,j]\\cr A[i+k,j]+A[(i+1)+k,j+l] \\leq A[i+k,j+l]+A[i+k+1,j]\\cr \\implies A[i,j]+A[i+k+1,j+l]\\leq A[i,j+l]+A[i+k+1,j]\\cr A[i,j+l]+A[i+k,j+l+1]\\leq A[i,j+l+1]+A[i+k,j+1]\\cr \\implies A[i,j]+A[i+k,(j+1)+l]\\leq A[i,j+l+1]+A[i+k,j]\\cr \\end{aligned} $$ $$ \\begin{array}{} 37 & 23 & 22 & 32\\cr 21 & 6 & (5) & 10\\cr 53 & 34 & 30 & 31\\cr 32 & 13 & 9 & 6\\cr 43 & 21 & 15 & 8\\cr \\end{array} $$ $$ \\begin{aligned} if(k)\\cr \\implies A[i,f(k)]+A[k,f(i)] \\leq A[i,f(i)]+A[k,f(k)]\\cr \\implies A[i,f(k)]+A[k,f(i)] = A[i,f(i)]+A[k,f(k)]\\cr A[i,f(i)]\\leq A[i,f(k)]\\cr A[k,f(k)]\\leq A[k,(f(i))]\\cr \\implies A[i,f(i)] = A[i,f(k)]\\cr \\implies A[k,f(k)]= A[k,(f(i))]\\cr \\implies f(i)\\leq f(k)\\cr \\text{Contradictory to assumption}\\cr \\implies f(i)\\leq f(k)\\cr \\end{aligned} $$"},{"location":"Chap04/Problems/4-7/#d","title":"d","text":" since $f(2k) \\leq f(2k+1)\\leq f(2k+2)$ in each odd row, we only need to compare $f(2k+2)-f(2k)+1$ elements. totally need to deal with $O(m)+O(n)=O(m+n)$ over all odd rows. $$ \\begin{aligned} T(m,n) & = T(m/2,n)+O(m+n)\\cr T(m,n) & = \\sum_{i=0}^{\\lg m}O(m/2^i+n)\\cr & = O(n\\lg m)+\\sum_{i=1}^{\\lg m}O(m/2^i)\\cr & = O(n\\lg m)+O(m)\\cr & = O(m+n\\lg m)\\cr \\end{aligned} $$ For a clear commit history, I rebase my repository regularly. Therefore, if you have forked the repository before, consider re-forking it again. Licensed under the MIT License. Describe your own real-world example that requires sorting. Describe one that requires finding the shortest distance between two points. Other than speed, what other measures of efficiency might you need to consider in a real-world setting? Memory efficiency and coding efficiency. Select a data structure that you have seen, and discuss its strengths and limitations. arrays: How are the shortest-path and traveling-salesperson problems given above similar?How are they different? Suggest a real-world problem in which only the best solution will do. Then come up with one in which \"approximately\" the best solution is good enough. Describe a real-world problem in which sometimes the entire input is available before you need to solve the problem, but other times the input is not entirely available in advance and arrives over time. In team work: Give an example of an application that requires algorithmic content at the application level, and discuss the function of the algorithms involved. Suppose we are comparing implementations of insertion sort and merge sort on the same machine. For inputs of size $n$ , insertion sort runs in $8n^2$ steps, while merge sort runs in $64n\\lg n$ steps. For which values of $n$ does insertion sort beat merge sort? $$ \\begin{aligned} 8n^2 < 64n \\lg n \\cr 0 < n < 8 \\lg n \\cr 1< 2^n < n^8 \\cr 2 \\leq n \\leq 43 \\cr \\end{aligned} $$ What is the smallest value of n such that an algorithm whose running time is $100n^2$ runs faster than an algorithm whose running time is $2^n$ on the same machine? $$ \\begin{aligned} 100n^2 < 2^n \\cr n \\ge 15.\\cr \\end{aligned} $$ For each function $f(n)$ and time $t$ in the following table, determine the largest size $n$ of a problem that can be solved in time $t$, assuming that the algorithm to solve the problem takes $f(n)$ microseconds. Using Figure 2.2 as a model, illustrate the operation of $\\text{INSERTION-SORT}$ on the array $A = \\langle 31, 41, 59, 26, 41, 58 \\rangle$. as shown in the figure above, the array A change as follow $$ \\begin{aligned} A = \\langle 31, 41, 59, 26, 41, 58 \\rangle\\cr A = \\langle 31, 41, 59, 26, 41, 58 \\rangle\\cr A = \\langle 26, 31, 41, 59, 41, 58 \\rangle\\cr A = \\langle 26, 31, 41, 41, 59, 58 \\rangle\\cr A = \\langle 26, 31, 41, 41, 58, 59 \\rangle\\cr A = \\langle 26, 31, 41, 41, 58, 59 \\rangle\\cr \\end{aligned} $$ Consider the procedure SUM-ARRAY on the facing page. It computes the sum of the n numbers in array $A[1:n]$. State a loop invariant for this procedure, and use its initialization, maintenance, and termination properties to show that the SUMARRAY procedure returns the sum of the numbers in $A[1:n]$. Rewrite the INSERTION-SORT procedure to sort into monotonically decreasing instead of monotonically increasing order. Consider the searching problem: Input: A sequence of n numbers $\\langle a1 , a2,..., an \\rangle$stored in array A[1:n] and a value x. Output: An index i such that x equals $A[i]$ or the special value NIL if x does not appear in $A$. Write pseudocode for linear search, which scans through the array from beginning to end, looking for x. Using a loop invariant, prove that your algorithm is correct. Make sure that your loop invariant fulfills the three necessary properties. Consider the problem of adding two n-bit binary integers $a$ and $b$, stored in two $n$-element arrays $A[0:n-1]$ and $B[0:n-1]$, where each element is either $0$ or $1$. $a = \\sum_{i=0}^{n-1} A[i] \\cdot 2^i$, and $b = \\sum_{i=0}^{n-1} B[i] \\cdot 2^i$. The sum $c = a + b$ of the two integers should be stored in binary form in an ($n+1$)-element array $C[0:n]$, where $c = \\sum_{i=0}^{n} C[i] \\cdot 2^i$. Write a procedure ADD-BINARY-INTEGERS that takes as input arrays $A$ and $B$, along with the length n, and returns array C holding the sum. Express the function $n^3/1000+100n^2-100n+3$ in terms of $\\Theta$-notaion. $\\Theta (n^3)$ Consider sorting $n$ numbers stored in array $A[1:n]$ by first finding the smallest element of $A[1:n]$ and exchanging it with the element in $A[1]$. Then find the smallest element of $A[2:n]$, and exchange it with $A[2]$. Then find the smallest element of $A[3:n]$, and exchange it with $A[3]$. Continue in this manner for the first $n-1$ elements of $A$. Write pseudocode for this algorithm, which is known as selection sort. What loop invariant does this algorithm maintain? Why does it need to run for only the first $n-1$ elements, rather than for all $n$ elements? Give the worst-case running time of selection sort in $\\Theta$-notation. Is the best-case running time any better? Consider linear search again (see Exercise 2.1-4). How many elements of the input array need to be checked on the average, assuming that the element being searched for is equally likely to be any element in the array? How about in the worst case? Using $\\Theta$-notation, give the average-case and worst-case running times of linear search. Justify your answers. How can you modify any sorting algorithm to have a good best-case running time modidy combine with insertion sort ,such as check whether each element in array is smaller(bigger) than the righthand element, since it produce a best - case running time of $\\Theta(n)$, the same as travel throught the array, which is a neccessary step for sorting. Using Figure 2.4 as a model, illustrate the operation of merge sort on an array initially containing the sequence $\\langle 3, 41, 52, 26, 38, 57, 9, 49 \\rangle$. $$ [3|41|52|26|38|57|9|49]$$ $$\\downarrow$$ $$[3|41|52|26] \\quad [38|57|9|49]$$ $$\\downarrow$$ $$[3|41] \\quad [52|26] \\quad [38|57] \\quad [9|49]$$ $$\\downarrow$$ $$[3] \\quad [41] \\quad [52] \\quad [26] \\quad [38] \\quad [57] \\quad [9] \\quad [49]$$ $$\\downarrow$$ $$[3|41] \\quad [26|52] \\quad [38|57] \\quad [9|49]$$ $$\\downarrow$$ $$[3|26|41|52] \\quad [9|38|49|57]$$ $$\\downarrow$$ $$[3|9|26|38|41|49|52|57]$$ The test in line 1 of the MERGE-SORT procedure reads \"if $p \\geq r$\" rather than \"if $p \\neq r$.\" If MERGE-SORT is called with $p > r$, then the subarray $A[p:r]$ is empty. Argue that as long as the initial call of MERGE-SORT(A, 1, n) has $n \\geq 1$, the test \"if $p \\neq r$\" suffices to ensure that no recursive call has $p > r$. $$ \\begin{aligned} if \\quad p \\leq r\\cr if \\quad p < r\\cr q = \\lfloor (p+r)/2 \\rfloor\\cr q \\geq p\\cr q+1 \\leq r\\cr MERGE-SORT(A,p,q)\\cr MERGE-SORT(A,q+1,r)\\cr \\text{else return}\\cr \\text{no recursive call has p>r}\\cr p \\leq r \\text{ holds in new recursive call}\\cr \\end{aligned} $$ $$ \\begin{aligned} MERGE-SORT(A,1,n)\\cr n \\geq 1\\cr p \\leq r holds\\cr \\end{aligned} $$ \"if $p \\neq r$\" suffices to ensure that no recursive call has $p > r$. State a loop invariant for the while loop of lines 12\u201318 of the MERGE procedure. Show how to use it, along with the while loops of lines 20\u201323 and 24\u201327, to prove that the MERGE procedure is correct. loop invariant: at the start of each iteration, $A[p:k-1]$ consist of the smallest elements of union of $L$ and $R$ at sort order. when the while loop of lines 12-18 finish $A[p:k-1]$ consist of the smallest elements of union of $L$ and $R$ at sort order, either while loop of lines 20\u201323 or 24\u201327 will be run, $L$ or $R$ has the biggest rest elements in sort order.when the second while loop finish, they are added to the end of A, and A consist of all elements at sort order. Use mathematical induction to show that when $n \\geq 2$ is an exact power of 2, the solution of the recurrence $$ \\begin{aligned} T(n) = \\begin{cases} 2 & \\text{if } n = 2, \\cr 2T(n/2) + n & \\text{if } n > 2 \\end{cases} \\end{aligned} $$ is $T(n) = n \\lg n$. $$ \\begin{aligned} T(n) & = T(2^t)\\cr & = 2T(2^{t-1}) + 2^t\\cr & = 4T(2^{t-2}) + 2\\cdot{2^t}\\cr & = 2^{t-1}T(2) + 2^t\\cdot{\\lg 2^t }\\cr & = n\\lg n \\end{aligned} $$ You can also think of insertion sort as a recursive algorithm. In order to sort $A[1:n]$, recursively sort the subarray $A[1:n-1]$ and then insert $A[n]$ into the sorted subarray $A[1:n-1]$. Write pseudocode for this recursive version of insertion sort. Give a recurrence for its worst-case running time. Referring back to the searching problem (see Exercise 2.1-4), observe that if the subarray being searched is already sorted, the searching algorithm can check the midpoint of the subarray against v and eliminate half of the subarray from further consideration. The binary search algorithm repeats this procedure, halving the size of the remaining portion of the subarray each time. Write pseudocode, either iterative or recursive, for binary search. Argue that the worst-case running time of binary search is $\\Theta (\\lg n)$. iterative: recursive: the worst-case is no x in $A$, BINARY SEARCH will end al condition l>r which occurs when A[l:r] is empty. since BINARY SEARCH halving the size of A[l:r] each time, it cost $\\Theta (\\lg n)$ to make A[l:r] empty The while loop of lines 5-7 of the INSERTION-SORT procedure in Section 2.1 uses a linear search to scan (backward) through the sorted subarray $A[1:j-1]$. What if insertion sort used a binary search (see Exercise 2.3-6) instead of a linear search? Would that improve the overall worst-case running time of insertion sort to $\\Theta(n \\lg n)$ It can save search time, but can not save movement time. At worst -case, it cost $j-1 = \\Theta(j)$ time to move elements bigger than $A[j]$ in $A[1:j-1]$.The same as the original INSERTION-SORT Therefore, that improve the overall worst-case running time of insertion sort to $\\Theta(n\\lg n)$ Dscribe Describe an algorithm that, given a set $S$ of $n$ integers and another integer $x$, determines whether $S$ contains two elements that sum to exactly $x$. Your algorithm should take $\\Theta (n \\lg n)$ time in the worst case. The time complexity of the algorithm is $\\Theta (n \\lg n)+\\Theta (n)$ Although merge sort runs in $\\Theta(n \\lg n)$ worst-case time and insertion sort runs in $\\Theta (n^2)$ worst-case time, the constant factors in insertion sort can make it faster in practice for small problem sizes on many machines. Thus it makes sense to coarsen the leaves of the recursion by using insertion sort within merge sort when subproblems become sufficiently small. Consider a modification to merge sort in which $n/k$ sublists of length $k$ are sorted using insertion sort and then merged using the standard merging mechanism, where $k$ is a value to be determined. a. Show that insertion sort can sort the $n/k$ sublists, each of length k, in $\\Theta (nk)$ worst-case time. b. Show how to merge the sublists in $\\Theta (n \\lg (n/k))$ worst-case time. c. Given that the modified alogotithm runs in $\\Theta (nk+n \\lg (n/k))$ worst-case time, what is the largest value of k as a function of n for which the modified algorithm has the same running time as standard merge sort, in terms of $\\Theta$-notation? d. How should you choose k in practice? Each sublists of length can be sorted in $\\Theta (k^2)$, there are n/k sublists, so in n $n/k \\cdot \\Theta(k^2) = \\Theta (nk)$ sorst-case time. Each level of merging need to merge $n$ elements, and there are $\\lg (n/k)$levels(since we merge each two sublists in one in each level). So the worse-case time to merge the sublists is $\\Theta (n \\lg (n/k))$. $$ \\begin{aligned} & if \\quad \\Theta (nk+n \\lg (n/k)) = \\Theta (n \\lg n)\\cr & \\exist \\quad c_{1} , c_{2} , n_{0} \\quad satisfies:\\cr & c_{1}(n \\lg n) \\leq nk+n \\lg (n/k) \\leq c_{2}(n \\lg n) \\quad for \\quad n > n_{0}\\cr &(c_{1}-1)(\\lg n )\\leq k - \\lg k \\leq (c_{2}-1)(\\lg n ) \\quad for \\quad n > n_{0}\\cr & since \\quad lg k \\llless k, \\quad \\text{we have } k = \\Theta (\\lg n)\\cr \\end{aligned} $$ Choose k be the largest size of the sublists that insertion-sort is faster than merge-sort. Bubblesort is a popular, but inefficient, sorting algorithm. It works by repeatedly swapping adjacent elements that are out of order. The procedure BUBBLESORT sorts array $A[1:n]$. a. Let $A'$ denote the array $A$ after BUBBLESORT> (A, n) is executed. To prove that it terminates and that $$ A'[1] \\leq A'[2] \\leq \\cdots \\leq A'[n] $$ In order to show that BUBBLESORT actually sorts, what else do you need to prove? The next two parts prove inequality (2.5). b. State precisely a loop invariant for the for loop in lines 2-4, and prove that this loop invariant holds. Your proof should use the structure of the loop-nvariant proof presented in this chapter. c. Using the termination condition of the loop invariant proved in part (b), state a loop invariant for the for loop in lines 1-4 that allows you to prove inequal ity (2.5). Your proof should use the structure of the loop-invariant proof presented in this chapter. d. What is the worst-case running time of BUBBLESORT? How does it compare with the running time of INSERTION-SORT? $A'$ consist of all the elements of $A$ loop invariant: At the start of each iteration of $for$ loop in lines 2-4, $A[j:n]$ consist of the elements originally in $A[j:n]$ before entering the loop but possibly in a different order, and $A[j]$ is the smallest one of them. Initialization: $j=n$, $A[n]$ consist of the elements originally in A$[n]$ and it is the smallest. Maintenance: During the iteration, if$A[j-1]<A[j]$, we swap them. So A[j-1] will be the smallest and A[j-1:n] consist of the elements originally in $A[j-1:n]$. Decresementing j preserves the loop invariant. Termination: the loop termination when $j = i$, $A[i:n]$ consist of the elements originally in $A[i:n]$ before entering the loop but possibly in a different order, and $A[i]$ is the smallest one of them. Loop invariant: At the start of each iteration of the for loop of lines 1-4, the subarray $A[1..i \u2212 1]$ consists of the $i - 1$ smallest elements in $A[1..n]$ in sorted order. $A[i..n]$ consists of the $n - i + 1$ remaining elements in $A[1..n]$. Initialization: Initially the subarray $A[1..i \u2212 1]$ is empty and trivially this is the smallest element of the subarray. Maintenance: From part (b), after the execution of the inner loop, $A[i]$ will be the smallest element of the subarray $A[i..n]$. And in the beginning of the outer loop, $A[1..i \u2212 1]$ consists of elements that are smaller than the elements of $A[i..n]$, in sorted order. So, after the execution of the outer loop, subarray $A[1..i]$ will consists of elements that are smaller than the elements of $A[i + 1..n]$, in sorted order. Termination: The loop terminates when $i = A.length$. At that point the array $A[1..n]$ will consists of all elements in sorted order. The $i$th iteration of the for loop of lines 1-4 will cause $n \u2212 i$ iterations of the for loop of lines 2-4, each with constant time execution, so the worst-case running time of bubble sort is $\\Theta(n^2)$ which is same as the worst-case running time of insertion sort. You are given the coefficents $a_0; a_1; a_2; \\dots; a_n$ of a polynomial $$ \\begin{aligned} P(x) & = \\sum_{k=0}^n a_kx^k \\cr &=a_0+a_1x+a_2x^2+\\cdots +a_{n-1}x^{n-1}+a_nx^n, \\end{aligned} $$ and you want to evaluate this polynomial for a given value of x. Horner's rule says to evalusate the polynomial according to this > parenthesization: $$ P(x) = a_0 + x ( a_1 + x ( a_2 + \\cdots + x ( a_{n-1} + x a_n) > \\cdots )). $$ The procedure HORNER implements Horner**\u2019s rule to > evaluate $P(x)$, > givem the coefficients $a_0,a_1,a_2,\\dots,> a_n$ in an array $A[0:n]$ and > the value of $x$. a. In terms of $\\Theta$-notation, what is the tunning time of this procedure? b. Write pseudocode to implement the naive polynomial-evaluation algorithm that computes each term of the polynomial from scratch. What is the running time of this algorithm? How does it compare with HORNER? c. Consider the following loop invariant for the procedure HORNER: At the start of each iteration of the for loop of lines 2-3, $$ p = \\sum _{k=0}^{n-(i+1)} A[k+i+1] \\cdot x^k $$ Interpret a summation with no terms at equaling 0. Following the structure of th loop-invariant proof presented in this chapter, use this loop invariant to show that, at termination, $p = \\sum _{k=0} ^{n} A[k] \\cdot x^k$ $\\Theta(n)$ pseudocode: $\\text{running time:} \\Theta (n^2)$, HORNER beat it asymptotically. Initialization: $i=n, p=\\sum_{k=0}^{n-(n+1)} A[k+n+1] \\cdot{x^k} = 0$, loop invariant holds. Maintenance: in the end of each iteration: $$ \\begin{aligned} p & = A[i] + x \\cdot{\\sum_{k=0}^{n-(i+1)} A[k+i+1] \\cdot {x^k}} \\cr & = A[i] + \\sum_{k=0}^{n-(i+1)} A[k+i+1] \\cdot{x^{k+1}}\\cr & = A[i] \\cdot{x^0} + \\sum_{k=1}^{n-i} A[k+i] \\cdot{x^{k}}\\cr & = \\sum_{k=0}^{n-i} A[k+i] \\cdot{x^k}\\cr \\end{aligned} $$ Decrementing i preserves the loop invariant. Termination: The loop terminates at $i=\u22121$. If we substitute, $$ p = \\sum_{k=0}^{n-(i+1)} A[k+i+1] \\cdot{x^k} = \\sum_{k=0}^{n} A[k] \\cdot{x^k} $$ Let $A[1:n]$ be an array of n distinct numbers. If $i < j$ and $A[i] > A[j]$, then the pair $(i,j)$ is called an inversion of A. a. List the five inversions of the array $\\langle 2,3,8,6,1 \\rangle$. b. What array with elements from the set $\\langle 1,2,\\dots ,n \\rangle$. has the most inversions? How many does it have? c. What is the relationship between the running time of insertion sort and the number of inversions in the input array? Justify your answer. d. Give an algorithm that determines the number of inversions in any permutation. on n elementa in $\\Theta (n \\lg n)$ worse-case time. (Hint: Modify merge sort.) a. (1,5),(2,5),(3,4),(3,5),(4,5) b. $\\langle n,n-1,\\dots ,1\\rangle \\text{ totally has } {n \\choose 2} = n(n-1)/2 \\text{ inversions}$ c. The running time of insertsion sort is a constant times the number of inversions, since each time exchanging elements in insertion sort reduces Inversions num by 1. d. Modify the lower-bound argument for insertion sort to handle input sizes that are not necessarily a multiple of 3. At least $\\lfloor \\frac n 3 \\rfloor \\geq \\frac 3n-1$ elements have to pass through at least $\\lfloor \\frac n 3 \\rfloor \\geq \\frac 3n-1$ the time taken by INSERTION-SORT in the worst case is at least proportional to $(\\frac 3n-1)(\\frac 3n-1)=\\Omega(n^2)$ Using reasoning similar to what we used for insertion sort, analyze the running time of the selection sort algorithm from Exercise 2.2-2. Inner loop in selection sort iterates $(n-i+1)$ times, for i = 1 to n-1, so the running time of selection sort is $\\sum_{i=1}^{n-1} (n-i+1)=(n+2)(n-1)/2=\\Theta(n^2)$ Suppose that $\\alpha$ is a fraction in the range $0 < \\alpha < 1$. Show how to generalize the lower-bound argument for insertion sort to consider an input in which the $\\alpha n$ largest values start in the first $\\alpha n$ positions. What additional restriction do you need to put on $\\alpha$? What value of $\\alpha$ maximizes the number of times that the $\\alpha n$ largest values must pass through each of the middle $(1-2 \\alpha)$ array positions? At least $\\alpha n$ values have to pass through at least $(1-2 \\alpha)n$ times. insertion sort is at least proportional to $(\\alpha n)(1-2 \\alpha)n = \\alpha(1-2\\alpha)n^2 = \\Omega(n^2)$ if $\\alpha(1-2\\alpha)=\\Omega(1)$ $\\max(\\alpha(1-2\\alpha))=\\frac 1 8$ when $\\alpha = \\frac 1 4$ Let $f(n)$ and $g(n)$ be the asympotically nonnegative functions. Using the basic definition of $\\Theta$-notation, prove that max ${f(n),g(n)} = \\Theta (f(n) + g(n))$. $$ \\begin{aligned} \\text{Let } n_0 \\text{ be a value that makes f(n) greater than 0 for n >}n_0\\cr \\text{Then for }n>n_0\\cr \\frac 1 2 (f(n)+g(n)) \\leq \\max(f(n),g(n)) \\leq 1(f(n)+g(n))\\cr \\max(f(n),g(n))=\\Theta(f(n)+g(n))\\cr \\end{aligned} $$ Tips: I will omit details like \"$\\text{Let } n_0 \\text{ be a value that makes f(n) greater than 0 for n >}n_0$\"\"$\\exist c,n_0$\" and \"for $n > n_0$\" for convenience from here. Explain why the statement, \"The running time of algorithm A is at least $O(n^2)$,\"if meaningless. $A = O(n^2)$ means $\\exist n_0,c:A \\leq cn^2$ for $n>n_0$. $A$ is at least $O(n)$ means $A \\geq \\text{a function that }\\leq cn^2$ You can say A can be any function or no function. Both are meaningless. Is $2^{n+1}=O(2^n)$? Is $2^{2n}=O(2^n)$? Prove Theorem 3.1. $$ \\begin{aligned} & f(n) = O(g(n)) \\quad and \\quad f(n)=\\Omega(g(n))\\cr & \\implies c_1 g(n)\\leq f(n) \\leq c_2g(n)\\cr & \\implies f(n) = \\Theta(g(n))\\cr & f(n) = \\Theta(g(n))\\cr & \\implies c_1 g(n)\\leq f(n) \\leq c_2g(n)\\cr & \\implies f(n) = O(g(n)) \\quad and \\quad f(n)=\\Omega(g(n))\\cr \\end{aligned} $$ We have $f(n) = \\Theta(g(n))$ if and only if $f(n) = \\Omega(g(n))$ and $f(n) = O(g(n))$ Prove that the running time of an algorithm is $\\Theta(g(n))$ if and only if its worst-case running time $O(g(n))$ and its best-casse running time is $\\Omega (g(n))$. $$ \\begin{aligned} A=\\Theta(g(n)) & \\implies c_1 g(n) \\leq A \\leq c_2 g(n) \\text{ for all cases}\\cr & \\implies \\text{worst-case running time}= O(n)\\cr & \\And \\text{best-case running time}=\\Omega(g(n))\\cr \\text{worst-case running time}= O(n) & \\implies A \\leq c_2 g(n) (1)\\cr \\text{best-case running time}=\\Omega(g(n)) & \\implies c_1 g(n) \\leq A (2)\\cr (1) \\And (2) & \\implies A=\\Theta(g(n))\\cr \\end{aligned} $$ Prove that $o(g(n)) \\cap \\omega (g(n))$ is empty set. $f(n)c(g(n)) \\implies NIL$"},{"location":"Chap03/3.2/#32-7","title":"3.2-7","text":" We can extend our notation to the case of two parameters n and m that can go to $\\infty$ independently at different rates. For a given function $g(n,m)$, we denote by $O(g(n,m))$ the set of functions $$ \\begin{aligned} O(g(n, m)) =\\lbrace f(n, m): & \\text{ there exist positive constants } c, n_0, \\text{ and } m_0 \\cr & \\text{ such that } 0 \\leq f(n, m) \\leq cg(n, m)\\cr & \\text{ for all } n \\geq n_0 \\text{ or } m \\geq m_0.\\rbrace\\cr \\end{aligned} $$ Give corresponding definitionss for $\\Omega(g(n,m))$ and $\\Theta(g(n,m))$. $$ \\begin{aligned} \\Theta(g(n, m)) =\\lbrace f(n, m): & \\text{ there exist positive constants } c_1,c_2, n_0, \\text{ and } m_0 \\cr & \\text{ such that } 0 \\leq c_1g(n,m)\\leq f(n, m) \\leq c_2g(n, m)\\cr & \\text{ for all } n \\geq n_0 \\text{ or } m \\geq m_0.\\rbrace\\cr \\Omega(g(n, m)) =\\lbrace f(n, m): & \\text{ there exist positive constants } c, n_0, \\text{ and } m_0 \\cr & \\text{ such that } 0 \\leq cg(n,m)\\leq f(n, m)\\cr & \\text{ for all } n \\geq n_0 \\text{ or } m \\geq m_0.\\rbrace\\cr \\end{aligned} $$ Show that if $f(n)$ and $g(n)$ are monotonically increasing functions, then so are the functions $f(n)+g(n)$ and $f(g(n))$, and if $f(n)$ and $g(n)$ are in addition nonnegative, then $f(n)\\cdot g(n)$ $$ \\begin{aligned} \\forall n_1 \\geq n_2: & f(n_1) \\geq f(n_2) \\And g(n_1) \\geq g(n_2)\\cr \\implies & f(n_1)+g(n_1) \\geq f(n_2)+g(n_2)\\cr \\implies & f(g(n_1) \\geq g(n_2)) \\geq f(g(n_2))\\cr & if \\quad f(n) \\geq 0 \\And g(n) \\geq 0\\cr \\implies & f(g(n_1)\\geq g(n_2)\\geq 0)\\geq f(g(n_2)) \\end{aligned} $$ Prove that $\\lfloor \\alpha n \\rfloor +\\lceil (1-\\alpha )n \\rceil = n$ for any integer n and real number $\\alpha$ in the range $0 \\leq \\alpha \\leq 1$ $$ \\begin{aligned} \\lfloor \\alpha n \\rfloor +\\lceil (1-\\alpha )n \\rceil & = \\lfloor \\alpha n \\rfloor +\\lceil -\\alpha n\\rceil + n\\cr &=\\lfloor \\alpha n \\rfloor - \\lfloor \\alpha n \\rfloor +n\\cr &=n \\end{aligned} $$ Use equation (3.14) or other means to show that $(n+o(n))^k = \\Theta(n^k)$ for any real constant k. Conclude that $\\lceil n \\rceil ^k$ and $\\lfloor n \\rfloor ^k = \\Theta(n^k)$. $$ \\begin{aligned} n^k\\leq(n+o(n))^k=n^k(1+\\frac {o(n)}n)^k \\leq n^k e^{\\frac {o(n)}n} \\leq en^k\\cr \\implies (n+o(n))^k = \\Theta(n^k)\\cr \\lceil n \\rceil ^k= (n+o(n))^k=\\Theta(n^k)\\cr \\lfloor n \\rfloor ^k = (n+o(n))^k=\\Theta(n^k)\\cr \\end{aligned} $$ Prove the following a. Equation(3.21). b. Equations(3.26)-(3.28). c. $\\lg(\\Theta(n))=\\Theta(\\lg n)$ $$ \\begin{aligned} n! = \\sqrt {2\\pi n}(\\frac n e)^n(1+\\Theta(\\frac{1}{n}))\\cr \\forall c>0,\\lim_{n \\to \\infin} \\frac {n!=\\frac{\\sqrt {2\\pi n}} {e^n}(1+\\Theta(\\frac 1 n))n^n} {cn^n} = \\lim_{n \\to \\infin} \\frac{\\sqrt {2\\pi n}} {e^n}(1+\\Theta(\\frac 1 n)) = 0\\cr \\implies n!=o(n^n)\\text{ (Equation(3.21))}\\cr \\lim_{n \\to \\infin} \\frac {n!=\\frac{\\sqrt {2\\pi n}} {e^n}(1+\\Theta(\\frac 1 n))n^n} {2^n} = \\lim_{n \\to \\infin }\\sqrt{2\\pi n}(1+\\Theta(\\frac 1 n))(\\frac{n}{2e})^n = \\infin \\cr \\implies n! =\\omega(n) \\text{ (Equation(3.27))}\\cr \\lg (n!)=n\\lg n+ \\frac{1}{2}\\lg(2\\pi n)+n\\lg e+\\lg(1+\\Theta(\\frac{1}{n})) = \\Theta(n\\lg n)\\text{ (Equation(3.28))}\\cr \\end{aligned} $$ $$ \\begin{aligned} \\forall f(n) \\in \\Theta(n)\\cr c_3\\lg n\\leq \\lg {c_1 n} \\leq \\lg(f(n))\\leq \\lg{c_2 n} \\leq c_4\\lg n\\cr \\implies \\lg(\\Theta(n))=\\Theta(\\lg n) \\end{aligned} $$ Is the function $\\lceil \\lg n \\rceil !$ polynomially bounded? Is the function $\\lceil \\lg \\lg n \\rceil !$ polynomially bounded? Polynomially bounded:$f(n) \\leq cn^k \\implies \\lg(f(n))=ck\\lg(n)\\implies \\lg(f(n))=O(\\lg n)$ $$ \\begin{aligned} \\lg(\\lceil \\lg n\\rceil !)& =\\Theta(\\lceil \\lg n\\rceil \\lg(\\lceil \\lg n\\rceil))\\cr &=\\Theta(\\lg n \\lg(\\lg n))\\cr &\\implies \\lceil \\lg n\\rceil !\\text{ is not polynomially bouded}\\cr \\lg \\lceil \\lg \\lg n \\rceil !&=\\Theta(\\lceil \\lg \\lg n\\rceil \\lg\\lceil\\lg\\lg n\\rceil)\\cr & =O(\\lg^2\\lg n)\\cr & =O(\\lg n)\\cr & \\implies\\lceil\\lg\\lg n\\rceil \\text{ is Polynomially bounded} \\end{aligned} $$ Which is asymptotically larger: $\\lg(\\lg^{\\ast}n)$ or $\\lg^{\\ast}(\\lg n)$? $$ \\begin{aligned} n=2^k\\cr \\lim_{n\\to\\infin}\\frac{\\lg(\\lg^{\\ast}n)}{\\lg^{\\ast}(\\lg n)}& = \\lim_{k\\to\\infin}\\frac{\\lg(\\lg^{\\ast} 2^k)}{\\lg^{\\ast}(\\lg 2^k)}\\cr &= \\lim_{k\\to\\infin} \\frac{\\lg(1+\\lg^{\\ast}k)}{\\lg^{\\ast}k}\\cr & =\\lim_{t\\to \\infin}\\frac{\\lg(1+t)}{t}\\cr & = 0 \\end{aligned} $$ $\\lg^{\\ast}(\\lg n)$ is larger. Show that the golden ratio $\\phi$ and its conjugate $\\hat{\\phi}$ both satisfy the equation $x^2 = x + 1$ $$ \\begin{aligned} ax^2+bx+c=0\\implies x=\\frac{-b\\pm\\sqrt{b^2-4ac}}{2a}\\cr x^2-x-1=0 \\implies x=\\frac{1 \\pm\\sqrt{5}}{2}=\\phi \\And \\hat\\phi \\end{aligned} $$ prove by induction that the i-th Fibonacci num satisfies the equation $$ F_i = (\\phi^i - \\hat{\\phi}^i)/\\sqrt{5} $$ where $\\phi$ is the golden ratio and $\\hat{\\phi}$ is its conjugate. For inductive case: $$ \\begin{aligned} & \\text{truth: }\\phi\\hat\\phi=-1,\\phi+\\hat\\phi=1\\cr \\text{assume }:\\cr F_{i-1} &= (\\phi^{i-1} - \\hat{\\phi}^{i-1})/\\sqrt{5}\\cr F_{i} &= (\\phi^{i} - \\hat{\\phi}^{i})/\\sqrt{5}\\cr \\text{then:}\\cr F_{i-1}+F_i & =(\\phi^{i-1} - \\hat{\\phi}^{i-1})/\\sqrt{5}+(\\phi+\\hat\\phi)(\\phi^i - \\hat{\\phi}^i)/\\sqrt{5}\\cr & = (\\phi^{i+1}-\\hat\\phi^{i+1})/\\sqrt 5+((1+\\phi\\hat\\phi)\\phi^{i-1}-(1+\\phi\\hat\\phi)\\hat\\phi^{i-1})/\\sqrt 5\\cr & =F_{i+1}\\cr \\end{aligned} $$ For base case: $$ \\begin{aligned} (\\phi^{1} - \\hat{\\phi}^{1})/\\sqrt{5}=1=F_1\\cr (\\phi^{2} - \\hat{\\phi}^{2})/\\sqrt{5}=(\\phi^{1}+1 - (\\hat{\\phi}^{1}+1))/\\sqrt{5}=1=F_2\\cr \\end{aligned} $$ Tips: I will omit details like\"assume $F_i$ $F_{i-1}$\" and proof of base case from now on for convenience. Show that $k\\lg k = \\Theta(n)$ implies $k = \\Theta(n/\\lg n)$. $$ \\begin{aligned} & k\\lg k=\\Theta(n)\\cr \\implies & n=\\Theta(k\\lg k)\\cr \\implies & \\lg n = \\Theta(\\lg k + \\lg\\lg k)=\\Theta(\\lg k)\\cr \\implies & n/\\lg n = \\Theta(k\\lg k)/\\Theta(\\lg k)=\\Theta (k)\\cr \\implies & k = \\Theta(n/\\lg n) \\end{aligned} $$ Let $$ P(n) = \\sum_{i=0}^d a_i n^i, $$ where $a_d > 0$, be a degree-d polynomial in n, and let k be a constant. Use the definitions of the asymptotic notations to prove the following properties. a. if $k \\geq d$, then $p(n)=O(n^k)$ b. if $k \\leq d$, then $p(n)=\\Omega(n^k)$ c. if $k =d$, then $p(n)=\\Theta(n^k)$ d. if $k > d$, then $p(n)=o(n^k)$ e. if $k < d$, then $p(n)=\\omega(n^k)$ To prove $p(n)=O(n^k)$, we only need to prove: $$ \\begin{aligned} \\sum_{i=0}^{d}a_in^i \\leq cn^k (\\text{ for }n \\geq n_0)\\cr \\end{aligned} $$ Which is implied by $$ \\begin{aligned} c \\geq \\sum_{i=0}^da_in^{i-k} (\\text{ for }n \\geq n_0)\\cr \\end{aligned} $$ since $\\sum_{i=0}^{d}a_i \\geq \\sum_{i=0}^da_in^{i-k} (\\text{ for }n \\geq 1)$ $\\text{choose } c = \\sum_{i=0}^{d}a_i$ and $n_0 = 1$ satisfies. We only need to prove: $$ \\sum_{i=0}^{d}a_in^i \\geq cn^k $$ which is implied by: $$ \\begin{aligned} c \\leq \\sum_{i=0}^da_in^{i-k}\\cr c=a_d \\leq \\sum_{i=0}^da_in^{i-k}\\cr n_0=1 \\end{aligned} $$ We only need to prove: $$ c_1n^d\\leq \\sum_{i=0}^{d}a_in^i \\leq c_2n^{d} $$ which is implied by: $$ \\begin{aligned} c_1 \\leq \\sum_{i=0}^da_in^{i-d} \\leq c_2\\cr c_1=a_d \\leq \\sum_{i=0}^da_in^{i-d} \\leq \\sum_{i=0}^{d} a_i=c_2\\cr n_0=1 \\end{aligned} $$ We only need to prove: $$ \\begin{aligned} \\forall c >0,\\exist n_0 \\text{ satisfies: }\\cr \\sum_{i=0}^{d} a_i n^i < cn^k (\\text{ for } n > n_0)\\cr \\impliedby c > \\sum_{i=0}^{d} a_in^{i-k}\\cr \\text{since }\\sum_{i=0}^{d} a_in^{d-k} \\geq \\sum_{i=0}^{d} a_in^{i-k}\\cr n_0=\\sqrt[d-k]{\\frac {c} {\\sum_{i=0}^d a_i}} \\end{aligned} $$ We only need to prove: $$ \\begin{aligned} \\forall c >0,\\exist n_0 \\text{ satisfies: }\\cr \\sum_{i=0}^{d} a_in^i > cn^k (\\text{ for }n>n_0)\\cr \\impliedby c < \\sum_{i=0}^{d} a_in^{i-k}\\cr \\text{since } a_d n^{d-k} \\leq \\sum_{i=0}^{d} a_in^{i-k}\\cr n_0=\\sqrt[d-k]{\\frac {c} {a_d}} \\end{aligned} $$ Indicate, for each pair of expressions $(A, B)$ in the table below whether A is $O, o,\\Omega,\\omega$, or $\\Theta$ of B. Assume that $k \\geq 1,\\epsilon >,$ and $c>1$ are constants. Write your answer in the foem of the table with \"yes\" or \"no\" written in each box. a. Rank the following fuctions vy oefer of growth. That is, find an arrangement $g_1,g_2,\\dots,g_{30}$ of the functions satisfying $g_1=\\Omega(g_2),g_2=\\Omega(g_3),\\dots ,g_{29}=\\Omega(g_{30})$. Partion your list into equivalence classes such that functions $f(n)$ and $g(n)$ belong to the same class if and only if $f(n)$ = $\\Theta g(n)$ $$ \\begin{array}{} & \\lg(\\lg^{\\ast}n) & 2^{\\lg^{\\ast}n} & (\\sqrt {2})^{\\lg n} & n^2 & n! & (\\lg n)!\\cr & (3/2)^n & n^3 & \\lg^2n & \\lg(n!) & 2^{2^n} & n^{1/\\lg n}\\cr & \\ln\\ln n & \\lg^{\\ast}n & n\\cdot 2^n & n^{\\lg\\lg n} & \\ln n & 1\\cr & 2^{\\lg n} & (\\lg n)^{\\lg n} & e^n & 4^{\\lg n} & (n+1)! & \\sqrt{\\lg n}\\cr & \\lg^{\\ast}(\\lg n) & 2^{\\sqrt{2 \\lg n}}& n & 2^n& n\\lg n & 2^{2^{n+1}}\\cr \\end{array} $$ b. Give an example of a single nonnegative function $f(n)$ such that for all functions $g_i(n)$ in part (a), $f(n)$ is neither $O(g_i(n))$ nor $\\Omega (g_i(n))$. $$ \\begin{aligned} 2^{2^{n+1}}\\cr 2^{2^n}\\cr (n+1)!\\cr n!\\cr e^n\\cr n\\cdot 2^n\\cr 2^n\\cr (3/2)^n\\cr n^{\\lg\\lg n} \\And (\\lg n)^{\\lg n}\\cr (\\lg n)!\\cr n^3\\cr n^2 \\And 4^{\\lg n} \\cr \\lg(n!) \\And n\\lg n\\cr 2^{\\lg n} \\And n\\cr (\\sqrt {2})^{\\lg n}\\cr 2^{\\sqrt{2\\lg n}}\\cr \\lg^2 n\\cr \\ln n\\cr \\sqrt{\\lg n}\\cr \\ln\\ln n\\cr 2^{\\lg^{\\ast}n}\\cr \\lg^{\\ast}n \\And \\lg^{\\ast}(\\lg n)\\cr \\lg(\\lg^{\\ast}n) \\cr n^{1/\\lg n}\\And 1\\cr \\end{aligned} $$ $|\\sin{n}|\\cdot 2^{2^{2^{n+1}}}$ Let $f(n)$ and $g(n)$ be asymptotically positive functions. Prove or disprove each of the following conjectures. a. $f(n)=O(g(n))$ implies $g(n)=O(f(n))$. b. $f(n)+g(n)=\\Theta(\\min(f(n),g(n)))$. c. $f(n)=O(g(n))$ implies $\\lg(f(n))=O(\\lg(g(n)))$, where $\\lg g(n)\\geq 1$ and $f(n)\\geq 1$ for all sufficiently large $n$. d. $f(n)=O(g(n))$ implies $2^{f(n)} = O(2^{g(n)})$. e. $f(n)=O((f(n))^2)$. f. $f(n)=O(g(n))$ implies $g(n)=\\Omega(f(n))$. g. $f(n)=\\Theta(f(n/2))$. h. $f(n)+o(f(n))=\\Theta(f(n))$. Disprove: $n=O(n^2)$ but $n^2\\neq O(n)$ Disprove:$n+n^2\\neq \\Theta(\\min(n,n^2))$ prove: $$ \\begin{aligned} 0\\leq f(n)\\leq cg(n) & \\implies \\lg(f(n))\\leq \\lg c+\\lg(g(n))\\cr \\text{to prove: }&\\lg(f(n)) \\leq d\\lg(g(n))\\cr \\text{only need to prove: }& \\lg c + \\lg(g(n)) \\leq d\\lg(g(n))\\cr \\text{choose } & \\quad d = 1+\\lg c \\text{ satisfies} \\end{aligned} $$ prove: $$ \\begin{aligned} 0\\leq f(n)\\leq cg(n) \\implies 2^{f(n)}\\leq 2^{cg(n)}=2^c\\cdot c^{g(n) } \\end{aligned} $$ disprove: $\\frac{1}{n}\\neq O(\\frac{1}{n^2})$ prove: $0\\leq f(n)\\leq cg(n) \\implies g(n)=\\Omega (f(n))$ disprove:$2^n\\neq\\Theta(2^{\\frac{n}{2}})$ prove: $f(n)\\leq f(n)+o(f(n))\\leq 2f(n)$ since $0\\leq o(f(n))< f(n)$ Let $f(n)$ and $g(n)$ be asymptotically positive functions. Prove the following identities: a. $\\Theta(\\Theta(f(n)))=\\Theta(f(n))$. b. $\\Theta(f(n))+O(f(n))=\\Theta(f(n))$. c. $\\Theta(f(n))+\\Theta(g(n)) =\\Theta(f(n)+g(n))$. d. $\\Theta(f(n))\\cdot \\Theta(g(n))=\\Theta(f(n)\\cdot g(n))$. e. Argue that for any real constants $a_1,b_1>0$ and integer constants $k_1,k_2$, the following asymptotic bound holds: $(a_1 n)^{k_1}\\lg^{k_2}(a_2 n)=\\Theta(n^{k_1}\\lg^{k_2}n)$. $\\star$ f. Prove that for $S\\subseteq \\Z$, we have $$ \\sum_{k\\in S}\\Theta(f(k))=\\Theta(\\sum_{k\\in S}f(k)), $$ assuming that both sums converge. $\\star$ g. Show that for $S \\subseteq \\Z$, the following asymptotic bound does not necessarily hold, even assuming that both products converge, by giving a counterexample: $$ \\prod_{k\\in S}\\Theta(f(k))=\\Theta(\\prod_{k\\in S}f(k)). $$ $$ \\begin{aligned} g(n)=\\Theta(f(n))\\cr \\implies c_1 f(n)\\leq g(n) \\leq c_2 f(n)\\cr h(n)=\\Theta(g(n))\\cr \\implies c_1 d_1 f(n)\\leq d_1 g(n)\\leq h(n)\\leq d_2g(n)\\leq c_2 d_2 f(n)\\cr \\implies h(n)=\\Theta(f(n))\\cr \\implies\\Theta(\\Theta(f(n)))=\\Theta(f(n))\\cr \\end{aligned} $$ $$ \\begin{aligned} \\forall g(n)=\\Theta(f(n)) \\And h(n)=\\Theta(f(n))\\cr \\implies c_2 f(n) \\leq g(n)\\leq c_1 f(n) \\And 0 \\leq h(n)\\leq df(n)\\cr \\implies c_1f(n)\\leq g(n)+h(n)\\leq (c_1+d)f(n)\\cr \\implies g(n)+h(n)=\\Theta(f(n))\\cr \\implies \\Theta(f(n))+O(f(n))=\\Theta(f(n))\\cr \\end{aligned} $$ $$ \\begin{aligned} \\forall A(n)=\\Theta(f(n))\\And B(n)=\\Theta(g(n))\\cr \\implies c_1 f(n)\\leq A(n)\\leq c_2 f(n)\\And d_1 f(n)\\leq B(n) \\leq d_2 f(n)\\cr \\implies \\min(c_1,d_1)(f(n)+g(n))\\leq A(n)+B(n)\\leq \\max(c_2+d_2)(f(n)+g(n))\\cr \\implies \\Theta(f(n))+\\Theta(g(n)) =\\Theta(f(n)+g(n)) \\end{aligned} $$ $$ \\begin{aligned} \\forall A(n)=\\Theta(f(n))\\And B(n)=\\Theta(g(n))\\cr \\implies c_1 f(n)\\leq A(n)\\leq c_2 f(n)\\And d_1 f(n)\\leq B(n) \\leq d_2 f(n)\\cr \\implies c_1 d_1f(n)\\cdot g(n)\\leq A(n)\\cdot B(n)\\leq c_2 d_2f(n)\\cdot g(n)\\cr \\implies \\Theta(f(n))\\cdot \\Theta(g(n)) =\\Theta(f(n)\\cdot g(n)) \\end{aligned} $$ $$ \\begin{aligned} \\text{we only need to prove: }c_1 n^{k_1}\\lg^{k_2}n\\leq (a_1 n)^{k_1}\\lg^{k_2}(a_2 n)\\leq c_2 n^{k_1}\\lg^{k_2}n\\cr \\text{which is implied by: }d_1 lg^{k_2}n\\leq\\lg^{k_2}(a_2 n)\\leq d_2\\lg^{k_2}n\\cr \\text{which is implied by: }e_1\\lg n\\leq \\lg a+\\lg n \\leq e_2 \\lg n\\cr \\text{which is implied by: }e_1\\leq \\lg a/\\lg n +1 \\leq e_2\\cr \\text{choose }e_1=1/2 \\quad e_2=2 \\quad n_0=\\lg max(a^2,2^{\\lg a}) \\text{ satisfies}\\cr \\end{aligned} $$ $$ \\begin{aligned} g(k)=\\Theta(f(k))\\cr \\implies \\sum_{k\\in S} c_kf(k)\\leq \\sum_{k\\in S} g(k)\\leq \\sum_{k\\in S} d_kf(k)\\cr \\implies \\min( c_k) \\sum_{k\\in S} f(k)\\leq \\sum_{k\\in S} g(k)\\leq \\max(d_k)\\sum_{k\\in S} f(k)\\cr \\implies \\sum_{k\\in S}\\Theta(f(k))=\\Theta(\\sum_{k\\in S}f(k))\\cr \\end{aligned} $$ $\\prod_{k\\in S} \\frac 1 2=\\prod_{k\\in S} \\Theta(\\frac{1}{2^n})=\\Theta(0)\\neq \\Theta(\\prod_{k\\in S} 1)=\\Theta(1)$ Some authors define $\\Omega$-notation in a slightly different way than this textbook does. We'll use the nomenclature $\\stackrel{\\infty}{\\Omega}$ (read \"omega infinity\") for this alternative definition. We say that f(n) = $\\stackrel{\\infty}{\\Omega}(g(n))$if there exists a positive constant $c$ such that $f(n) \\geq cg(n) \\geq 0$ for infinitely many integers n. a. Show that for any two asymptotically nonnegative functions $f(n)$ and $g(n)$, we have $f(n)=O(g(n))$ or $f(n)=\\stackrel{\\infty}{\\Omega}(g(n))$ (or both). b. Show that there exist two asymptotically nonnegative functions $f(n)$ and $g(n)$ for which neither $f(n) = O(g(n))$ nor $f(n) = \\Omega(g(n))$ holds. c. Describe the potential advantages and disadvantages of using $\\stackrel{\\infty}{\\Omega}$-notation instead of $\\Omega$-notation to characterize the running times of programs. d. Some authors also define $O$ in a slightly different manner. We\u2019ll use $O'$ for the alternative definition: $f(n)=O'(g(n))$ if and only if $|f(n)|=O(g(n))$. What happens to each direction of the \"if and only if\" in Theorem 3.1 on page 56 if we substitute $O'$ for $O$ but still use $\\Omega$? e Some authors define $\\tilde{O}$(read \"soft-oh\") to mean O with logarithmic factors ignored: $$ \\begin{aligned} \\tilde{O}(g(n))=\\lbrace f(n) : & \\text{ there exist positive constants }c, k, \\text{ and } n_0 \\text{ such that }\\cr & 0\\leq f(n)\\leq cg(n)\\lg^k(n) \\text{ for all } n\\geq n_0\\rbrace .\\cr \\end{aligned} $$ Define $\\tilde{\\Omega}$ and $\\tilde{\\Theta}$ in a similar manner. Prove the corresponding analog to Theorem 3.1. $$ \\begin{aligned} \\text{if } f(n)\\neq O(g(n))\\cr \\text{there must be infinite integers that }f(n) \\geq cg(n)\\geq 0\\cr \\text{otherwise choose the last } n = \\text{ that } f(n)\\geq cg(n) \\text{ as } n_0\\cr \\text{for all }n\\geq n_0,f(n)\\leq cg(n)\\implies f(n)=O(g(n))\\cr \\end{aligned} $$ $f(n)=n|\\sin n|,g(n)=1$ $$ \\begin{aligned} \\forall f(n) , g(n):\\cr f(n)=\\Theta(g(n)) \\implies f(n)=\\Omega(g(n)) \\And f(n)=O'(g(n))\\cr \\text{but the conversion is not true} \\end{aligned} $$ $$ \\begin{aligned} \\tilde\\Omega(g(n))=\\lbrace f(n): & \\text{ there exist positive constants }c,k,\\text{ and } n_0 \\text{ such that}\\cr & 0\\leq f(n) \\leq cg(n)\\lg^{k}(n) \\text{ for all }n \\geq n_0\\rbrace .\\cr \\tilde\\Theta(g(n))=\\lbrace f(n): & \\text{ there exist positive constants }c_1,c_2,k,\\text{ and } n_0 \\text{ such that}\\cr & 0\\leq c_1g(n)\\lg^{k}(n)\\leq f(n) \\leq c_2g(n)\\lg^{k}(n) \\text{ for all }n \\geq n_0\\rbrace .\\cr \\text{according to there definity: } & f(n)=\\tilde\\Theta(g(n))\\iff f(n)=\\tilde O(g(n)) \\And f(n)=\\tilde\\Omega (g(n))\\cr \\end{aligned} $$ We can apply the iteration operator * used in the $\\lg^{\\ast}$ function to any monotonically increasing function $f(n)$ over the real. For a given constant $c \\in \\R$, we define the iterated dunction $f_{c}^{\\ast}$ by $f_{c}^{\\ast}=\\min \\lbrace i\\geq0:f^{(i)}\\leq c\\rbrace$, which need not be well defined in all cases. In other words, the quantity $f_{c}^{\\ast}(n)$ is the minimum number of iterated applications of the function $f$ required to reduce its argument down to $c$ or less. For each of the functions $f(n)$ and constants $c$ in the table below, give as tight a bound as possible on $f_{c}^{\\ast}(n)$. If there is no $i$ such that $f^{(i)} \\leq c$, write \"undefined\" as your answer. Note: You may wish to read Section 4.5 before attempting some of these exercises. Generalize MATRIX-MULTIPLY-RECURSIVE to multiple $n\\times n$ matrices for which $n$ is not necessarily an exact power of 2. Give a recurrence describing its running time. Argue that it runs in $\\Theta(n^3)$ time in the worst case. running time = $\\Theta(n^3)+O(n^2)+O(1)+O(n)=\\Theta(n^3)$ How quickly can you multiply a $kn\\times n$ matrix (kn rows and n columns) by an $n\\times kn$ matrix, where $k\\geq 1$ , using MATRIX-MULTIPLY-RECURSIVE as a subtoutine? Answer the same question for multiplying an $n\\times kn$ matrix by a $kn\\times n$ matrix. Which is asymptotically faster, and by how much? Suppose that instead of partitioning matrices by index calculation in MATRIX-MULTIPLY-RECURSIVE, you copy the appropriate elements of A,B,and C into separate $n/2\\times n/2$ submarices $A_{11},A_{12},A_{21},A_{22};B_{11},B_{12},B_{21},B_{22}$; and $C_{11},C_{12},C_{21},C_{22}$, respectively. After the recursive calls, you copy the results from $C_{11},C_{12},C_{21}$, and $C_{22}$ back into the appropriate places in $C$. How does recurrence (4.9) change, and what is its solution? $T(n)=8T(n/2)+\\Theta(n^2)$ According to the master theorem, $T(n)=\\Theta(n^3)$ Write pseudocode for a divide-and-conquer algorithm MATRIX-ADD-RECURSIVE that sums two $n\\times n$ matrices $A$ and $B$ by partitioning each of them into four $n/2\\times n/2$ submatrices and then recursively summing corresponding pairs of submatrices. Assume that matrix partitioning uses $\\Theta(1)$-time index calculations. Write a recurrence for the worst-case running time of MATRIX-ADD-RECURSIVEE, and solve your recurrence. What happens if you use $\\Theta(n^2)$-time copying to implement the partitioning instead of index calculations? $T(n)=4T(n/2)+\\Theta(1)$ According to master theorem, $T(n)=\\Theta(n^2)$ if use copying,$T(n)=4T(n/2)+\\Theta(n^2)\\implies T(n)=\\Theta(n^2\\lg n)$ Note: You may wish to read Section 4.5 before attempting some of these exercises. Use Strassen\u2019s algorithm to compute the matrix product $$ \\begin{pmatrix} 1 & 3\\cr 7 & 5 \\end{pmatrix} \\begin{pmatrix} 6 & 8\\cr 4 & 2 \\end{pmatrix} $$ Show your work. $$ \\begin{aligned} S_1 & = B_{12} - B_{22} = 8 - 2 = 6\\cr S_2 & = A_{11} + A_{12} = 1 + 3 = 4\\cr S_3 & = A_{21} + A_{22} = 7 + 5 = 12\\cr S_4 & = B_{21} - B_{11} = 4 - 6 = -2\\cr S_5 & = A_{11} + A_{22} = 1 + 5 = 6\\cr S_6 & = B_{11} + B_{22} = 6 + 2 = 8\\cr S_7 & = A_{12} - A_{22} = 3 - 5 = -2\\cr S_8 & = B_{21} + B_{22} = 4 + 2 = 6\\cr S_9 & = A_{11} - A_{21} = 1 - 7 = -6\\cr S_{10} & = B_{11} + B_{12} = 6 + 8 = 14\\cr P_1 & = A_{11} \\cdot S_1 = 1 * 6 = 6\\cr P_2 & = S_2 \\cdot B_{22}=4 * 2 = 8\\cr P_3 & = S_3 \\cdot B_{11} = 12 * 6 = 72\\cr P_4 & = A_{22} \\cdot S_{4} = 5 * (-2) = -10\\cr P_5 & = S_5 \\cdot S_6 = 6 * 8 = 48\\cr P_6 & = S_7 \\cdot S_8 = -2 * 6 = -12\\cr P_7 & = S_9 \\cdot S_10 = -6 * 14 = -84\\cr C_{11} & = C_{11} + P_5 + P_4 - P_2 + P_6 = 48 + (-10) - 8 + (-12) = 18\\cr C_{12} & = C_{12} + P_1 + P_2 = 6 + 8 = 14\\cr C_{21} & = C_{21} + P_3 + P_4 = 72 + (-10) = 62\\cr C_{22} & =C_{22} + P_5 + P_1 - P_3 - P_7 = 48 + 6 - 72 - (-84) = 66\\cr C & = \\begin{pmatrix} 18 & 14\\cr 62 & 66\\cr \\end{pmatrix} \\end{aligned} $$ Write pseudocode for Strassen\u2019s algorithm. What is the largest $k$ such that if you can multiply $3\\times 3$ matrices using $k$ multiplications (not assuming commutativity of multiplication), then you can multiply $n\\times n$ matrices in $o(n^{\\lg 7})$ time? What is the running time of this algorithm? $$ \\begin{aligned} T(n)=kT(n/3)+O(1)=o(n^{\\lg 7})\\cr \\implies T(n)=\\Theta(n^{\\log_{3} k})=o(n^{\\lg 7})\\cr \\implies \\log_{3} k < \\lg 7\\cr \\implies k < 21.8\\cr \\implies \\max(k) = 21\\cr \\text{running time }= n^{\\log_{3} 21}\\cr \\end{aligned} $$ V. Pan discovered a way of multiplying $68\\times 68$ matrices using 132,464 multiplications, a way of multiplying $70\\times 70$ matrices using 143,640 multiplications, and a way of multiplying $72\\times 72$ matrices using 155,424 multiplications. Which method yields the best asymptotic running time when used in a divide-and-conquer matrix-multiplication algorithm? How does it compare with Strassen\u2019s algorithm? $n^{\\log_{68}132464}=n^{2.79513};n^{\\log_{70} 143640}=n^{2.79512};n^{\\log_{72} 155424}=n^{2.79515};n^{\\lg 7}=n^{2.80735}$ the second method is the best; better than Strassen's algorithm. Show how to multiply the complex numbers $a+bi$ and $c+di$ using only three multiplications of real numbers. The algorithm should take $a,b,c$, and $d$ as input and produce the real component $ac-bd$ and the imaginary component $ad+bc$ separately. $$ \\begin{aligned} S_1=a(c-d)=ac-ad\\cr S_2=b(c+d)=bc+bd\\cr S_3=d(a-b)=ad-bd\\cr S_1+S_3=ac-bd\\cr S_2+S_3=ad+bc\\cr \\end{aligned} $$ Suppose that you have a $\\Theta(n^{\\alpha})$-time algorithm for squaring $n \\times n$ matrices, where $\\alpha \\geq 2$. Show how to use that algorithm to multiply two different $n \\times n$ matrices in $\\Theta(n^{\\alpha})$ time. $$ \\begin{aligned} A\\cdot B=\\frac{(A+B)^{2}-(A-B)^2}{4} \\end{aligned} $$ Use the substitution method to show that each of the following recurrences defined on the reals has the asymptotic solution specified: a. $T(n)=T(n-1)+n$ has solution $T(n) = O(n^2)$ b. $T(n)=T(n/2)+\\Theta(1)$ has solution $T(n)=O(\\lg n)$ c. $T(n)=2T(n/2)+n$ has solution $T(n)=O(n\\lg n)$. d. $T(n)=2T(n/2+17)+n$ has solution $T(n) = O(n\\lg n)$. e. $T(n)=2T(n/3)+\\Theta(n)$ has solution $T(n)=\\Theta(n)$ f. $T(n) = 4T(n/2)+\\Theta(n)$ has solution $T(n)=\\Theta(n^2)$. a. $$ \\begin{aligned} \\text{ inductive case: } &\\cr & \\text{assume }T(n')\\leq cn'^2 \\text{ for all } n_0 \\leq n'\\leq n\\cr T(n)& =T(n-1)+n\\cr & \\leq c(n-1)^2+n\\cr & = cn^2+(1-2c)n+c\\cr & \\leq cn^2\\cr & \\text{while the last step holds for }c > \\frac{1}{2}\\cr \\text{base case: } &\\cr T(n) & \\leq c \\leq cn \\text{ for all }n \\leq n_0 \\text{ is true if and only if choose a large enough } n_0\\cr \\end{aligned} $$ b. assume $T(n) \\leq c\\lg n$ $$ \\begin{aligned} T(n)&=T(n/2)+c'\\cr &\\leq c\\lg n -c\\lg 2 +c'\\cr &\\leq c\\lg n\\cr \\end{aligned} $$ the last step holds for $c>c'/\\lg 2$. c. assume $T(n)\\leq cn\\lg n$ $$ \\begin{aligned} T(n) & = 2T(n/2)+n\\cr & \\leq 2c(n/2)\\lg (n/2) +n\\cr & = cn\\lg n + (1-c\\lg 2)n\\cr & \\leq cn\\lg n \\end{aligned} $$ the last step holds for $c>1/\\lg 2$. d. assume $T(n)\\leq c(n-a)\\lg (n-a)$ $$ \\begin{aligned} T(n)&=2T(n/2+17)+n\\cr & \\leq 2(c(n/2+17-a))\\lg(n/2-a+17)+n\\cr & = c(n-2a+34)\\lg(n -a +34)+(1-c\\lg 2)n +c(-2a+34)\\lg 2\\cr & \\leq c(n-a)\\lg (n-a)\\cr \\end{aligned} $$ the last step holds for $n\\geq 34$ e. assume $c_{1}n\\leq T(n)\\leq c_{2}n$ $$ \\begin{aligned} T(n) & =2T(n/3)+\\Theta(n)\\cr & \\geq 2(c_{1}n/3)+c_{3}n\\cr & =(2c_{1}/3+c_{3})n\\cr & \\geq c_{1}n\\cr \\end{aligned} $$ the last step holds for $c_{1}\\leq 3c_{3}$ $c_{3}$ is the lowwer bound constant in $\\Theta(n)$. $$ \\begin{aligned} T(n) & =2T(n/3)+c_{4}n\\cr & \\leq (2c_{2}/3+c_{4})n\\cr & \\leq c_{2}n\\cr \\end{aligned} $$ the last step holds for$c_{2}\\geq 3c_{4}$ $c_{4}$ is the upper bound constant in $\\Theta(n)$. f. assume $c_{1}n^2 \\leq T(n)\\leq c_{2}n^2-c_{0}n$ $$ \\begin{aligned} T(n) & =4T(n/2)+\\Theta(n)\\cr & \\geq c_{1}n^2+c_{3}n\\cr & \\geq c_{1}n^2\\cr T(n) & =4T(n/2)+\\Theta(n)\\cr & \\leq c_{2}n^2-2c_{0}n+c_{4}n\\cr & \\leq c_{2}n^2-c_{0}n\\cr \\end{aligned} $$ the last step holds for $c_{0}\\geq c_{4}$ $c_{3}$ is the lowwer bound constant in $\\Theta(n)$ and $c_{4}$ is the upper bound constant in $\\Theta(n)$. The solution to the recurrence $T(n) = 4T(n/2) + n$ turns out to be $T(n) = \\Theta(n^2)$.Show that a substitution proof with the assumption $T(n) \\leq cn^2$ fails. Then show how to subtract a lower-order term to make a substitution proof work. assume $T(n)\\leq cn^2$ $$ \\begin{aligned} T(n) & =4T(n/2)+n\\cr & \\leq cn^2+n\\cr \\end{aligned} $$ It can not further imply $T(n)\\leq cn^2+n$ assume $T(n)\\leq c_{1}n^2-c_{2}n$ $$ \\begin{aligned} T(n) & =4T(n/2)+n\\cr & \\leq c_{1}n^2-2c_{2}n+n\\cr & \\leq c_{1}n^2-c_{2}n\\cr \\end{aligned} $$ the last step holds for $c_{2}\\geq 1$ The recurrence $T(n)=2T(n-1)+1$ has the solution T(n) = O(2^n). Show that a subtitution proof fails with the assumption $T(n)\\leq c2^n$, where $c > 0$ is constant. Then show how to subtract a lower-order term to make a substitution proof work. assume $T(n)\\leq c2^{n}$ $$ \\begin{aligned} T(n) & =2T(n-1)+1\\cr & \\leq 2\\cdot c2^{n-1}+1\\cr & =c2^{n}+1\\cr \\end{aligned} $$ It can not further imply $T(n)\\leq c2^{n}$ assume $T(n)\\leq c2^{n}-d$ $$ \\begin{aligned} T(n) & =2T(n-1)+1\\cr & \\leq 2\\cdot c2^{n-1}-2d+1\\cr & =c2^{n}-d\\cr \\end{aligned} $$ the last step holds for $d\\geq 1$ For each of the following recurrences, sketch its recursion tree, and guess a good asymptotic upper bound on its solution. Then use the substitution method to verify your answer. a. $T(n)=T(n/2)+n^3$. b. $T(n) = 4T(n/3)+n$. c. $T(n) = 4T(n/2)+n$. d. $T(n) = 3T(n-1)+1$. a. guess $T(n)=O(n^3)$,assume $T(n)\\leq cn^3$ $$ \\begin{aligned} T(n) & = T(n/2)+n^3\\cr & \\leq \\frac{cn^3}{8}+n^3\\cr & \\leq (1+\\frac{c}{8})n\\cr & \\leq cn^3\\cr \\end{aligned} $$ the last step holds for $c\\geq 7/8$ b. guess $T(n)=O(n^{\\log_{3} 4})$, assume $T(n)\\leq cn^{\\log_3 4}-dn$ $$ \\begin{aligned} T(n) & =4T(n/3)+n\\cr & \\leq cn^{\\log_{3} 4} -\\frac{4dn}{3}+ n\\cr & \\leq cn^{\\log_{3} 4}\\cr \\end{aligned} $$ the last step holds for $d\\geq 4/3$ c. guess $T(n)=O(n^2)$, assume $T(n)\\leq cn^2-dn$ $$ \\begin{aligned} T(n) & =4T(n/2)+n\\cr & \\leq cn^2-2dn+n\\cr & \\leq cn^2\\cr \\end{aligned} $$ the last step holds for $d\\geq 1/2$ d. guess $T(n)=O(3^n)$, assume $T(n)\\leq c3^n-d$ $$ \\begin{aligned} T(n) & =3T(n-1)+1\\cr & \\leq 3\\cdot 3^{n-1}-3d +1\\cr & \\leq 3^n\\cr \\end{aligned} $$ the last step holds for $d\\geq 1/3$ Use the substitution method to prove that recurrence (4.15) has the asymptotic lower bound $L(n)=\\Omega(n)$. Conclude that $L(n)=\\Theta(n)$. assume$L(n)\\geq cn$ $$ \\begin{aligned} L(n) & =L(n/3)+L(2n/3)\\cr & \\geq c(n/3)+c(n2/3)\\cr & = cn\\cr \\end{aligned} $$ since the textbook has proved that $L(n)=O(n)$, and now $L(n)=\\Omega(n)$ is proved, we can conclude that $L(n)=\\Theta(n)$. Use the substitution method to prove that recurrence (4.14) has the solution $T(n) = \\Omega(n\\lg n)$. Conclude that $T(n)=\\Theta(n\\lg n)$. guess $T(n)\\leq cn\\lg n$ $$ \\begin{aligned} T(n) & =T(n/3)+T(2n/3)+\\Theta(n)\\cr & \\leq c(n/3)\\lg(n/3)+c(2n/3)\\lg(2n/3)+dn\\cr & = cn\\lg{n}-\\frac{cn}{3}(3\\lg{3}-2-3d/c)\\cr & \\leq cn\\lg{n}\\cr \\end{aligned} $$ the last step holds for $c\\geq 3d/2$. Use a recursion tree to justify a good guess for the solution to the recurrence $T(n)=T(\\alpha n)+T((1-\\alpha)n)+\\Theta(n)$, where $\\alpha$ is a constant in the range $0 < \\alpha 1$. assume $\\alpha<1/2$, since otherwise let $\\beta=1-\\alpha =$ and solve it for $\\beta$. let $L(n)$ donates leaves number, then $L(n)=L(\\alpha n)+L((1-\\alpha)n)\\implies L(n)=\\Theta(n)$ the smallest depth of leaves is $\\log_{1/\\alpha}n$, while the largest depth is $\\log_{\\alpha}n$, total cost over all nodes at depth $i$, for $i=0,1,\\dots,\\log_{1/\\alpha}n-1$ is $\\Theta(n)$ $$ \\begin{aligned} & \\sum_{i=0}^{\\log_{1/\\alpha}n-1}c_{1}n+d_{1}n\\leq T(n)\\leq \\sum_{i=0}^{\\log_{\\alpha}n-1}c_{2}n+d_{2}n\\cr \\implies & c_{1}n\\log_{1/\\alpha}n+d_{1}n \\leq T(n) \\leq c_{2}n\\log_{\\alpha}n+d_{2}n\\cr \\implies & T(n)=\\Theta(n\\lg n)\\cr \\end{aligned} $$ Use the master method to give tight asymptotic bounds for the following recurrences. a. $T(n)=2T(n/4)+1$. b. $T(n)=2T(n/4)+\\sqrt{n}$. c. $T(n)=2T(n/4)+\\sqrt{n}\\lg^2 n$. d. $T(n)=2T(n/4)+n$. e. $T(n)=2T(n/4)+n^2$. a. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=1=O(n^{\\log_{b}a-1/2})\\cr T(n)=\\Theta(n^{1/2})\\cr \\end{aligned} $$ b. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n^{1/2}=\\Theta(n^{\\log_{b}a})\\cr T(n)=\\Theta(n^{1/2}\\lg n)\\cr \\end{aligned} $$ c. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n^{1/2}\\lg^{2}n=\\Theta(n^{\\log_{b}a}\\lg^{2}n)\\cr T(n)=\\Theta(n^{1/2}\\lg^{3} n)\\cr \\end{aligned} $$ d. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n=\\Omega(n^{\\log_{b}a+1/2})\\cr \\text{regularity condition: }af(n/b)=n/2\\leq (1/2)n\\cr T(n)=\\Theta(n)\\cr \\end{aligned} $$ e. $$ \\begin{aligned} a=2,b=4,n^{\\log_{b}a}=n^{1/2},f(n)=n^2=\\Omega(n^{\\log_{b}a+3/2})\\cr \\text{regularity condition: }af(n/b)=n/8\\leq (1/8)n\\cr T(n)=\\Theta(n^2)\\cr \\end{aligned} $$ Professor Caesar wants to develop a matrix-multiplication algorithm that is asymptotically faster than Strassen\u2019s algorithm. His algorithm will use the divide-and-conquer method, dividing each matrix into $n/4\\times n/4$ submatrices, and the divide and combine steps together will take $\\Theta(n^2)$ time. Suppose that the professor\u2019s algorithm creates a recursive subproblems of size $n/4$. What is the largest integer value of $a$ for which his algorithm could possibly run asymptotically faster than Strassen\u2019s? $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{4}a} < n^{\\lg 7}\\cr \\implies a < 49\\cr \\end{aligned} $$ Use the master method to show that the solution to the binary-search recurrence $T(n) = T(n/2) + \\Theta(1)$ is $T(n) = \\Theta(\\lg n)$. (See Exercise 2.3-6 for a description of binary search.) $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr f(n)=1=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T(n)=\\Theta(\\lg n)\\cr \\end{aligned} $$ Consider the function $f(n) = \\lg n$. Argue that although $f(n/2) < f(n)$, the regularity condition $af(n/b)\\leq cf(n)$ with $a=1$ and $b=2$ does not hold for any constant $c<1$. Argue further that for any $\\epsilon > 0$, the condition in case 3 that $f(n) = \\Omega(n^{\\log_{b}a+\\epsilon})$ does not hold. $$ \\begin{aligned} &\\forall c<1\\cr &f(n/2)\\leq cf(n)\\cr \\iff & \\lg n-1\\leq c\\lg n\\cr \\iff & \\lg n\\leq 1/(1-c)\\cr \\iff & n\\leq 2^{1/(1-c)}\\cr \\end{aligned} $$ so the regularity condition does not hold for $n\\geq 2^{1/(1-c)}$ $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr \\forall \\epsilon>0\\cr cn^{\\epsilon}\\leq \\lg n \\text{ does not hold for }n\\to \\infty\\cr \\end{aligned} $$ so $f(n) = \\Omega(n^{\\log_{b}a+\\epsilon})$ does not hold. Show that for suitable constants $a,b$, and $\\epsilon$, the function $f(n)=2^{\\lceil\\lg n\\rceil}$ satisfies all the conditions in case 3 of the master theorem except the regularity condition. $$ \\begin{aligned} a=1.1,b=1.2,n^{\\log_{b}a}=n^{\\log_{1.2}1.1}\\cr f(n)\\geq 2^{\\lg n}=n\\geq n^{\\log_{1.2}1.1+\\log_{1.2}1.001}\\cr \\implies \\exist \\epsilon>0 \\text{ such that }f(n)=\\Omega(n^{\\log_{b}a+\\epsilon})\\cr \\end{aligned} $$ $f(n)$ satisfies all the conditions in case 3 of the master theorem except the regularity condition. $1.1f(\\frac{1.3\\cdot 2^k}{1.2})=1.1\\cdot 2^{k+1}\\leq cf(1.3\\cdot 2^k)=c\\cdot 2^{k+1}$ dose not hold for any c<1. Show that $\\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(\\log_{b} n-j)^k=\\Omega(\\log_b^{k+1}n)$. $$ \\begin{aligned} \\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(\\log_{b} n-j)^k & \\geq \\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(\\lfloor \\log_{b}n\\rfloor n-j)^k\\cr & = \\sum_{j=0}^{\\lfloor \\log_{b}n\\rfloor}(j)^k\\cr & \\geq \\sum_{j=0}^{\\log_{b}n-1}(j)^k\\cr & = \\Theta((\\log_{b}n-1)^{k+1})\\cr & = \\Omega(\\log_b^{k+1}n)\\cr \\end{aligned} $$ Show that case 3 of the master theorem is overstated (which is also why case 3 of Lemma 4.3 does not require that $f(n)=\\Omega(n^{\\log_b a+\\epsilon})$) in the sense that the regularity condition $af(n/b)\\leq cf(n)$ for some constant $c<1$ implies that there exists a constant $\\epsilon > 0$ sunch that $f(n)=\\Omega(n^{\\log_b a+\\epsilon})$. $$ \\begin{aligned} & af(n/b) \\leq cf(n)\\cr \\implies & f(n)\\geq \\frac{a}{c}f(n/b)\\cr \\implies & f(n) \\geq (\\frac{a}{c})^i f(n/{b^i})\\cr & \\text{let } n=b^i\\cr \\implies & f(n)\\geq (\\frac{a}{c})^{\\log_{b}n}f(1)\\cr \\implies & f(n) \\geq n^{\\log_{b}\\frac{a}{c}}\\cr & \\because c<0,\\therefore \\frac {a}{c}=a+\\epsilon \\text{ for some constant }\\epsilon >0\\cr \\implies & f(n)=\\Omega(n^{\\log_{b}a+\\epsilon})\\cr \\end{aligned} $$ For $f(n)=\\Theta(n^{\\log_b a}/\\lg n)$, prove that the summation in equation (4.19) has solution $g(n)=\\Theta(n^{\\log_b{a}}\\lg\\lg n)$. Conclude that a master recurrence $T(n)$ using $f(n)$ as its driving function has solution $T(n) = \\Theta(n^{\\log_b a}\\lg\\lg n)$. $$ \\begin{aligned} & \\text{let }n=b^i\\cr g(n) = & \\sum_{j=0}^{i}a^jf(n/b^j)\\cr = & \\sum_{j=0}^{i}a^jf(b^{i-j})\\cr = & \\sum_{j=0}^{i}a^j\\Theta(a^{i-j}/(i-j))\\cr = & \\sum_{j=0}^{i}a^i\\Theta(1/j)\\cr = & a^i\\Theta(\\lg i)\\cr = & \\Theta(a^{\\log_{b}n} \\lg\\log_{b}n)\\cr = & \\Theta(n^{\\log_{b}a} \\lg\\lg n)\\cr g(bn) = & \\Theta((bn)^{\\log_{b}a} \\lg\\lg bn)=(n^{\\log_{b}a} \\lg\\lg n)\\cr \\implies g(n)=& (n^{\\log_{b}a} \\lg\\lg n)\\text{ even if }n \\neq b^i\\cr \\end{aligned} $$ We can concluded that $T(n)=g(n)+n^{\\log_{b}a}=n^{\\log_{b}a} \\lg\\lg n$. Consider an Akra-Bazzi recurrence $T(n)$ on the reals as given in recurrence (4.22), and define $T'(n)$ as $$ \\begin{aligned} T'(n)=cf(n)+\\sum_{i=1}^k a_iT'(n/b_i), \\end{aligned} $$ where $c > 0$ is constant. Prove that whatever the implicit initial conditions for T(n) might be, there exist initial conditions for $T'(n)$ such that $T'(n) = cT(n)$ for all $n>0$. Conclude that we can drop the asymptotics on a driving function in any Akra-Bazzi recurrence without affecting its asymptotic solution. assume $T'(n)=cT(n)$ $$ \\begin{aligned} T'(n) & =cf(n)+\\sum_{i=1}^k a_iT'(n/b_i)\\cr & =cf(n)+\\sum_{i=1}^k a_icT(n/b_i)\\cr & =c(f(n)+\\sum_{i=1}^k a_iT(n/b_i))\\cr & =cT(n)\\cr \\end{aligned} $$ $T'(n)=T(n)$ holds only if the base case(initial conditions) that $T'(n)=T(n)$ for all $n<n_{}0$ is true Show that $f(n)=n^2$ satisfies the polynomial-growth condition but that $f(n)=2^n$ does not. $$ \\begin{aligned} & \\forall \\phi \\forall \\psi:1 \\leq \\psi \\leq \\phi\\cr & n^2/d\\leq (\\psi n)^2\\leq dn^2\\cr \\iff & \\max(1/\\psi^{2})\\leq d \\wedge \\max(\\psi^{2}) \\leq d\\cr \\iff & \\phi^{2}\\leq d\\qquad(1)\\cr & 2^n/d\\leq 2^{\\psi n}\\leq d2^n\\cr \\iff & 2^{n-\\psi n}\\leq d\\wedge 2^{\\psi n-n}\\leq d\\qquad (2)\\cr \\end{aligned} $$ $d=\\phi^{2}$ satisfies $(1)$, since $2^{(\\psi-1)n}\\to\\infty$ therefore no constant $d$ satisfies $(2)$ Let f(n) be a function that satisfies the polynomial-growth condition. Prove that $f(n)$ is asymptotically positive, that is, there exists a constant $n_0\\geq 0$ such that $f(n)\\geq 0$ for all $n\\geq n_0$. $$ \\begin{aligned} \\exist \\hat{n} \\text{ that }\\forall n > \\hat{n}:\\cr f(n)/d \\leq f(\\psi n) \\leq df(n)\\cr \\implies 0\\leq (d^2-1) f(n)\\cr \\because d>1\\cr \\therefore f(n)>0\\cr \\end{aligned} $$ Give an example of a function $f(n)$ that does not satisfy the polynomial-growth condition but for which $f(\\Theta(n)) = \\Theta(f(n))$. Use the Akra-Bazzi method to solve the following recurrences. a. $T(n) = T(n/2)+T(n/3)+T(n/6)+n\\lg n$. b. $T(n) = 3T(n/3)+8T(n/4)+n^2/\\lg n$. c. $T(n) = (2/3)T(n/3)+(1/3)T(2n/3)+\\lg n$. d. $T(n) = (1/3)T(n/3) + 1/n$. e. $T(n) = 3T(n/3) + 3T(2n/3)+n^2$. a. $$ \\begin{aligned} & (\\frac{1}{2})^p+(\\frac{1}{3})^p+(\\frac{1}{6})^p=1\\cr \\implies & p=1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{x\\lg x}{x^{2}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{\\ln x}{x\\ln 2}dx))\\cr & = \\Theta(n^1(1+\\left[\\frac{\\ln^{2} x}{2\\ln 2}\\right]_{1}^{n}))\\cr & = \\Theta(n(1+\\frac{\\ln^{2} n}{2\\ln 2}))\\cr & =\\Theta(n\\lg^{2} n) \\end{aligned} $$ b. $$ \\begin{aligned} & \\frac{3}{3^p}+\\frac{8}{4^p}=1\\cr & \\frac{3}{3^{1.5}}+\\frac{8}{4^{1.5}}=3^{-(1/2)}+1\\cr & \\frac{3}{3^2}+\\frac{8}{4^2}=\\frac{5}{6}\\cr \\implies & 1.5<p<2\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{x^2/\\lg x}{x^{p+1}}dx))\\cr & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{1}{x^{p-1}\\lg x}dx))\\cr & =\\Omega(n^p(1+\\lg n\\int_{1}^{n}\\frac{1}{x^{p-1}}dx))\\cr & =\\Omega(n^p(1+n^{2-p}\\lg n ))\\cr & =\\Omega(n^2/\\lg n)\\cr T(n) & =O(n^p(1+\\lg 1(\\text{regarded as constant})\\int_{1}^{\\sqrt{n}}\\frac{1}{x^{p-1}}dx+\\lg \\sqrt{n}\\int_{\\sqrt{n}}^{n}\\frac{1}{x^{p-1}}dx))\\cr & = O(n^p(1+\\Theta(n^{(p-1)/2})+\\Theta(n^{p-1}/\\lg n)))\\cr & =O(n^2/\\lg n)\\cr \\end{aligned} $$ c. $$ \\begin{aligned} & \\frac{2}{3}(\\frac{1}{3})^p+\\frac{1}{3}(\\frac{2}{3})^p=1\\cr \\implies & \\frac{2+2^p}{3^{p+1}}=1\\cr \\implies & p=0\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(1+\\int_{1}^{n}\\frac{\\lg x}{x}dx)\\cr & =\\Theta(\\lg^{2}n) \\end{aligned} $$ d. $$ \\begin{aligned} & \\frac{1}{3}(\\frac{1}{3})^p=1\\cr \\implies & p=-1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^{-1}(1+\\int_{1}^{n}\\frac{1}{x}dx))\\cr & =\\Theta(n^{-1}(1+\\left[lg n\\right]_{1}^{n}))\\cr & =\\Theta(n^{-1}lg n)\\cr \\end{aligned} $$ e. $T(n) = 3T(n/3) + 3T(2n/3)+n^2$. $$ \\begin{aligned} & 3(\\frac{1}{3})^p + 3(\\frac{2}{3})^p=1\\cr \\implies & p=2\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^{2}(1+\\int_{1}^{n}\\frac{x^2}{x^3}dx))\\cr & =\\Theta(n^2\\lg n) \\end{aligned} $$ Use the Akra-Bazzi method to prove the continuous master theorem. $$ \\begin{aligned} & T(n)=aT(n/b)+f(n)\\cr & \\frac{a}{b^p}=1\\cr \\implies & p=\\log_{b}a\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & =\\Theta(n^{\\log_{b}a}(1+\\int_{1}^{n}\\frac{f(x)}{x^{\\log_{b}a+1}}dx))\\cr & G(n)=\\int_{1}^{n}\\frac{f(x)}{x^{\\log_{b}a+1}}dx\\cr \\text{case 1:} & \\cr & f(n)=O(n^{\\log_{b}a-\\epsilon})(\\epsilon>0)\\cr G(n) & = O(\\int_{1}^{n}\\frac{1}{x^{1+\\epsilon}}dx)\\cr & = O(x^{-\\epsilon})\\cr T(n) & =\\Theta(n^{\\log_{b}a}(1+G(n)))\\cr & = \\Theta(n^{\\log_{b}a})\\cr \\text{case 2:} & \\cr & f(n)=\\Theta(n^{\\log_{b}a}\\lg^{k}n)(k\\geq 0)\\cr G(n) & = \\Theta(\\int_{1}^{n}\\frac{lg^{k}n}{x^{1}}dx)\\cr & = \\Theta(\\lg^{k+1}n)\\cr T(n) & =\\Theta(n^{\\log_{b}a}(1+G(n)))\\cr & = \\Theta(n^{\\log_{b}a}\\lg^{k+1}n)\\cr \\text{case 3:} & \\cr T(n) & =\\Omega(f(n))\\cr & f(n)=\\Omega(n^{\\log_{b}a+\\epsilon})(\\epsilon>0)\\cr &\\text{if } af(n/b)<cf(n)\\text{ for some }c<1\\cr G(n) & = O(f(n)\\int_{1}^{n}\\frac{1}{x^{\\log_{b}a+1}}dx)\\cr & = O(\\frac{f(n)}{x^{\\log_{b}a}})\\cr T(n) & =\\Theta(n^{\\log_{b}a}(1+G(n)))\\cr & =O(f(n))\\cr T(n)& =\\Theta(f(n)) \\end{aligned} $$ Give asymptotically tight upper and lower bounds for $T(n)$ in each of the following algorithmic recurrences. Justify your answers. a. $T(n) = 2T(n/2)+n^3$. b. $T(n) = T(8n/11)+n$. c. $T(n) = 16T(n/4)+n^2$. d. $T(n)=4T(n/2)+n^2\\lg n$. e. $T(n) = 8T(n/3)+n^2$. f. $T(n)=7T(n/2)+n^2\\lg n$. g. $T(n)=2T(n/4)+\\sqrt{n}$. h. $T(n)=T(n-2)+n^2$. $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{2}2}=n\\cr f(n)=n^3=\\Omega(n^{2+\\log_{b}a})\\cr 2f(n/2)=n^3/4\\leq \\frac{1}{4}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n^3)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}a}=n^{\\log_{11/8}1}=1\\cr f(n)=n=\\Omega(n^{1+\\log_{b}a})\\cr f(8n/11)=8n/11\\leq \\frac{8}{11}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{4}{16}}=n^{2}\\cr f(n)=n^2=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T(n)=\\Theta(n^2\\lg^{} n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{2}{4}}=n^{2}\\cr f(n)=n^2\\lg^{} n=\\Theta(n^{\\log_{b}a}\\lg^{}n)\\cr T(n)=\\Theta(n^2\\lg^{2} n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{3}{8}}\\cr f(n)=n^2=\\Omega(n^{\\log_{b}a})\\cr f8(n/3)=\\frac{8n^2}{9}\\leq \\frac{8}{9}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n^2)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{2}{7}}\\cr f(n)=n^2\\lg^{} n=O(n^{\\log_{b}a-\\log_2{(7/4)}})\\cr T(n)=\\Theta(n^{\\log_{2}{7}})\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}}=n^{\\log_{4}{2}}=n^{1/2}\\cr f(n)=n^{1/2}=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T(n)=\\Theta(n^{1/2}\\lg^{} n)\\cr \\end{aligned} $$ $$ \\begin{aligned} T(n)=\\frac{1}{2}\\sum_{i=1}^{n}i^2=\\Theta(n^3)\\cr \\end{aligned} $$ Throughout this book, we assume that parameter passing during procedure calls takes constant time, even if an N -element array is being passed. This assumption is valid in most systems because a pointer to the array is passed, not the array itself. This problem examines the implications of three parameter-passing strategies: Arrays are passed by pointer. Time $= \\Theta(1)$. Arrays are passed by copying. Time $=\\Theta(N)$, where $N$ is the size of the array. Arrays are passed by copying only the subrange that might be accessed by the called procedure. Time $=\\Theta(n)$ if the subarray contains $n$ elements. Consider the following three algorithms: a. The recursive binary-search algorithm for finding a number in a sorted array(see Exercise 2.3-6). b. The MERGE-SORT procedure from Section 2.3.1. c. The MATRIX-MULTIPLY-RECURSIVE procedure from Section 4.1. Give nine recurrences $T_{a1}(N,n),T_{a2}(N,n),\\dots ,T_{c3}(T,n)$ for the worst-case running times of each of the three algorithms above when arrays and matrices are passed using each of the three parameter-passing strategies above. Solve your recurrences, giving tight asymptotic bounds. a1 $$ \\begin{aligned} T_{a1}(N,n)=T(N,\\frac{n}{2})+\\Theta(1)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr f(n)=\\Theta(1)=\\Theta(n^{\\log_{b}a})\\cr T_{a1}(N,n)=\\Theta(\\lg n)\\cr \\end{aligned} $$ a2 $$ \\begin{aligned} T_{a2}(N,n)=T(N,\\frac{n}{2})+\\Theta(N)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr T_{a2}(N,n)=\\Theta(N\\lg n)\\cr \\end{aligned} $$ a3 $$ \\begin{aligned} T_{a3}(N,n)=T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=1\\cr f(n)=\\Theta(n)=\\Omega(n^{1+\\log_{b}a})\\cr af(\\frac{n}{b})=f(\\frac{n}{2})\\leq \\frac{1}{2}f(n)\\cr T_{a3}(N,n)=\\Theta(f(n))=\\Theta(n)\\cr \\end{aligned} $$ b1 $$ \\begin{aligned} T_{b1}(N,n)=2T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}2}=n\\cr f(n)=\\Theta(n)=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T_{b1}(N,n)=\\Theta(n \\lg n)\\cr \\end{aligned} $$ b2 $$ \\begin{aligned} T_{b2}(N,n)=2T(N,\\frac{n}{2})+\\Theta(N+n)=2T(N,\\frac{n}{2})+\\Theta(N)\\cr n^{\\log_{b}a}=n^{\\log_{2}1}=n\\cr T_{b2}(N,n)=\\Theta(n+N\\lg n)=\\Theta(N\\lg n)\\cr \\end{aligned} $$ b3 $$ \\begin{aligned} T_{b3}(N,n)=2T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}2}=n\\cr f(n)=\\Theta(n)=\\Theta(n^{\\log_{b}a}\\lg^{0}n)\\cr T_{b3}(N,n)=\\Theta(n \\lg n)\\cr \\end{aligned} $$ c1 $$ \\begin{aligned} T_{c1}(N,n)=8T(N,\\frac{n}{2})+\\Theta(1)\\cr n^{\\log_{b}a}=n^{\\log_{2}8}=n^3\\cr f(n)=\\Theta(1)=O(n^{\\log_{b}a-3})\\cr T_{c1}(N,n)=\\Theta(n^3)\\cr \\end{aligned} $$ c2 $$ \\begin{aligned} T_{c2}(N,n)=8T(N,\\frac{n}{2})+\\Theta(N)\\cr n^{\\log_{b}a}=n^{\\log_{2}8}=n^3\\cr T_{c2}(N,n)=\\Theta(n^3+N\\lg N)\\cr \\end{aligned} $$ c3 $$ \\begin{aligned} T_{c3}(N,n)=8T(N,\\frac{n}{2})+\\Theta(n)\\cr n^{\\log_{b}a}=n^{\\log_{2}8}=n^3\\cr f(n)=\\Theta(n)=O(n^{\\log_{b}a-2})\\cr T_{c3}(N,n)=\\Theta(n^3)\\cr \\end{aligned} $$ Sometimes, a little algebraic manipulation can make an unknown recurrence similar to one you have seen before. Let\u2019s solve the recurrence $T(n)=2T(\\sqrt{n})+\\Theta(\\lg n)$ (4.25) by using the change-of-variables method. a. Define $m=\\lg n$ and $S(m)=T(2^m)$. Rewrite recurrence (4.25) in terms of m and $S(m)$. b. Solve your recurrence for $S(m)$. c. Use your solution for $S(m)$ to conclude that $T(n)=\\Theta(\\lg n\\lg\\lg n)$. d. Sketch the recursion tree for recurrence (4.25), and use it to explain intuitively why the solution is $T(n)=\\Theta(\\lg n \\lg\\lg n)$. Solve the following recurrences by changing variables: e. $T(n)=2T(\\sqrt{n})+\\Theta(1)$. f. $T(n)=3T(\\sqrt[3]{n})+\\Theta(n)$. $S(m)=T(2^m)=2T(\\sqrt{2^m})+\\Theta(m)=2T(2^{m/2})+\\Theta(m)=2S(m/2)+\\Theta(m)$ $$ \\begin{aligned} m^{\\log_{b}a}=m\\cr f(m)=m=\\Theta(m^{\\log_{b}a})\\cr S(m)=\\Theta(m\\lg m) \\end{aligned} $$ $T(n)=S(\\lg n)=\\Theta(\\lg n\\lg\\lg n)$ depth:$\\lg\\lg n +1$, leaves: $2^{\\lg\\lg n}$ The total cost over all nodes at depth $i$,for $i=0,1,\\dots,\\lg\\lg n-1$ is $2^i\\cdot \\frac{1}{2^i}\\Theta(\\lg n)=\\Theta(\\lg n)$ $$ \\begin{aligned} T(n) & =2^{\\lg\\lg n}\\Theta(1)+\\sum_{i=0}^{\\lg\\lg n-1}\\Theta(\\lg n)\\cr & =\\Theta(\\lg n)+\\lg\\lg n\\Theta(\\lg n)\\cr & = \\Theta(\\lg n\\lg\\lg n)\\cr \\end{aligned} $$ depth:$\\lg\\lg n +1$, leaves: $2^{\\lg\\lg n}$ The total cost over all nodes at depth $i$,for $i=0,1,\\dots,\\lg\\lg n-1$ is $2^i\\cdot \\Theta(1)=\\Theta(2^i)$ $$ \\begin{aligned} T(n) & =2^{\\lg\\lg n}\\Theta(1)+\\sum_{i=0}^{\\lg\\lg n-1}2^i\\Theta(1)\\cr & =\\Theta(\\lg n)+(2^{\\lg\\lg n}-1)\\Theta(1)\\cr & = \\Theta(\\lg n)\\cr \\end{aligned} $$ $S(m)=T(3^m)=3T(3^{m/3})+\\Theta(3^m)=3S(m/3)+\\Theta(3^m)$. $$ \\begin{aligned} m^{\\log_{b}a}=m^{\\log_{3}3}=m\\cr f(m)=3^m=\\Omega(m^{2+\\log_{b}a})\\cr 3f(m/3)=3\\cdot 3^{m/3}\\leq (1/3)f(m)\\cr S(m)=\\Theta(f(m))=\\Theta(3^m)\\cr T(n)=S(\\log_{3}n)=\\Theta(n)\\cr \\end{aligned} $$ Give asymptotically tight upper and lower bounds for T(n) in each of the following recurrences. Justify your answers. a. $T(n)=5T(n/3)+n\\lg n$. b. $T(n)=3T(n/3)+n/\\lg n$. c. $T(n)= 8T(n/2)+n^3\\sqrt{n}$. d. $T(n)=2T(n/2-2)+n/2$. e. $T(n)=2T(n/2)+n/\\lg n$. f. $T(n)=T(n/2)+T(n/4)+T(n/8)+n$. g. $T(n)=T(n-2)+1/n$. h. $T(n)=T(n-1)+\\lg n$. i. $T(n)=T(n-2)+1/\\lg n$. j. $T(n)=\\sqrt{n}T(\\sqrt{n})+n$. $$ \\begin{aligned} n^{\\log_{b}{a}} = n^{\\log_{3}{5}}\\cr f(n) = n\\lg{n} = O(n^{\\log_{b}{a}-\\log_{3}{(5/4)}})\\cr T(n)=\\Theta({n^{\\log_{b}{a}}})=\\Theta(n^{\\log_{3}{5}})\\cr \\end{aligned} $$ $$ \\begin{aligned} & 3(\\frac{1}{3})^p=1\\cr \\implies & p = 1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{x/\\lg x}{x^{2}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{1}{x^{}\\lg x}dx))\\cr & = \\Theta(n^1(1+\\left[\\lg\\lg x\\right]_{1}^{n}))\\cr & = \\Theta (n\\lg\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} n^{\\log_{b}{a}} = n^{\\log_{2}{8}}=n^{3}\\cr f(n) = n^{7/2} = \\Omega(n^{\\log_{b}{a}+1/2})\\cr af(n/b)=2^{-1/2}(n^{7/2})\\leq 2^{-1/2}f(n)\\cr T(n)=\\Theta(f(n))=\\Theta(n^{7/2})\\cr \\end{aligned} $$ $$ \\begin{aligned} f(n)=n/2 \\text{ satisfies polynomial-growth condition}\\cr T(n)=2T(n/2)+n/2\\cr n^{\\log_{b}{a}} = n^{\\log_{2}{2}}=n^{}\\cr f(n) = n^{}/2 = \\Theta(n^{\\log_{b}{a}}\\lg^{0}n)\\cr T(n)=\\Theta(n^{\\log_{b}{a}}\\lg n)=\\Theta(n\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} & 2(\\frac{1}{2})^p=1\\cr \\implies & p = 1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{x/\\lg x}{x^{2}}dx))\\cr & = \\Theta(n^1(1+\\int_{1}^{n}\\frac{1}{x^{}\\lg x}dx))\\cr & = \\Theta(n^1(1+\\left[\\lg\\lg x\\right]_{1}^{n}))\\cr & = \\Theta(n\\lg\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} & (\\frac{1}{2})^{p}+(\\frac{1}{4})^{p}+(\\frac{1}{8})^{p}=1\\cr & 0.5<p<1\\cr T(n) & =\\Theta(n^p(1+\\int_{1}^{n}\\frac{f(x)}{x^{p+1}}dx))\\cr & = \\Theta(n^p(1+\\int_{1}^{n}\\frac{x}{x^{p+1}}dx))\\cr & = \\Theta(n^p(1+\\int_{1}^{n}\\frac{1}{x^{p}}dx))\\cr & = \\Theta(n^p(1+\\left[x^{1-p}\\right]_{1}^{n}))\\cr & = \\Theta(n)\\cr \\end{aligned} $$ $$ \\begin{aligned} T(n) & = \\frac{1}{2}\\sum_{i=1}^{n}(\\frac{1}{i})\\cr & \\leq \\frac{1}{2}+1/2\\int_{1}^{n-1}\\frac{1}{x}\\cr & = \\Theta(\\frac{1}{2}+\\frac{1}{2}\\lg (n-1))\\cr & = \\Theta(\\lg n)\\cr \\end{aligned} $$ $T(n)=T(n-1)+\\lg n$ $$ \\begin{aligned} T(n) & = \\sum_{i=1}^{n}\\lg i\\cr & = \\lg(n!)\\cr & = \\Theta(n\\lg n)\\cr \\end{aligned} $$ $$ \\begin{aligned} T(n) & =1+\\sum_{i=2}^{n}\\frac{1}{\\lg i}\\cr & = \\Omega(\\frac{n^2}{\\sum_{i=2}^{n}\\lg n})\\cr & = \\Omega(\\frac{n}{\\lg n})\\cr T(n) & =O(\\sum_{i=2}^{\\sqrt{n}-1}\\frac{1}{\\lg 2}+\\sum_{i=\\sqrt{n}}^{n}\\frac{1}{\\lg \\sqrt{n}})\\cr & = O(\\sqrt{n}+\\frac{n}{\\lg n})\\cr T(n) & = \\Theta(n)\\cr \\end{aligned} $$ depth: $\\lg\\lg n+1$, leaves: $n$ total cost of over all nodes at depth $i$, for $i=0,1,...,\\lg \\lg n-1$, is n $$ \\begin{aligned} T(n)=\\Theta(n\\lg\\lg n)\\cr \\end{aligned} $$ This problem develops properties of the Fibonacci numbers, which are defined by recurrence (3.31) on page 69. We\u2019ll explore the technique of generating functions to solve the Fibonacci recurrence. Define the generating function (or formal power series) $\\mathcal{F}$ as $$ \\begin{aligned} \\mathcal{F}(z) & = \\sum_{i=0}^{\\infty}F_iz^i\\cr & = 0+z+z^2+2z^3+3z^4+5z^5+8z^6+13z^7+21z^8+\\cdots,\\cr \\end{aligned} $$ where $F_i$ is the $i$th Fibonaccci nunber. a. Show that $\\mathcal{F}(z)=z+z\\mathcal{F}(z)+z^2\\mathcal{F}(z)$. b. Show that $$ \\begin{aligned} \\mathcal{F}(z) & = \\frac{z}{1-z-z^2}\\cr & = \\frac{z}{(1-\\phi z)(1-\\hat\\phi z)}\\cr & = \\frac{1}{\\sqrt{5}}(\\frac{1}{1-\\phi z}-\\frac{1}{1-\\hat\\phi z})\\cr \\end{aligned} $$ where $\\phi$ is the golden ratio, and $\\hat{\\phi}$ is its conjugatr(see page 69). c. Show that $$ \\mathcal{F}(z)=\\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\phi^{i}-\\hat{\\phi}^i)z^i. $$ You may use without proof the generating-function version of equation (A.7) on page 1142, $\\sum_{k=0}^{\\infty} x^k=1/(1-x)$. Because this equation involves a generating function, $x$ is formal variable, not a real-valued variable, so that you don\u2019t have to worry about convergence of the summation or about the requirement in equation (A.7) that $|x|<1$, which doesn\u2019t make sense here. d. Use part (c) to prove that $F_i=\\phi^i/\\sqrt{5}$ for $i>0$, rounded to the nearest integer. (Hint: Observe that $|\\hat\\phi<1|$.) e Prove that $F_{i+2}\\geq \\phi^{i}$ for $i\\geq 0$. $$ \\begin{aligned} z+z\\mathcal{F}(z)+z^2\\mathcal{F}(z) & =z+z\\sum_{i=0}^{\\infty}F_iz^i + z^{2}\\sum_{i=0}^{\\infty}F_iz^i\\cr & = z+\\sum_{i=1}^{\\infty}F_{i-1}z^i + \\sum_{i=2}^{\\infty}F_{i-2}z^i\\cr & = z+F_{0}z^1 + \\sum_{i=2}^{\\infty}(F_{i-2}+F_{i-1})z^i\\cr & = z+0 + \\sum_{i=2}^{\\infty}F_{i}z^i\\cr & =\\sum_{i=0}^{\\infty}F_{i}z^i\\cr & =\\mathcal{F}(z)\\cr \\end{aligned} $$ $$ \\begin{aligned} &\\mathcal{F}(z)=z+z\\mathcal{F}(z)+z^2\\mathcal{F}(z)\\cr \\implies & \\mathcal{F}(z) = \\frac{z}{1-z-z^2}\\cr & \\phi\\hat{\\phi}=-1;\\phi + \\hat{\\phi}=1;\\phi - \\hat{\\phi}=\\sqrt{5}\\cr \\implies & \\mathcal{F}(z) = \\frac{z}{1-(\\phi + \\hat{\\phi})z-(-\\phi\\hat{\\phi})z^2}\\cr \\implies & \\mathcal{F}(z) = \\frac{z}{(1-\\phi z)(1-\\hat\\phi z)}\\cr \\implies & \\mathcal{F}(z) = \\frac{1}{\\sqrt{5}}\\frac{(\\phi - \\hat{\\phi}) z}{(1-\\phi z)(1-\\hat\\phi z)}\\cr \\implies & \\mathcal{F}(z) = \\frac{1}{\\sqrt{5}}(\\frac{1}{1-\\phi z}-\\frac{1}{1-\\hat\\phi z})\\cr \\end{aligned} $$ $$ \\begin{aligned} \\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\phi^{i}-\\hat{\\phi}^i)z^i & = \\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\phi z)^i - \\sum_{i=0}^{\\infty}\\frac{1}{\\sqrt{5}}(\\hat{\\phi}z)^i\\cr & = \\frac{1}{\\sqrt{5}}(\\frac{1}{1-\\phi z}-\\frac{1}{1-\\hat{\\phi} z})\\cr & = \\mathcal{F}(z)\\cr \\end{aligned} $$ $$ \\begin{aligned} F_{i} & =\\frac{1}{\\sqrt{5}}(\\phi^{i}-\\hat{\\phi}^i)\\cr & = \\frac{\\phi^{i}}{\\sqrt{5}}-\\frac{\\hat{\\phi}^i}{\\sqrt{5}}\\cr & \\because |\\hat{\\phi}^i|<1 \\therefore |\\hat{\\phi}^i/\\sqrt{5}|<0.5\\cr \\implies & F_{i}=round(\\frac{\\phi^{i}}{\\sqrt{5}})\\cr \\end{aligned} $$ $$ \\begin{aligned} F_{2} & =1\\geq\\phi^{0}=1\\cr F_{3} & =2\\geq\\phi^{1}=\\frac{1+\\sqrt{5}}{2}\\cr F_{i+2} & = F_{i+1}+F_{i}\\cr & \\geq \\phi^{i-1}+\\phi^{i-2}\\cr & =\\phi^{i-2}(\\phi+1)\\cr & =\\phi^{i-2}(\\phi^2)\\cr & = \\phi^{i}\\cr \\end{aligned} $$ Professor Diogenes has $n$ supposedly identical integrated-circuit chips that in principle are capable of testing each other. The professor\u2019s test jig accommodates two chips at a time. When the jig is loaded, each chip tests the other and reports whether it is good or bad. A good chip always reports accurately whether the other chip is good or bad, but the professor cannot trust the answer of a bad chip. Thus, the four possible outcomes of a test are as follows: a. Show that if at least $n/2$ chips are bad, the professor cannot necessarily determine which chips are good using any strategy based on this kind of pairwise test. Assume that the bad chips can conspire to fool the professor. Now you will design an algorithm to identify which chips are good and which are bad, assuming that more than $n/2$ of the chips are good. First, you will determine how to identify one good chip. b. Show that $\\lfloor n/2 \\rfloor$ pairwise tests are sufficient to reduce the problem to one of nearly half the size. That is, show how to use $\\lfloor n/2 \\rfloor$ pairwise tests to obtain a set with at most $\\lceil n/2 \\rceil$ chips that still has the property that more than half of the chips are good. c. Show how to apply the solution to part (b) recursively to identify one good chip. Give and solve the recurrence that describes the number of tests needed to identify one good chip. You have now determined how to identify one good chip. d. Show how to identify all the good chips with an additional $\\Theta(n)$ pairwise tests. If the professor want to determine which chip is good, he must get case 1 that A and B both say the other is good, and ensure there is no bad chip left. So he need to use case 2,3,4 to drop bad chips, since bad chips conspire to fool the professor, each time will drop a bad chip together a good one. If bad chips number is more than good chips, the professor cannot necessarily determine which chips are good. already apply the solution recursively in part(b), $$ \\begin{aligned} T(n)=n/2+T(n/2)\\cr T(n)=\\Theta(n)\\cr \\end{aligned} $$ use the good chip to test other chips$\\Theta(n)$. An $m\\times n$ array $A$ of real numbers is a Monge array if for all $i,j,k$, and $l$ such that $1\\leq i <k \\leq m$ and $i \\leq j < l \\leq n$, we have $A[i,j]+A[k,l]\\leq A[i,l]+A[k,j]$. In other words, whenever we pick two rows and two columns of a Monge array and consider the four elements at the intersections of the rows and the columns, the sum of the upper-left and lower-right elements is less than or equal to the sum of the lower-left and upper-right elements. For example, the following array is Monge: $$ \\begin{array}{} 10 & 17 & 13 & 28 & 23\\cr 17 & 22 & 16 & 29 & 23\\cr 24 & 28 & 22 & 34 & 24\\cr 11 & 13 & 6 & 17 & 7\\cr 45 & 44 & 32 & 37 & 23\\cr 36 & 33 & 19 & 21 & 6\\cr 75 & 66 & 51 & 53 & 34\\cr \\end{array} $$ a. Prove that an array is Monge if and only if for all $i=1,2,\\dots,m-1$ and $j=1,2,\\dots,n-1$, we have $$ A[i,j]+A[i+1,j+1]\\leq A[i,j+1]+A[i+1,j]. $$ (Hint: For the \"if\" part, use induction separately on rows and columns.) b. The following array is not Monge. Change one element in order to make it Monge. (Hint: Use part (a).) $$ \\begin{array}{} 37 & 23 & 22 & 32\\cr 21 & 6 & 7 & 10\\cr 53 & 34 & 30 & 31\\cr 32 & 13 & 9 & 6\\cr 43 & 21 & 15 & 8\\cr \\end{array} $$ c. Let $f(i)$ be the index of the column containing the leftmost minimum elementof row $i$. Prove that $f(1)\\leq f(2)\\leq \\cdots \\leq f(m)$ for any $m\\times n$ Monge array. d. Here is a description of a divide-and-conquer algorithm that computes the left most minimum element in each row of an $m\\times n$ Monge array A: Construct a submatrix $A'$ of $A$ consisting of the even-numbered rows of $A$. Recursively determine the leftmost minimum for each row of $A'$. Then compute the leftmost minimum in the odd-numbered rows of $A'$. Explain how to compute the leftmost minimum in the odd-numbered rows of $A$ (given that the leftmost minimum of the even-numbered rows is known) in $O(m+n) time. e. Write the recurrence for the running time of the algorithm in part (d). Show that its solution is $O(m+n\\log m)$. $$ \\begin{aligned} \\text{assume }A[i,j]+A[i+k,j+l]\\leq A[i,j+l]+A[i+k,j]\\cr A[i+k,j]+A[(i+1)+k,j+l] \\leq A[i+k,j+l]+A[i+k+1,j]\\cr \\implies A[i,j]+A[i+k+1,j+l]\\leq A[i,j+l]+A[i+k+1,j]\\cr A[i,j+l]+A[i+k,j+l+1]\\leq A[i,j+l+1]+A[i+k,j+1]\\cr \\implies A[i,j]+A[i+k,(j+1)+l]\\leq A[i,j+l+1]+A[i+k,j]\\cr \\end{aligned} $$ $$ \\begin{array}{} 37 & 23 & 22 & 32\\cr 21 & 6 & (5) & 10\\cr 53 & 34 & 30 & 31\\cr 32 & 13 & 9 & 6\\cr 43 & 21 & 15 & 8\\cr \\end{array} $$ $$ \\begin{aligned} i \\lt k\\cr \\text{since } A[i,f(i)] = \\min(A[i,x]),A[k,f(k)] = \\min(A[k,x])\\cr A[i,f(i)] + A[k,f(k)] \\leq A[i,f(k)] + A[k,f(i)]\\cr \\text{assume }f(i) \\gt f(k)\\cr \\implies A[i,f(k)] + A[k,f(i)] \\leq A[i,f(i)] + A[k,f(k)]\\cr \\implies A[i,f(k)] + A[k,f(i)] = A[i,f(i)] + A[k,f(k)]\\cr A[i,f(i)] \\leq A[i,f(k)]\\cr A[k,f(k)] \\leq A[k,(f(i))]\\cr \\implies A[i,f(i)] = A[i,f(k)]\\cr \\implies A[k,f(k)]= A[k,(f(i))]\\cr \\implies f(i)\\leq f(k)\\cr \\text{Contradictory to assumption}\\cr \\implies f(i)\\leq f(k)\\cr \\end{aligned} $$ since $f(2k) \\leq f(2k+1)\\leq f(2k+2)$ in each odd row, we only need to compare $f(2k+2)-f(2k)+1$ elements. totally need to deal with $O(m)+O(n)=O(m+n)$ over all odd rows. $$ \\begin{aligned} T(m,n) & = T(m/2,n)+O(m+n)\\cr T(m,n) & = \\sum_{i=0}^{\\lg m}O(m/2^i+n)\\cr & = O(n\\lg m)+\\sum_{i=1}^{\\lg m}O(m/2^i)\\cr & = O(n\\lg m)+O(m)\\cr & = O(m+n\\lg m)\\cr \\end{aligned} $$
"},{"location":"#more-information","title":"More Information","text":"
"},{"location":"Chap01/1.1/#11-2","title":"1.1-2","text":"
"},{"location":"Chap01/1.1/#11-4","title":"1.1-4","text":"
"},{"location":"Chap01/1.1/#11-5","title":"1.1-5","text":"
"},{"location":"Chap01/1.1/#11-6","title":"1.1-6","text":"
"},{"location":"Chap01/1.2/","title":"1.2 Algorithms as a technology","text":""},{"location":"Chap01/1.2/#12-1","title":"1.2-1","text":"
"},{"location":"Chap01/1.2/#12-2","title":"1.2-2","text":"SUM-ARRAY(A,n)\nsum = 0\nfor i = 1 to n\n sum = sum + A[i]\nreturn sum\n
"},{"location":"Chap02/2.1/#21-3","title":"2.1-3","text":"sum = sum + A[i] results $sum = \\sum_{j=1}^{i-1} A[j] + A[i] = \\sum_{j=1}^{i} A[j]$, at the end of this iteration, Incrementing i for the next iteration of the for loop then preserves the loop invariant.
"},{"location":"Chap02/2.1/#21-4","title":"2.1-4","text":"INSERTION_SORT_NONINCEASE(A)\n for i=2 to A.length()\n key = A[i]\n j = i -1\n while j >=1 and A[j] > key\n A[j+1] = A[j]\n A[j+1] = key \nLINEAR_SEARCH (A,x)\n for i = 1 to A.length()\n if A[i] == x \n return i\n return NIL\n
"},{"location":"Chap02/2.1/#21-5","title":"2.1-5","text":"
"},{"location":"Chap02/2.2/","title":"2.2 Analyzing algorithms","text":""},{"location":"Chap02/2.2/#22-1","title":"2.2-1","text":"ADD_BINARY_INTEGERS(A,B,n)\n C[0:n]\n carry=0\n for i = 1 to n\n C[i-1]=(A[i]+B[i]+carry)%2\n carry =(A[i]+B[i]+carry)/2\n C[n] = carry\n return C;\n
SELECTION_SORT(A)\n for i = 1 to A.length()-1\n minindex = i\n for j = i to A.length()\n if A[j]<A[minindex]\n minindex = j\n swap(A[i],A[minindex]) \n
"},{"location":"Chap02/2.2/#22-3","title":"2.2-3","text":"
"},{"location":"Chap02/2.2/#22-4","title":"2.2-4","text":"
"},{"location":"Chap02/2.3/#23-4","title":"2.3-4","text":"
"},{"location":"Chap02/2.3/#23-6","title":"2.3-6","text":"INSERTIONSORT_RECURXIVE(A,n)\n if n = 1\n return\n INSERTIONSORT_RECURXIVE(A,n-1)\n i = n-1\n key = A[n]\n while A[i]>key and i>=1\n A[i+1]=A[i]\n i--\n A[i+1]=key\nBINARY_SEARCH_ITERATIVE(A,l,r,x)\n while r>=l \n mid = (l+r)/2\n if A[mid] == x\n return mid\n else if A[mid] > x\n r = mid -1\n else\n l = mid +1\n return NIL\nBINARY_SEARCH_RECURVISE(A,l,r,x)\n if(l>r)\n return NIL\n mid= (l+r)/2\n if A[mid] == x\n return x\n if A[mid] < x\n return A[A,mid+1,r,x]\n if A[mid] > x\n return A[A,l,mid-1,x]\nFUCTION(S,x)\n n=S.lenfth()\n MERGE_SORT(S,1,n) //cost $\\Theta n \\lgn$\n i=1\n j=n\n sum=A[i]+A[j]\n while sum !=x and j>i //cost \\Theta n\n if sum > x \n j--\n else i++ \n if sum == x\n return i j\n else\n return NIL\nBUBBLESORT(A,n)\n for i = 1 to n - 1\n for j = n downto i + 1\n if A[j] < A[j-1]\n exchange A[j] with A[j-1]\nHORNER (A,n,x)\n p=0\n for i = n downto 0\n p = A[i] + x * p\n return p\nnaive_polynomia_evaluation (A,n,x)\n p=0\n for i = 0 to n\n term = A[i]\n for j = 1 to i\n term = term * x\n p = term + p\n return p\n
"},{"location":"Chap03/3.1/","title":"3.1 O-notation, $\\Omega$-notation, and $\\Theta$-notation","text":""},{"location":"Chap03/3.1/#31-1","title":"3.1-1","text":"MERGE_INVERSIONS (A,l,r,mid)\n inversions = 0\n n1 = mid - l\n n2 = r - mid -1\n L[0:n1]\n for i = 0 to n1\n L[i] = A[l + i]\n R[0:n2]\n for j = 0 to n2\n R[j] = A[mid + 1 + j]\n i=0 \n j=0\n k=l\n while i <= n1 and j <= n2\n if L[i] <= R[j]\n A[k] = L[i]\n i++\n k++\n else\n A[k] = R[j]\n j++\n k++\n inversions + = n1 - i + 1\n while i <= n1\n A[k] = L[i]\n k++\n i++\n while j <= n2\n A[k] = R[j]\n k++\n j++\nMERGE_SORT_INVERSIONS(A,l,r)\n inversions = 0\n if l >=r\n return inversions\n mid = floor((l+r)/2)\n inversions + = MERGE_SORT_INVERSIONS(A,l,mid)\n inversions + = MERGE_SORT_INVERSIONS(A,mid+1,r)\n inversions + = MERGE_INVERSIONS(A,l,mid,r)\n return inversions\n
"},{"location":"Chap03/3.2/#32-4","title":"3.2-4","text":"
"},{"location":"Chap03/Problems/3-1/#b","title":"b","text":"
"},{"location":"Chap03/Problems/3-6/#d","title":"d","text":"MATRIX_MULTIPLY_RECURSIVE_GENERALIZE(A,B,C,n)\n if n is not exact power of 2\n new_n = next power of 2 of n//O(n)\n use 0 to extend A B C into new_n*new_n matrix//O(n^2)\n MATRIX_MULTIPLY_RECURSIVE(A,B,C,new_n)//Theta(n^3)\n choose the n*n part of C as result by index method//O(1)\n return \n
"},{"location":"Chap04/4.1/#41-3","title":"4.1-3","text":"MATRIX_ADD_RECURSIVEE(A,B,C,n)\n if n==1\n c11=a11+b11\n partition A,B,C into n/2*n/2 submatrixs\n A11,A12,A21,A22;B11,B12,B21,B22;and C11,C12,C21,C22; \n MATRIX_ADD_RECURSIVEE(A11,B11,C11,n/2)\n MATRIX_ADD_RECURSIVEE(A12,B12,C12,n/2)\n MATRIX_ADD_RECURSIVEE(A21,B21,C21,n/2)\n MATRIX_ADD_RECURSIVEE(A22,B22,C22,n/2)\n
"},{"location":"Chap04/4.2/#42-3","title":"4.2-3","text":"Strassen(A,B,C,n)\n if n ==1\n c11=c11+a11*b11\n partition A,B,C into (n/2)*(n/2) submatrixs\n A11,A12,A21,A22;B11,B12,B21,B22;C11,C12,C21,C22\n //get S O(n^2)\n S1=B12-B22\n S2=A11+A12\n S3=A21+A22\n S4=B21-B11\n S5=A11+A22\n S6=B11+B22\n S7=A12-A22\n S8=B21+B22\n S9=A11-A21\n S10=B11+B12\n //creat P O(n^2)\n creat P1...7\n //recursion\n Strassen(A11,S1,P1,n/2)\n Strassen(S2,B22,P2,n/2)\n Strassen(S3,B11,P3,n/2)\n Strassen(A22,S4,P4,n/2)\n Strassen(S5,S6,P5,n/2)\n Strassen(S7,S8,P6,n/2)\n Strassen(S9,S10,P7,n/2)\n //conquer O(n^2)\n C11=C11+P5+P4-P2+P6\n C12=C12+P1+P2\n C21=C21+P3+P4\n C22=C22+P5+P1-P3-P7\n
"},{"location":"Chap04/4.7/#47-5","title":"4.7-5","text":"
"},{"location":"Chap04/Problems/4-4/#c","title":"c","text":"
"},{"location":"Chap04/Problems/4-6/#c","title":"c","text":"findgoodCchip(chips)\n if chips num is zero//end is good chips number equal bad chips \n return NIL\n if chips num is one//must be good chips\n return this one\n if chips num is odd//good chips is more than bad chips \n A=left one chip aside \n else \n A=NIL\n divide rest chips into pairs\n //floor(n/2) operation\n for each pairs//this for loop ensure good chips number are great than or eqaul bad chips\n if the report says at least one of them is bad\n drop both\n else\n drop one in the pair//worst-case: halve the size\n flag=findgoodCchip(chips)\n if flag = NIL and A!=NIL//A is good chip\n return A\n if flag = NIL and A==NIL//no good chip from this call stack\n return flag\n if flag\n return flag\n
"},{"location":"#more-information","title":"More Information","text":"
"},{"location":"Chap01/1.1/#11-2","title":"1.1-2","text":"
"},{"location":"Chap01/1.1/#11-4","title":"1.1-4","text":"
"},{"location":"Chap01/1.1/#11-5","title":"1.1-5","text":"
"},{"location":"Chap01/1.1/#11-6","title":"1.1-6","text":"
"},{"location":"Chap01/1.2/","title":"1.2 Algorithms as a technology","text":""},{"location":"Chap01/1.2/#12-1","title":"1.2-1","text":"
"},{"location":"Chap01/1.2/#12-2","title":"1.2-2","text":"SUM-ARRAY(A,n)\nsum = 0\nfor i = 1 to n\n sum = sum + A[i]\nreturn sum\n
"},{"location":"Chap02/2.1/#21-3","title":"2.1-3","text":"sum = sum + A[i] results $sum = \\sum_{j=1}^{i-1} A[j] + A[i] = \\sum_{j=1}^{i} A[j]$, at the end of this iteration, Incrementing i for the next iteration of the for loop then preserves the loop invariant.
"},{"location":"Chap02/2.1/#21-4","title":"2.1-4","text":"INSERTION_SORT_NONINCEASE(A)\n for i=2 to A.length()\n key = A[i]\n j = i -1\n while j >=1 and A[j] > key\n A[j+1] = A[j]\n A[j+1] = key \nLINEAR_SEARCH (A,x)\n for i = 1 to A.length()\n if A[i] == x \n return i\n return NIL\n
"},{"location":"Chap02/2.1/#21-5","title":"2.1-5","text":"
"},{"location":"Chap02/2.2/","title":"2.2 Analyzing algorithms","text":""},{"location":"Chap02/2.2/#22-1","title":"2.2-1","text":"ADD_BINARY_INTEGERS(A,B,n)\n C[0:n]\n carry=0\n for i = 1 to n\n C[i-1]=(A[i]+B[i]+carry)%2\n carry =(A[i]+B[i]+carry)/2\n C[n] = carry\n return C;\n
SELECTION_SORT(A)\n for i = 1 to A.length()-1\n minindex = i\n for j = i to A.length()\n if A[j]<A[minindex]\n minindex = j\n swap(A[i],A[minindex]) \n
"},{"location":"Chap02/2.2/#22-3","title":"2.2-3","text":"
"},{"location":"Chap02/2.2/#22-4","title":"2.2-4","text":"
"},{"location":"Chap02/2.3/#23-4","title":"2.3-4","text":"
"},{"location":"Chap02/2.3/#23-6","title":"2.3-6","text":"INSERTIONSORT_RECURXIVE(A,n)\n if n = 1\n return\n INSERTIONSORT_RECURXIVE(A,n-1)\n i = n-1\n key = A[n]\n while A[i]>key and i>=1\n A[i+1]=A[i]\n i--\n A[i+1]=key\nBINARY_SEARCH_ITERATIVE(A,l,r,x)\n while r>=l \n mid = (l+r)/2\n if A[mid] == x\n return mid\n else if A[mid] > x\n r = mid -1\n else\n l = mid +1\n return NIL\nBINARY_SEARCH_RECURVISE(A,l,r,x)\n if(l>r)\n return NIL\n mid= (l+r)/2\n if A[mid] == x\n return x\n if A[mid] < x\n return A[A,mid+1,r,x]\n if A[mid] > x\n return A[A,l,mid-1,x]\nFUCTION(S,x)\n n=S.lenfth()\n MERGE_SORT(S,1,n) //cost $\\Theta n \\lgn$\n i=1\n j=n\n sum=A[i]+A[j]\n while sum !=x and j>i //cost \\Theta n\n if sum > x \n j--\n else i++ \n if sum == x\n return i j\n else\n return NIL\nBUBBLESORT(A,n)\n for i = 1 to n - 1\n for j = n downto i + 1\n if A[j] < A[j-1]\n exchange A[j] with A[j-1]\nHORNER (A,n,x)\n p=0\n for i = n downto 0\n p = A[i] + x * p\n return p\nnaive_polynomia_evaluation (A,n,x)\n p=0\n for i = 0 to n\n term = A[i]\n for j = 1 to i\n term = term * x\n p = term + p\n return p\n
"},{"location":"Chap03/3.1/","title":"3.1 O-notation, $\\Omega$-notation, and $\\Theta$-notation","text":""},{"location":"Chap03/3.1/#31-1","title":"3.1-1","text":"MERGE_INVERSIONS (A,l,r,mid)\n inversions = 0\n n1 = mid - l\n n2 = r - mid -1\n L[0:n1]\n for i = 0 to n1\n L[i] = A[l + i]\n R[0:n2]\n for j = 0 to n2\n R[j] = A[mid + 1 + j]\n i=0 \n j=0\n k=l\n while i <= n1 and j <= n2\n if L[i] <= R[j]\n A[k] = L[i]\n i++\n k++\n else\n A[k] = R[j]\n j++\n k++\n inversions + = n1 - i + 1\n while i <= n1\n A[k] = L[i]\n k++\n i++\n while j <= n2\n A[k] = R[j]\n k++\n j++\nMERGE_SORT_INVERSIONS(A,l,r)\n inversions = 0\n if l >=r\n return inversions\n mid = floor((l+r)/2)\n inversions + = MERGE_SORT_INVERSIONS(A,l,mid)\n inversions + = MERGE_SORT_INVERSIONS(A,mid+1,r)\n inversions + = MERGE_INVERSIONS(A,l,mid,r)\n return inversions\n
"},{"location":"Chap03/3.2/#32-4","title":"3.2-4","text":"
"},{"location":"Chap03/Problems/3-1/#b","title":"b","text":"
"},{"location":"Chap03/Problems/3-6/#d","title":"d","text":"MATRIX_MULTIPLY_RECURSIVE_GENERALIZE(A,B,C,n)\n if n is not exact power of 2\n new_n = next power of 2 of n//O(n)\n use 0 to extend A B C into new_n*new_n matrix//O(n^2)\n MATRIX_MULTIPLY_RECURSIVE(A,B,C,new_n)//Theta(n^3)\n choose the n*n part of C as result by index method//O(1)\n return \n
"},{"location":"Chap04/4.1/#41-3","title":"4.1-3","text":"MATRIX_ADD_RECURSIVEE(A,B,C,n)\n if n==1\n c11=a11+b11\n partition A,B,C into n/2*n/2 submatrixs\n A11,A12,A21,A22;B11,B12,B21,B22;and C11,C12,C21,C22; \n MATRIX_ADD_RECURSIVEE(A11,B11,C11,n/2)\n MATRIX_ADD_RECURSIVEE(A12,B12,C12,n/2)\n MATRIX_ADD_RECURSIVEE(A21,B21,C21,n/2)\n MATRIX_ADD_RECURSIVEE(A22,B22,C22,n/2)\n
"},{"location":"Chap04/4.2/#42-3","title":"4.2-3","text":"Strassen(A,B,C,n)\n if n ==1\n c11=c11+a11*b11\n partition A,B,C into (n/2)*(n/2) submatrixs\n A11,A12,A21,A22;B11,B12,B21,B22;C11,C12,C21,C22\n //get S O(n^2)\n S1=B12-B22\n S2=A11+A12\n S3=A21+A22\n S4=B21-B11\n S5=A11+A22\n S6=B11+B22\n S7=A12-A22\n S8=B21+B22\n S9=A11-A21\n S10=B11+B12\n //creat P O(n^2)\n creat P1...7\n //recursion\n Strassen(A11,S1,P1,n/2)\n Strassen(S2,B22,P2,n/2)\n Strassen(S3,B11,P3,n/2)\n Strassen(A22,S4,P4,n/2)\n Strassen(S5,S6,P5,n/2)\n Strassen(S7,S8,P6,n/2)\n Strassen(S9,S10,P7,n/2)\n //conquer O(n^2)\n C11=C11+P5+P4-P2+P6\n C12=C12+P1+P2\n C21=C21+P3+P4\n C22=C22+P5+P1-P3-P7\n
"},{"location":"Chap04/4.7/#47-5","title":"4.7-5","text":"
"},{"location":"Chap04/Problems/4-4/#c","title":"c","text":"
"},{"location":"Chap04/Problems/4-6/#c","title":"c","text":"findgoodCchip(chips)\n if chips num is zero//end is good chips number equal bad chips \n return NIL\n if chips num is one//must be good chips\n return this one\n if chips num is odd//good chips is more than bad chips \n A=left one chip aside \n else \n A=NIL\n divide rest chips into pairs\n //floor(n/2) operation\n for each pairs//this for loop ensure good chips number are great than or eqaul bad chips\n if the report says at least one of them is bad\n drop both\n else\n drop one in the pair//worst-case: halve the size\n flag=findgoodCchip(chips)\n if flag = NIL and A!=NIL//A is good chip\n return A\n if flag = NIL and A==NIL//no good chip from this call stack\n return flag\n if flag\n return flag\n