Design and Analysis Algorithm. Ahmad Afif Supianto, S.Si., M.Kom. Pertemuan 04

Transkripsi

1 Design and Analysis Algorithm Ahmad Afif Supianto, S.Si., M.Kom Pertemuan 04

2 Contents 31 2 Asymptotic Analysis Brute Force Algorithm 1 2

3 Asymptotic Analysis

4 Asymptotic Notation Think of n as the number of records we wish to sort with an algorithm that takes f(n) to run. How long will it take to sort n records? What if n is big? We are interested in the range of a function as n gets large. Will f stay bounded? Will f grow linearly? Will f grow exponentially? Our goal is to find out just how fast f grows with respect to n.

5 Asymptotic Notation Memperkirakan formula untuk run-time Misalkan: T(n) = 5n 2 + 6n + 25 Indikasi kinerja algoritma (untuk jumlah data yang sangat besar) T(n) proporsional untuk ordo n 2 untuk data yang sangat besar. 5

6 Asymptotic Notation Indikator efisiensi algoritma bedasar pada OoG pada basic operation suatu algoritma. Penggunaan notasi sebagai pembanding urutan OoG: O (big oh) Ω (big omega) Ө (big theta) t(n) : algoritma running time (diindikasikan dengan basic operation count (C(n)) g(n) : simple function to compare the count 6

7 Classifying functions by their Asymptotic Growth Rates (1/2) asymptotic growth rate, asymptotic order, or order of functions Comparing and classifying functions that ignores constant factors and small inputs. O(g(n)), Big-Oh of g of n, the Asymptotic Upper Bound; W(g(n)), Omega of g of n, the Asymptotic Lower Bound. Q(g(n)), Theta of g of n, the Asymptotic Tight Bound; and

8 Example Example: f(n) = n 2-5n The constant 13 doesn't change as n grows, so it is not crucial. The low order term, -5n, doesn't have much effect on f compared to the quadratic term, n 2. We will show that f(n) = Q(n 2 ). Q: What does it mean to say f(n) = Q(g(n))? A: Intuitively, it means that function f is the same order of magnitude as g.

9 Example (cont.) Q: What does it mean to say f 1 (n) = Q(1)? A: f 1 (n) = Q(1) means after a few n, f 1 is bounded above & below by a constant. Q: What does it mean to say f 2 (n) = Q(n log n)? A: f 2 (n) = Q(n log n) means that after a few n, f 2 is bounded above and below by a constant times n log n. In other words, f 2 is the same order of magnitude as n log n. More generally, f(n) = Q(g(n)) means that f(n) is a member of Q(g(n)) where Q(g(n)) is a set of functions of the same order of magnitude.

10 Big-Oh The O symbol was introduced in 1927 to indicate relative growth of two functions based on asymptotic behavior of the functions now used to classify functions and families of functions

11 Upper Bound Notation We say Insertion Sort s run time is O(n 2 ) Properly we should say run time is in O(n 2 ) Read O as Big-O (you ll also hear it as order ) In general a function f(n) is O(g(n)) if positive constants c and n 0 such that f(n) c g(n) n n 0 e.g. if f(n)=1000n and g(n)=n 2, n 0 > 1000 and c = 1 then f(n 0 ) < 1.g(n 0 ) and we say that f(n) = O(g(n))

12 Asymptotic Upper Bound f(n) c g(n) for all n n 0 g(n) is called an asymptotic upper bound of f(n). We write f(n)=o(g(n)) It reads f(n) is big oh of g(n). c g(n) f(n) g(n) n 0

13 Big-Oh, the Asymptotic Upper Bound This is the most popular notation for run time since we're usually looking for worst case time. If Running Time of Algorithm X is O(n 2 ), then for any input the running time of algorithm X is at most a quadratic function, for sufficiently large n. e.g. 2n 2 = O(n 3 ). From the definition using c = 1 and n 0 = 2. O(n 2 ) is tighter than O(n 3 ).

14 Example 1 for all n>6, g(n) > 1 f(n). Thus the function f is in the big-o of g. that is, f(n) in O(g(n)). g(n) f(n) 6

15 Example 2 g(n) There exists a n 0 s.t. for all n>n 0, f(n) < 1 g(n). Thus, f(n) is in O(g(n)). f(n) 5

16 Example h(n) There exists a n 0 =5, c=3.5, s.t. for all n>n 0, f(n) < c h(n). Thus, f(n) is in O(h(n)). f(n) h(n) 5

17 Example of Asymptotic Upper Bound 4 g(n) = 4n 2 = 3n 2 + n 2 3n for all n 3 > 3n = f(n) Thus, f(n)=o(g(n)). 4g(n)=4n 2 f(n)=3n 2 +5 g(n)=n 2 3

18 Exercise on O-notation Show that 3n 2 +2n+5 = O(n 2 ) 10 n 2 = 3n 2 + 2n 2 + 5n 2 3n 2 + 2n + 5 for n 1 c = 10, n 0 = 1

19 Exercise on O-notation f1(n) = 10 n + 25 n 2 f2(n) = 20 n log n + 5 n f3(n) = 12 n log n n 2 f4(n) = n 1/2 + 3 n log n O(n 2 ) O(n log n) O(n 2 ) O(n log n)

20 Classification of Function : BIG O (1/2) A function f(n) is said to be of at most logarithmic growth if f(n) = O(log n) A function f(n) is said to be of at most quadratic growth if f(n) = O(n 2 ) A function f(n) is said to be of at most polynomial growth if f(n) = O(n k ), for some natural number k > 1 A function f(n) is said to be of at most exponential growth if there is a constant c, such that f(n) = O(c n ), and c > 1 A function f(n) is said to be of at most factorial growth if f(n) = O(n!).

21 Classification of Function : BIG O (2/2) A function f(n) is said to have constant running time if the size of the input n has no effect on the running time of the algorithm (e.g., assignment of a value to a variable). The equation for this algorithm is f(n) = c Other logarithmic classifications: f(n) = O(n log n) f(n) = O(log log n)

22 Lower Bound Notation We say InsertionSort s run time is W(n) In general a function f(n) is W(g(n)) if positive constants c and n 0 such that 0 c g(n) f(n) n n 0 Proof: Suppose run time is an + b Assume a and b are positive (what if b is negative?) an an + b

23 Big W Asymptotic Lower Bound f(n) c g(n) for all n n 0 g(n) is called an asymptotic lower bound of f(n). We write f(n)=w(g(n)) It reads f(n) is omega of g(n). f(n) c g(n) n 0

24 Example of Asymptotic Lower Bound g(n)/4 = n 2 /4 = n 2 /2 n 2 /4 n 2 /2 9 for all n 6 < n 2 /2 7 Thus, f(n)= W(g(n)). g(n)=n 2 g(n)=n 2 f(n)=n 2 /2-7 c g(n)=n 2 /4 6

25 Example: Big Omega Example: n 1/2 = W( log n). Use the definition with c = 1 and n 0 = 16. Checks OK. Let n > 16 : n 1/2 (1) log n if and only if n = ( log n ) 2 by squaring both sides. This is an example of polynomial vs. log.

26 Big Theta Notation Definition: Two functions f and g are said to be of equal growth, f = Big Theta(g) if and only if both f=q(g) and g = Q(f). Definition: f(n) = O(g(n)) means positive constants c 1, c 2, and n 0 such that c 1 g(n) f(n) c 2 g(n) n n 0 If f(n) = O(g(n)) and f(n) = W(g(n)) then f(n) = Q(g(n)) (e.g. f(n) = n 2 and g(n) = 2n 2 )

27 Theta, the Asymptotic Tight Bound Theta means that f is bounded above and below by g; BigTheta implies the "best fit". f(n) does not have to be linear itself in order to be of linear growth; it just has to be between two linear functions,

28 Asymptotically Tight Bound f(n) = O(g(n)) and f(n) = W(g(n)) g(n) is called an asymptotically tight bound of f(n). We write f(n)=q(g(n)) It reads f(n) is theta of g(n). c 2 g(n) f(n) c 1 g(n) n 0

29 Other Asymptotic Notations A function f(n) is o(g(n)) if positive constants c and n 0 such that f(n) < c g(n) n n 0 A function f(n) is (g(n)) if positive constants c and n 0 such that c g(n) < f(n) n n 0 Intuitively, o() is like < O() is like () is like > W() is like Q() is like =

30 Examples 1. 2n 3 + 3n 2 + n = 2n 3 + 3n 2 + O(n) = 2n 3 + O( n 2 + n) = 2n 3 + O( n 2 ) = O(n 3 ) = O(n 4 ) 2. 2n 3 + 3n 2 + n = 2n 3 + 3n 2 + O(n) = 2n 3 + Q(n 2 + n) = 2n 3 + Q(n 2 ) = Q(n 3 )

31 Examples (cont.) 3. Suppose a program P is O(n 3 ), and a program Q is O(3 n ), and that currently both can solve problems of size 50 in 1 hour. If the programs are run on another system that executes exactly 729 times as fast as the original system, what size problems will they be able to solve?

32 Example (cont.) n 3 = 50 3 * n = 3 50 * n = 50 * 729 n = log 3 (729 * 3 50 ) n = n = log 3 (729) + log n = 50 * 9 n = 6 + log n = 50 * 9 = 450 n = = 56 Improvement: problem size increased by 9 times for n 3 algorithm but only a slight improvement in problem size (+6) for exponential algorithm.

33 More Examples (a) 0.5n 2-5n + 2 = W( n 2 ). Let c = 0.25 and n 0 = n 2-5n + 2 = 0.25( n 2 ) for all n = 25 (b) 0.5 n 2-5n + 2 = O( n 2 ). Let c = 0.5 and n 0 = ( n 2 ) = 0.5 n 2-5n + 2 for all n = 1 (c) 0.5 n 2-5n + 2 = Q( n 2 ) from (a) and (b) above. Use n 0 = 25, c 1 = 0.25, c 2 = 0.5 in the definition.

34 More Examples (d) 6 * 2 n + n 2 = O(2 n ). Let c = 7 and n 0 = 4. Note that 2 n = n 2 for n = 4. Not a tight upper bound, but it's true. (e) 10 n = O(n 4 ). There's nothing wrong with this, but usually we try to get the closest g(n). Better is to use O(n 2 ).

35 How to show a function f(n) is in/not in O(g(n)). f(n) is in O(g(n)): find a constant c and large n 0 such that for all n > n 0, f(n) < c g(n). f(n) is not in O(g(n)): for any constant c and any large n 0, we can find a m such that f(m) > c g(m). Usually it is more difficult to proof that a function f(n) is not in the big-o of another function g(n). c,n 0 n>n 0 s.t. f(n)< c g(n) c,n 0 n>n 0 s.t. f(n)>= c g(n)

36 Big O Again!!!! O(1) The cost of applying the algorithm can be bounded independently of the value of n. This is called constant complexity. O(log n) The cost of applying the algorithm to problems of sufficiently large size n can be bounded by a function of the form k log n, where k is a fixed constant. This is called logarithmic complexity. O(n) linear complexity O(n log n) n lg n complexity O(n 2 ) quadratic complexity

37 Big O Again!!!! O(n 3 ) cubic complexity O(n 4 ) quartic complexity O(n 32 ) polynomial complexity O(c n ) If constant c 1, then this is called exponential complexity O(2 n ) exponential complexity O(e n ) exponential complexity O(n!) factorial complexity O(n n )

38 Practical Complexity t < f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

39 Practical Complexity t < f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

40 Practical Complexity t < f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

41 Practical Complexity t < f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

42 Algoritma Brute Force 1

43 Definisi Brute Force Brute force : pendekatan straight forward untuk memecahkan suatu masalah Algoritma brute force memecahkan masalah dengan sangat sederhana, langsung, danjelas (obvious way)

44 Contoh-contoh (Berdasarkan pernyataan masalah) Mencari elemen terbesar (terkecil) Persoalan: Diberikan sebuah array yang beranggotakan n buah bilangan bulat (a 1, a 2,..., a n ).Carilah elemen terbesar di dalam array tersebut. Algoritma brute force: bandingkan setiap elemen array untuk menemukan elemen terbesar Kompleksitas O(n)

45 Contoh-contoh (Berdasarkan pernyataan masalah) Mencari elemen terbesar (terkecil)

46 Contoh-contoh (Berdasarkan pernyataan masalah) Pencarian beruntun (Sequential Search) Persoalan: Diberikan array yang berisi n buah bilangan bulat (a 1, a 2,..., a n ). Carilah nilai x di dalam array tersebut. Jika x ditemukan, maka keluarannya adalah indeks elemen array, jika x tidak ditemukan, maka keluarannya adalah 0. Algoritma brute force (sequential serach): setiap elemen array dibandingkan dengan x. Pencarian selesai jika x ditemukan atau elemen array sudah habis diperiksa. Kompleksitas O(n)

47 Contoh-contoh (Berdasarkan pernyataan masalah) Pencarian beruntun (sequential search)

48 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Menghitung a n (a > 0, n adalah bilangan bulat tak-negatif) Definisi: a n = a x a x... x a (n kali), jika n>0 = 1, jika n = 0 Algoritma brute force: kalikan 1 dengan a sebanyak n kali Kompleksitas O(n)

49 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Menghitung pangkat

50 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Menghitung n! (n bilangan bulat tak-negatif) Definisi: n!= n, jika n>0 = 1, jika n = 0 Algoritma brute force: kalikan n buah bilangan, yaitu 1, 2, 3,..., n, bersama-sama Kompleksitas O(n)

51 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Menghitung faktorial

52 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Mengalikan dua buah matriks, A dan B Definisi: Misalkan C = A B dan elemenelemen matrik dinyatakan sebagai c ij, a ij, dan b ij Algoritma brute force: hitung setiap elemen hasil perkalian satu per satu, dengan cara mengalikan dua vektor yang panjangnya n.

53 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Mengalikan dua buah matriks

54 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Menemukan semua faktor dari bilangan bulat n (selain dari 1 dan n itu sendiri). Definisi: Bilangan bulat a adalah faktor dari bilangan bulat b jika a habis membagi b. Algoritma brute force: bagi n denga n setiap i = 2, 3,..., n 1. Jika n habis membagi i, maka I adalah faktor dari n.

55 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Menemukan faktor bilangan bulat n

56 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Uji keprimaan Persoalan: Diberikan sebuah bilangan bilangan bulat positif. Ujilah apakah bilangan tersebut merupakan bilangan prima atau bukan. Definisi: bilangan prima adalah bilangan yang hanya habis dibagi oleh 1 dan dirinya sendiri. Algoritma brute force: bagi n dengan 2 sampai n 1. Jika semuanya tidak habis membagi n, maka n adalah bilangan prima Perbaikan: cukup membagi dengan 2 sampai n saja

57 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Uji bilangan prima

58 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Algoritma Pengurutan Brute Force, Algoritma apa yang paling lempang dalam memecahkan masalah pengurutan? Bubble sort dan selection sort! Kedua algoritma ini memperlihatkan teknik brute force dengan jelas sekali.

59 Bubble Sort Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Mulai dari elemen ke-n: 1. Jika s n < s n-1, pertukarkan 2. Jika s n-1 < s n-2, pertukarkan Jika s 2 < s 1, pertukarkan 1 kali pass Ulangi lagi untuk pass ke-i

60 Bubble sort Contoh-contoh (Berdasarkan definisi konsep yang terlibat)

61 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Selection Sort Pass ke 1: 1.Cari elemen terbesar mulai di dalam s[1..n] 2.Letakkan elemen terbesar pada posisi n (pertukaran) Pass ke-2: 1.Cari elemen terbesar mulai di dalam s[1..n - 1] 2.Letakkan elemen terbesar pada posisi n - 1 (pertukaran) Ulangi sampai hanya tersisa 1 elemen

62 Selection sort Contoh-contoh (Berdasarkan definisi konsep yang terlibat)

63 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Mengevaluasi polinom Persoalan: Hitung nilai polinom p(x)=a n x n +a n-1 x n a 1 x +a 0 untuk x = t. Algoritma brute force: x i dihitung secara brute force (seperti perhitungan a n ). Kalikan nilai x i dengan a i, lalu jumlahkan dengan suku-suku lainnya.

64 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Analisa polinom

65 Contoh-contoh (Berdasarkan definisi konsep yang terlibat) Perbaikan (improve)

66 Karakteristik Algoritma Brute Force Algoritma brute force umumnya tidak cerdas dan tidak cepat, karena ia membutuhkan jumlah langkah yang besar dalam penyelesaiannya. Kata force mengindikasikan tenaga ketimbang otak Kadang-kadang algoritma brute force disebut juga algoritma naif (naïve algorithm).

67 Karakteristik Algoritma Brute Force Algoritma brute force lebih cocok untuk masalah yang berukuran kecil. Pertimbangannya: sederhana, Implementasinya mudah Algoritma brute force sering digunakan sebagai basis pembanding dengan algoritma yang lebih cepat.

68 Karakteristik Algoritma Brute Force Meskipun bukan metode yang cepat, hampir semua masalah dapat diselesaikan dengan algoritma brute force. Sukar menunjukkan masalah yang tidak dapat diselesaikan dengan metode brute force. Bahkan, ada masalah yang hanya dapat diselesaikan dengan metode brute force. Contoh: mencari elemen terbesar di dalam array.

69 Tugas Buatlah algoritma berikut dengan kompleksitasnya Pencocokan String (String Matching) Mencari pasangan titik yang jaraknya terdekat (Closest pairs) Travelling Salesman Problem (TSP) Knapsack Problem

70 Click to edit subtitle style