// Komputasi Paralel Kuliah : Pendahuluan Yeni Herdiyeni http://www.cs.ipb.ac.id/~yeni/paralel Departemen Ilmu Komputer IPB Semester Genap Deskripsi Membahas kebutuhan dan klasifikasi mesin paralel (SISD, SIMD, MISD, MIMD, SPMD), komunikasi antar prosesor, memori persekutuan (shared ), pengiriman pesan (message passing), jaringan interkoneksi (interconnection network), Desain algoritma paralel, efisiensi dan percepatan pemrosesan paralel, dan contoh aplikasi pemprosesan paralel. Perangkat lunak yang digunakan : MPI (Message Passing Interface) Pengajar Dr. Yeni Herdiyeni, S.Si, M.Komp Hendra Rahmawan, S.Si, M.T Endang Purnama, S.Si, M.Komp Komponen Penilaian UTS UAS Tugas Quiz Project Materi Kuliah. Pendahuluan. Definisi dan motivasi pemrosesan paralel. Arsitektur system, Shared multiprocessor system, Message passing multicomputer distributed, Shared dan klasifikasi memori persekutuan MIMD dan SIMD. Topologi Network. Paradigma pengiriman pesan dengan menggunakan MPI. Prinsip-prinsip Desain Algoritme Paralel 7. Analisis kinerja Pemrosesan paralel. UTS Materi Kuliah # 9. Pemrograman Paralel : Distributed Memory. Pemrograman Paralel. Tinjauan ulang critical section dengan menggunakan Pthread, siknronisasi dengan Semaphore, Implementasi Semaphore dilingkungan MPI. Sorting. Dense Matrix Algorithm. Aplikasi pemrosesan (shared ): problema produsen-konsumen, problema writer reader, problema dining philosophy. Presentasi/diskusi proyek. UAS Buku Ajar Grama, Ananth., Gupta, Anshul., Karypis, George., Kumar, Vipin.. Introduction to Parallel Computing. Second Edition. Pearson Addision Wesley. Quinn, Michael J.. Parallel Programming in C with MPI and OpenMP. International Edition, McGraw-Hill. Wilkinson, Barry & Allen, Michael.. Parallel Programming. nd Edition,Pearson Educational International. Jordan, Harry F., Alaghband Gita.. Fundamentals of Parallel Processing. Prentice Hall.
// Motivation : Classical Science Nature Modern Scientific Method Nature Observation Observation Physical Experimentation Theory Numerical Simulation Physical Experimentation Theory Modern Parallel Architectures Two basic architectural scheme: Distributed Memory Shared Memory Now most computers have a mixed architecture What is Parallel and Distributed computing? Solving a single problem faster using multiple s Parallel = Shared Memory among all s Distributed = Local Memory/ Common Issues: Partition, Synchronization, Dependencies Distributed Memory Shared Memory NETWORK
// Seeking Concurrency Data dependence graphs Data parallelism Functional parallelism Task Parallelism Pipelining Interconnection Networks Uses of interconnection networks Connect processors to shared Connect processors to each other Interconnection media types Shared medium Switched medium switched Most Common Networks Cube, hypercube, n-cube Real Shared switch Memory banks Torus in,,...,n Dim Fat Tree System Bus Virtual Shared Network Mixed Architectures HUB HUB HUB HUB HUB HUB NETWORK
// MPI include file General MPI Program Structure The Message-Passing Programming Paradigm variable declarations Initialize MPI environment Do work and make message passing calls #include <mpi.h> void main (int argc, char *argv[]) { int np, rank, ierr; ierr = MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD,&rank); MPI_Comm_size(MPI_COMM_WORLD,&np); /* Do Some Works */ ierr = MPI_Finalize(); } Terminate MPI Environment Message Foster s Design Methodology Partitioning Communication Agglomeration Mapping Problem Foster s Methodology Partitioning Communication Example program () Calculating the value of by: dx x Mapping Agglomeration
// OK! OK! Sequential Algorithm =... Start calculation! 9 9 = 7 9 OK! Calculated by process OK! Phases of Parallel Algorithm Contoh Row i of A b Inner product computation x +x +x x = Row i of A b c i x +x x = x x x +x = b c Row i of A All-gather communication x +x x +x = Partitioning Communication P P P P
// Communication (CONT..) Communication (cont..) P P P 7 9 P 9 Communication (cont..) Shared multiprocessor using a single bus P P 9 Process A program in a run, a program in the. A high level view of a UNIX process Threads A stream of control in a process. A high level view of threads in a UNIX process
// Parallel Bubble Sort Iteration could start before previous iteration finished if does not overtake previous bubbling action: Virtual Topology Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers nd ed., by B. Wilkinson & M. Allen, @ Pearson Education Inc. All 7 7