Parallel performance<\/h3>\n![\"algorithm200x200\"](\"\/pasi\/files\/2010\/10\/algorithm200x200.png\" "\\"algorithm200x200\\"")
<\/a><\/h3>\nOne of the goals in high-performance computing (HPC) is to achieve the best possible performance<\/em> from parallel computers. But performance can be measured in a variety of ways; in HPC, we are generally interested in the speed at which floating-point computation is done. \u00a0Thus, we use for example the units of giga-flop\/s\u00a0(a billion floating point operations per second).<\/p>\nA given computer will have a maximum floating point performance, and a given code can never exceed this figure. Often, it falls short by a vast amount, especially in parallel. \u00a0It is much more difficult to achieve parallel performance than sequential performance, and it is also more difficult to measure. \u00a0Factors that become critical are synchronization, load balance, and communication costs.<\/p>\n
This part of the course will focus on parallel performance analysis; we will cover:<\/p>\n
\n- performance measures<\/li>\n
- limits to performance<\/li>\n
- scalability<\/li>\n
- load balancing<\/li>\n
- real-world parallelism<\/li>\n<\/ul>\n
Parallel algorithms<\/h3>\n
Central to the subject of parallel computing are algorithms. \u00a0As a general rule, no <\/em>parallelism of any significance occurs naturally; it must be created by removing dependencies<\/em>. \u00a0Often, entirely new<\/em> algorithms have to be created to achieve a certain task in parallel.<\/p>\nOne aspect in particular that requires efficient algorithms is collective operations<\/em>. These are operations involving a group of processors to produce a single data structure or task result. \u00a0For example, a barrier<\/em> in shared memory computing, and a broadcast<\/em> in message passing are collective operations.<\/p>\nIn this part of the course, we will cover:<\/p>\n
\n- collective operations: prefix and scan<\/li>\n
- tree\/ring algorithms<\/li>\n
- matrix-vector product: the power method<\/li>\n<\/ul>\n
We will also discuss briefly some aspects of linear systems of equations, in particular their dependence analysis.<\/p>\n
Supplementary material<\/h2>\n
<\/a><\/h3>\nOne of the goals in high-performance computing (HPC) is to achieve the best possible performance<\/em> from parallel computers. But performance can be measured in a variety of ways; in HPC, we are generally interested in the speed at which floating-point computation is done. \u00a0Thus, we use for example the units of giga-flop\/s\u00a0(a billion floating point operations per second).<\/p>\n A given computer will have a maximum floating point performance, and a given code can never exceed this figure. Often, it falls short by a vast amount, especially in parallel. \u00a0It is much more difficult to achieve parallel performance than sequential performance, and it is also more difficult to measure. \u00a0Factors that become critical are synchronization, load balance, and communication costs.<\/p>\n This part of the course will focus on parallel performance analysis; we will cover:<\/p>\n Central to the subject of parallel computing are algorithms. \u00a0As a general rule, no <\/em>parallelism of any significance occurs naturally; it must be created by removing dependencies<\/em>. \u00a0Often, entirely new<\/em> algorithms have to be created to achieve a certain task in parallel.<\/p>\n One aspect in particular that requires efficient algorithms is collective operations<\/em>. These are operations involving a group of processors to produce a single data structure or task result. \u00a0For example, a barrier<\/em> in shared memory computing, and a broadcast<\/em> in message passing are collective operations.<\/p>\n In this part of the course, we will cover:<\/p>\n We will also discuss briefly some aspects of linear systems of equations, in particular their dependence analysis.<\/p>\n\n
Parallel algorithms<\/h3>\n
\n
Supplementary material<\/h2>\n