High Perfomance Computing (11165)

Parallel architectures
Parallel programming paradigms
General parallel algorithm design methodologies
Problem-type specific parallel algorithmic strategies
Techniques to optimize the performance of parallel algorithms in the studied parallel architectures

Know how to:

Optimize sequential programs
Implement parallel algorithms to handle compute-intensive or data-intensive applications on GPU(s)(CUDA or OpenCL) and/or on distributed memory architectures (MPI or Spark).
Measure and analyze the performance of a parallel computation.

Soft-Skills:

Reason about and critically evaluate the algorithmic and technological alternatives available for solving a problem.
Design a solution for a problem a before going into the implementation phase.
Team work.
Structure and write project reports.

Motivation

Why Parallel Computing?
Why High Performance Computing?
The convergence of the Big Compute and Big Data trends of thought

Fundamentals of Parallel Computing

Parallel Architectures
Parallel Performance
Parallel Programming Paradigms
Designing Parallel Algorithms

Parallel Computing

Parallel Programming Patterns and Strategies
Shared Memory Processing
GPU Computing
Message-passing programming

Data-Centric High Performance Computing

MapReduce
Apache Spark

Parallel Algorithms (Putting it All Together)

Graph processing algorithms
Machine learning algorithms

The Future of High Performance Computing

Challenges in the industry
Open research topics

As usual in HPC courses, there is no required textbook. There are several books that cover the fundamental concepts of HPC. A list follows:

T Sterling, M Brodowicz, M Anderson. High Performance Computing - Modern Systems and Practices. Morgan Kaufmann, 2017.
T Rauber and G Rünger. Parallel Programming for Multicore and Cluster Systems. Springer, 2013
P Pacheco.An Introduction to Parallel Programming.Morgan Kaufmann,2011
Ian Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995,

GPU computing:

NVIDIA documentation
J Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming, Addison-Wesley, 2010

Data-intensive computing:

A Shook and D Miner. MapReduce Design Patterns.O’Reilly Media, 2012
High Performance Spark. O’Reilly Media, 2017

Students should have knowledge about computer architecture, computer networks, and operating systems, and good programming skills. The exercises will use the C/C++ and Java programming languages.

	Hours per credit		28
	Hours per week	Weeks	Hours
Aulas prÃ¡ticas e laboratoriais			26.0
Aulas teÃ³ricas			26.0
AvaliaÃ§Ã£o			4.0
Self study			50.0
Project			62.0
Total hours			168
ECTS			6.0