The analysis of large volumes of data is essential in HPC. Big data analytics covers concepts and tools to perform this task traditionally in the cloud environment – that, compared to HPC systems, utilizes cheap and less-reliable hardware. However, as tools and concepts matured, they are now applied in HPC environments as well.
HPC workloads utilize tools and methodology from Data Science (DS) and Artificial Intelligence (AI) to process data in order to obtain results quickly. AI can be used inside simulations or to steer workflows, while data science can be used to find interesting patterns inside the data.