Computer Science,
Control and
Geoinformation Doctorate

Reinforcement Learning for Runtime Performance Management - March/April 2024

Title

Reinforcement Learning for Runtime Performance Management.

Speaker

Gabriele Russo Russo, Tor Vergata University of Rome

When and Where

  • Schedule: March 25th, April 8th, 15th, 23rd and 29th 2024, from 15:00 to 17:00 (10 hours, 5 lectures)

All lectures in sala Riunioni (Ingegneria della Informazione building, ground floor, room AT 07)

Main Topics

RL comprises a collection of techniques revolving around the basic idea of learning to make optimal decisions through interaction with the controlled system.

In the first part of the course, we will introduce the main concepts underpinning the RL framework and the modeling steps required for its adoption, from the definition of state and action spaces (and the associated scalability concerns) to the selection of suitable reward functions. We will first present baseline model-free approaches, e.g., Q-learning and SARSA, and then move to more advanced concepts such as post-decision states and model-based RL, which accelerate the learning process exploiting available knowledge about the system.

In the second part of the course, we will present advanced RL topics: we will show how Function Approximation and Deep Learning techniques can be integrated in RL algorithms, to build approximate representations of the state space and achieve near-optimal solutions with reduced memory demand. these lectures will blend theory with practical examples, demonstrating the use of RL for real time auto-scaling in Cloud/edge applications.

Short Bio

Gabriele Russo Russo is a Research Associate in the Department of Civil Engineering and Computer Science Engineering of Tor Vergata University of Rome. He received his PhD degree in 2021 from Tor Vergata University of Rome. His research interests span the area of distributed computing systems with emphasis on run-time management and adaptation of distributed systems.

He has published more than 20 papers in international journals, books, and conference proceedings. He was co-chair of the Int’l Workshop on Artificial Intelligence for Autonomous computing Systems (AI4AS’23, co-located with IEEE ACSOS 2023), and the First Workshop on Quality of Service-aware Serverless Computing (QServ’23, co-located with IEEE/ACM UCC 2023). He has served as a reviewer for top-ranked journals (including ACM CSUR, IEEE TCC, IEEE TPDS, IEEE TNSM, Elsevier FGCS). Since 2021, he has been a member of the Technical Review Board of IEEE Transactions on Parallel and Distributed Systems.