Understanding how to deal with model uncertainty is key for building resilient agents that can overcome environments that are unforeseen. My research group has studied for years different approaches that build robust agents that can cope with different types of uncertainties. Robustness means that policies are immune to changes in the environment leading to better real time performance. In a sequence of papers we developed robust reinforcement learning and planning algorithms including scaling up such algorithms, learning the uncertainty set online, adapting quickly to unknown uncertainties, and online adaptation. The main application areas here are energy and transport services.
We consider a reinforcement learning scheme for selecting how and what to transfer in 5G networks. The problem at hand is to decide which bit-rate to use and which channels would yield the best tradeoff in terms of power, performance, and cost. We employ multi-objective, multi-agent reinforcement learning to best decide how to transmit the data. In previous work, we proposed to use multi-armed bandit algorithms that ignore the current channel and agent state (see O. Avner and S. Mannor, Multi-User Communication Networks: A Coordinated Multi-Armed Bandit Approach, IEEE/ACM Transactions on Networking ( Volume: 27, Issue: 6, Dec. 2019), https://ieeexplore.ieee.org/document/8875003), but in this project we go further and consider the state of the transmission, the real time requirements, and the changing channel.
We consider the potential role of language as a regularizer in reinforcement learning. The objective is to create hierarchical reinforcement learning algorithms that are explainable by design: they use language to describe what they do. The language models can be learned, dictated, imitated, or created. In a paper that appeared in ICML 2019, we introduced Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We showed how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then used these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.
