FB LinkEdIn Instagram

News & Events

Sep 22-23, 2024
Advances in Deep Learning: Transformers
Picture of Advances in Deep Learning: Transformers

Advances in Deep Learning: Transformers

Prof. Yossi Keshet

Sep 22-23, 2024


Please leave your information here, and we will get back to you shortly when registration opens.

  The Transformer architecture has emerged as a groundbreaking deep learning model, revolutionizing various domains with its powerful representation of sequential data. This course delves into the fascinating world of Transformers, exploring their fundamental concepts, theoretical underpinnings, and practical applications.   We begin by thoroughly examining the Transformer model itself, unpacking its innovative self-attention mechanism and its ability to capture long-range dependencies efficiently. This theoretical foundation will provide a solid understanding of the architectural principles that have propelled Transformers to the forefront of modern deep learning.   Building upon this knowledge, we will explore the transformative impact of Transformers across multiple fields. In natural language processing (NLP), we will study how Transformers have enabled the development of large language models like BERT and ChatGPT, pushing the boundaries of text generation, understanding, and analysis.   Furthermore, we will delve into the realm of automatic speech recognition (ASR), where Transformer-based models such as wav2vec, HuBERT, and Whisper have achieved state-of-the-art performance, revolutionizing the way we interact with spoken language.   Extending our exploration to computer vision, we will examine how Transformers have been adapted to handle visual data, leading to groundbreaking models that have reshaped image and video understanding tasks.   Throughout the course, we will critically analyze the latest research developments, discussing theoretical advancements, practical applications, and the potential future directions of Transformer architectures. By the end of this comprehensive journey, you will possess a deep understanding of Transformers, equipping you with the knowledge and skills to harness their power in your own research or industry projects.  
  • Introduction
  • The roots of the Transformer architecture and how it was developed: RNN, LSTM, sequence-to-sequence, attention mechanism, LAS
  • Transformers structure and the self-attention mechanism
  • Components of Transformers and their usage
  • What is the expressive power of Transformers?
  • Reinforcement learning from human feedback (RLHF) and Direct Preference Optimization (DPO)
  • Efficient Transformers: Flash Networks; RWKV
  • In-context learning
  • Prompts and the limitation of the alignment problem
  • Transformers implementation in NLP
  • Transformers Implementation in Speech
  • Transformers Implementation in Vision
June 18, 2024
Artificial Intelligence and Higher Education - Where Are We Headed?
Picture of Artificial Intelligence and Higher Education - Where Are We Headed?
Prof. Ido Roll: Associate Professor and Senior Vice Provost for the Advancement of Learning and Teaching, Faculty of Education in Science and Technology and Faculty of Data and Decision Sciences Tuesday, June 18, 2024, 17:30-18:30, Faculty of Data and Decision Sciences, Bloomfield Building, 5th Floor, Room 526. Artificial intelligence tools have the potential to revolutionize higher education or become just another item on a long list of tools that couldn't break tradition. I will briefly present some specific developments in the field of artificial intelligence in higher education, followed by a discussion on the topic, both generally and at the Technion in particular. The only rules: no overgeneralizations (robots will kill/serve us all) and no excessive anecdotes (my niece wrote a Eurovision song in Swahili/Yiddish with ChatGPT). The meeting will be held in Hebrew Please register for the meeting here: הרשמה למפגשי בינה מלאכותית בחינוך, הטכניון (google.com)

בינה מלאכותית והחינוך הגבוה - לאן אנו צועדים?

פרופ' עדו רול: פרופסור חבר וסגן המשנה הבכיר לנשיא לקידום הלמידה וההוראה, הפקולטה לחינוך למדע וטכנולוגיה והפקולטה למדעי הנתונים וההחלטות

יום שלישי, 18.6.24, 17:30-18:30, הפקולטה למדעי הנתונים וההחלטות, בניין בלומפילד, קומה 5, חדר 526.

לכלי בינה מלאכותית פוטנציאל לעשות מהפיכה של ממש בחינוך הגבוה, או להיות עוד הזדמנות ברשימה ארוכה של כלים שלא יכלו למסורת. אציג בקצרה מספר התפתחויות ספציפיות לתחום של בינה מלאכותית בחינוך הגבוה ואז נערוך דיון בנושא באופן כללי ובטכניון בפרט. הכללים היחידים: בלי הכללות יתר (הרובוטים יהרגו/ישרתו את כולנו) ובלי יותר מדי אנקדוטות (אחיינית שלי כתבה שיר לאירוויזיון בסוואהילית/יידיש בעזרת צ׳אט).

May 12, 2024
From Academic Innovation To Real World Impact
Picture of From Academic Innovation To Real World Impact
On Monday, 27.5.2024, the Tech.AI Biomed conference, will be held in the Technion Visitors Center. Registration now open!  
August 10, 2023
Meeting with Prof. Erez Karpas, this week with us
Picture of Meeting with Prof. Erez Karpas, this week with us
It was mR Tk ! We tried to touch on the future, to understand how to understand the shared space and how best to organize technologically and humanly for the moment when we will live together - an autonomous machine alongside man.
more details
August 1, 2023
Conclude Prof. Shen-Orr, Prof. Tzipi Horowitz-Kraus and Prof Itzhak Kehat
July 17, 2023
Another fascinating meeting that we were happy to host today at MATAM!
Picture of Another fascinating meeting that we were happy to host today at MATAM!
Today it was Prof. Yael Yaniv from the Faculty of Biomedical Engineering who spoke to us about the integration of AI in the medical world in general and in the field of cardiology in particular.
more details
July 10, 2023
Thank you, Yonatan (Yoni) Savir for the insightful talk on the incredible potential of AI in the medical world!
Picture of Thank you, Yonatan (Yoni) Savir for the insightful talk on the incredible potential of AI in the medical world!
Yoni left us inspired and motivated to explore the vast possibilities that AI brings to the field. Join us next week for another incredible meetup by Tech.AI where Professor Yael Yaniv will discuss the challenges of implementing artificial intelligence tools in medical practice, particularly in the field of cardiology.
more details
June 20, 2023
Thank you Sarah Keren for visiting us here at Matam Park and for sharing your knowledge
Picture of Thank you Sarah Keren for visiting us here at Matam Park and for sharing your knowledge
This talk was held in cooperation with Tech.AI as part of the Tech.AI on Stage series. A big thanks to our community that keeps on growing and joining our meetups and events!
more details
August 14, 2023
"Learning from Noisy Labeled Data Tomorrow" by Dr. Chaim Baskin
As part of the lecture series "Tech.AI on stage" . Tomorrow a fascinating lecture will be given by Dr. Chaim Baskin from the Faculty of Computer Science. Topic: Learning from Noisy Labeled Data Tomorrow, Monday 14.8 at 16:00.
more details
July 20, 2023
Technion and Cincinnati Children’s Hospital Medical Center Advance Leading Data-Driven Research and Education Program