The mathematical theory of voting goes back at least 240 years to the Condorcet Jury Theorem, and mainly deals with the question of finding a rule, or a “function” that best aggregates the preferences of many people. Yet the implicit underlying assumption, that all (or even most) people actually vote is rarely met in practice. We quantify the bias in the outcome as more people fail to vote, and study the effect of possible remedies such as voting by proxy.
Truth discovery is a general name for statistical methods aimed to extract the correct answers to questions, based on multiple answers coming from noisy sources. For example, workers in a crowdsourcing platform. We suggest a simple heuristic for estimating workers’ competence using average proximity to other workers. We prove this estimates well the actual competence level and enables separating high and low quality workers in a wide spectrum of domains and statistical models.
The focus is on recognizing and analyzing the challenges that arise when autonomous agents with different capabilities need to interact and collaborate on unknown tasks, on providing methods for the automated design of these environments to promote collaboration, and on specifying guarantees regarding the quality of the design solutions produced by our suggested methods. This research combines data-driven approaches with symbolic AI techniques and involves both theoretical work and evaluations on multi-agent reinforcement learning settings and on multi robot systems.
Promoting multi-agent collaboration via dynamic markets of information and skills in which AI agents and robots trade their physical capabilities and their ability to acquire new information. The value of these traded commodities is dynamically computed based on the agents’ objectives, sensors and actuation capabilities as well as their ability to communicate with each other and ask for assistance. This framework maximizes performance and team resilience, without relying on a centralized controller.
Most current approaches to robotic planning separate the low-level planning of basic behaviors and the high-level search for a sequence of behaviors that will accomplish a task. However, in complex settings such as packing, personal assistance, and cooking, this dichotomous view becomes inefficient, especially in environments shared by multiple autonomous agents. We therefore offer new ways for integrating task-level considerations when planning the robot’s movement, and for propagating motion-planning considerations into task planning.
This project seeks to build an intelligent environment that provides Data Science students with opportunities to construct their own data models. Though students often make mistakes in these tasks, we use an AI-driven combination of challenges, feedback, and instruction to help students become adaptive experts who are able to complex relationship in unfamiliar data.
The main objective of the study is to develop Creative Thinking measurement tools in the context of scientific and authentic problem solving. This project is focused on the synergy between the measurement of the process, using learning analytics techniques, and measurement of the products of Creative Thinking. It serves as a case-study for intelligent assessment of complex constructs.
This project seeks to support learning of scientific literacies with virtual labs using student facing learning-analytics dashboards. The design of student-facing dashboards is challenging due to the ill-defined nature of scientific skills and attitudes. As students are free to explore an open-ended design space, and as there are no “correct” answers, we identify metrics that can be extracted and interpreted so that students can understand the processes of doing science.
We study basic human decision making and learning processes when making repeated and/or sequential choice. Understanding the basic processes in these very common settings (e.g. driving, behavior in pandemics, using smartphone apps, health decisions) both improves our ability to predict behavior and to design mechanisms and policies that are robust to the likely behaviors of systems’ users.
We integrate psychological theories and models of human decision making into machine learning systems to predict human decision making in state-of-the-art levels. Focusing on the most fundamental choice task from behavioral economics and using the largest datasets currently available, we study which theories and models, which types of machine learning algorithms and tools, and which methods of integration lead to the best out-of-sample predictions.
Creativity is a complex, multidimensional, elusive concept, that is vital to personal and societal needs. In this project, we leverage computational network science methods with machine learning, combined with psycholinguistics to develop a computational model to predict ones’ creative ability level. We analyze a simple semantic fluency task (name all the animals you can think of) as a mental navigation process over a multiplex cognitive network. Features of this mental navigation process are then being used to build creativity prediction and classification models.
Modern recommendation platforms have become complex, dynamic eco-systems. Platforms often rely on machine learning models to successfully match users to content, but most methods neglect to account for how they affect user behavior, satisfaction, and well-being of over time. Here we propose a novel dynamical-systems perspective to recommendation that allows to reason about, and control, macro-temporal aspects of recommendation policies as they relate to user behavior.
The task of optimizing machines to support human decision-making is often conflated with that of optimizing machines for accuracy, even though they are materially different. Whereas typical learning systems prescribe actions through prediction, our framework learns to to reframe problems in a way that directly supports human decisions. Using a novel human-in-the-loop training procedure, our framework learns problem representations that directly optimize human performance.
Machine learning has become imperative for informing decisions that affect the lives of humans across a multitude of domains. But when people benefit from certain predictive outcomes, they are prone to act strategically to improve those outcomes. Our goal in this project is to develop a practical learning framework that accounts for how humans behaviourally respond to classification rules. Our framework provides robustness while also providing means to promote favourable social outcomes.
Non-invasive brain computer interfaces (BCIs) provide direct communication link from the brain to external devices. We develop non-invasive BCIs that are based on interpreting EEG measurements to identify user’s desired selection, action or movement. We focus on developing self- correction capabilities, based on error-related potentials (ErrPs), which are evoked in the brain when errors are detected. We investigate ErrPs, develop classifiers for detecting them and methods to integrate them to improve BCIs. This project is funded by Dr. Maria Ascoli Rossi Research Grant.
We are interested in developing intelligent systems that support students’ learning. One project develops “invention activities” for students learning data science, supported by automatic feedback mechanisms. This approach aims to facilitate improved understanding of data science concepts by letting students invent and test quantitative measures. In a second project, we are developing an intelligent system for supporting student collaboration on joint project. We are designing algorithms for analyzing students’ and design interfaces that will provide collaborators with actionable information regarding the group’s progress.
The growing trend of shifting from classroom to distance learning in ethics education programs raises the need to examine ways for adapting best instructional practices to online modes. To address this need, the current study is set to apply a social constructivist approach to an online course in research ethics and to examine its effect on the learning outcomes of science and engineering graduate students.
AugmentedWorld is an open, collaborative, and interactive location-based platform, purposefully designed to provide science teachers and students an online tool for generating multimedia-rich questions. It is based on the notion that questions are the source of all knowledge and that students should be skilled in generating questions and not only in answering them. Our goal is to examine the cognitive and social impact of AugmentedWorld on science teachers and students.
At the brink of the fourth industrial revolution, a significant transition is taking place from simple digitization to innovation-based technology. Innovation, the process of generating new ideas and transforming them into practical solutions, is a catalyst for progress in our fast-changing world. The goal of our study is to assess the innovation level of engineering students’ team projects and to examine the relationships between project innovation and team heterogeneity in online and F2F environments.
Database schema matching is a challenging task that call for improvement for several decades. Automatic algorithms fail to provide reliable enough results. We use human matching to overcome algorithm failures and vice versa. We refer to human and algorithmic matchers as imperfect matchers with different strengths and weaknesses. We use insights from cognitive research to predict human matchers behavior and identify those who can do better than others. We then merge their responses with algorithmic outcomes and get better results.
As a team, we currently work on several projects, with several challenging tasks, including riddle solving, database schema matching, and text design in a word processor. In all cases we aim to predict people’s confidence in their success in the task based on their mouse movements before choosing their response and while rating their confidence on a continuous scale.
Consider a setting where one agent holds private information and would like to use her information to motivate another agent to take some action. When agents’ interests co-incide the answer is easy – disclose the full information. In this project we study the optimal information design when agents’ incentives are mis-aligned.
Improvements in training speed are needed to develop the next generation of deep learning models. To perform such a massive amount of computation in a reasonable time, it is parallelized across multiple GPU cores. Perhaps the most popular parallelization method is to use a large batch of data in each iteration of SGD, so the gradient computation can be performed in parallel on multiple workers. We aim to enable massive parallelization without performance degradation, as commonly observed.
We aim to improve the resource efficiency of deep learning (e.g., energy, bandwidth) for training and inference. Our focus is decreasing the numerical precision of the neural network model is a simple and effective way to improve their resource efficiency. Nearly all recent deep learning related hardware relies heavily on lower precision math. The benefits are a reduction in the memory required to store the neural network, a reduction in chip area, and a drastic improvement in energy efficiency.
Significant research efforts are being invested in improving Deep Neural Networks (DNNs) via various modifications. However, such modifications often cause an unexplained degradation in the generalization performance DNNs to unseen data. Recent findings suggest that this degradation is caused by changes to the hidden algorithmic bias of the training algorithm and model. This bias determines which solution is selected from all solutions which fit the data. We aim to understand and control this algorithmic bias.
This project seeks to elucidate the mechanisms of information storage and processing in machine learning systems of human language, by (a) measuring localization and distributivity of information in complex models; (b) discovering causal relationships between model components and automatic (potentially biased) decisions; and (c) making language processing systems more interpretable and controllable. The research is expected to promote responsible and accountable adoption of language technology.
Consider a group of workers who answered questions, which have a correct yet unknown answers. The workers are heterogenous, they could be ordinary people, trained volunteers, a panel of experts, different computer algorithms, or a mix of all the above. Our approach is based on empirical Bayes methods and the aim is to construct an algorithm that aggregates all workers’ answers to a single output that is close to the unknown truth. (MSc student: Tsviel Ben-Shabat, co-advisor: Reshef Meir)
The goal of this research is to design classifiers robust to strategic behavior of the agents being classified. Here strategic behavior means incurring some cost in order to improve personal features and thus classification. This improvement can be superficial – i.e., gaming the classifier – or substantial, thus leading to true self-improvement. In the latter case (and only in this case), the robust classifier should actually encourage strategic behavior.
Consider two strategic players, one more informed about the state of the world and the other less informed. How should the more informed side select what data to communicate to the other side, in order to inspire actions that benefit goals like social welfare? Can this be done under constraints such as privacy, limited communication, limited attention span, fairness, etc.?
In the new economy, new kinds of organizations emerge. Among others, cross-sectorial collaborations are created, based on the recognition that they benefit all sectors: governmental organizations and local authorities (1st sector), for-profit organizations (2nd sector), and non-governmental non-profit organizations (3rd sector). This research, conducted in collaboration with tech-organizations in the context of STEM education, explores benefits that each sector earns from the collaboration.
Data science is a new interdisciplinary field of research that focuses on extracting knowledge and value from data. As data science is becoming relevant for many scientific, engineering and social research and applications, new data science education programs are being launched and adequate teaching methods are needed for different learning populations. This research, conducted by my doctoral student, Koby Mike, explores the essence of this new evolving discipline – data science education.
The effectiveness of learning systems depends on both the attributes of the learner and the teacher. Indeed, an optimal setup for learning is when the student and teacher/environment operate collaboratively to enhance learning, where the teacher’s task is to develop an appropriate learning curriculum that facilitates learning by the student. We develop approaches to enhance agents’ learning within a curriculum setting, focusing on the model-based Reinforcement Learning agents and continuous control settings.
Effective learning from data requires prior assumptions, referred to as inductive bias. A fundamental question pertains to the source of a ‘good’ inductive bias. One natural way to form such a bias is through lifelong learning, where an agent continually interacts with the world through a sequence of tasks, aiming to improve its performance on future tasks based on the tasks it has encountered so far. We develop a theoretical framework for incremental inductive bias formation, and demonstrate its effectiveness in problems of sequential learning and decision making.
