Philip Tetlock

Philip Tetlock
  • Leonore Annenberg University Professor in Democracy and Citizenship
  • Professor of Management
  • Professor of Psychology

Contact Information

  • office Address:

    3203 SH-DH
    3620 Locust Walk
    Philadelphia, PA 19104

Research Interests: social and cultural psychology, decision processes

Links: CV, Personal Website, Department of Psychology

Overview

Education

Ph.D. Yale University, 1979 (Psychology);
M.A. University of British Columbia, 1976;
B.A. University of British Columbia, 1975;

Academic Experience

2011 -present Leonore Annenberg University Professor, School of Arts and Sciences (Psychology) and Wharton School (Management), University of Pennsylvania;
2002- 2010 Mitchell Endowed Professorship, Haas School of Business, University of California Berkeley;
2005-2006 Russell Sage Scholar;
1996-2001 Harold Burtt Professor of Psychology and Political Science, The Ohio State University;
1993-1994 Fellow, Center for Advanced Study in the Behavioral Sciences, Stanford;
1993-1995 Distinguished Professor, University of California, Berkeley;
1988-1995 Director, Institute of Personality and Social Research, University of California, Berkeley;
1987-1996 Professor, Department of Psychology, University of California, Berkeley;
1984-1987 Associate Professor, Department of Psychology, University of California, Berkeley;
1980-1995 Research Psychologist, Survey Research Center, University of California, Berkeley;
1979-1984 Assistant Professor, Department of Psychology, University of California, Berkeley;

Administrative Experience

Group Chair, Organizational Behavior and Industrial Relations, Haas School of Business, University of California, Berkeley, 2002-present;
Associate Dean for Academic Affairs, Haas School of Business, University of California, Berkeley, 2003-2004;
Director, Ph.D. programs, Haas School of Business, University of California, Berkeley;
Director, Institute of Personality Assessment and Research (renamed in 1992 as Institute of Personality and Social Research), University of California, Berkeley, 1988-1995.

Continue Reading

Research

  • Pavel Atanasov, Jens Witkowski, Barbara Mellers, Philip Tetlock (2024), Crowd prediction systems: Markets, polls, and elite forecasters, International Journal of Forecasting.

    Abstract: What systems should we use to elicit and aggregate judgmental forecasts? Who should be asked to make such forecasts? We address these questions by assessing two widely-used crowd prediction systems: prediction markets and prediction polls. Our main test compares a prediction market against team-based prediction polls, using data from a large, multi-year forecasting competition. Each of these two systems uses inputs from either a large, sub-elite or a small, elite crowd. We find that small, elite crowds outperform larger ones, whereas the two systems are statistically tied. In addition to this main research question, we examine two complementary questions. First, we compare two market structures, continuous double auction (CDA) markets and logarithmic market scoring rule (LMSR) markets, and find that the LMSR market produces more accurate forecasts than the CDA market, especially on low-activity questions. Second, given the importance of elite forecasters, we compare the talent-spotting properties of the two systems, and find that markets and polls are equally effective at identifying elite forecasters. Overall, the performance benefits of "superforecasting" hold across systems. Managers should move towards identifying and deploying small, select crowds to maximize forecasting performance.

  • Philip Tetlock, Christopher Karvetski, Ville Satopää, Kevin Chen (2024), Long-range subjective-probability forecasts of slow-motion variables in world politics: Exploring limits on expert judgment, Futures and Foresight Science.

    Abstract: Skeptics see long-range geopolitical forecasting as quixotic. A more nuanced view is that although predictability tends to decline over time, its rate of descent is variable. The current study gives geopolitical forecasters a sporting chance by focusing on slow-motion variables with low base rates of change. Analyses of 5, 10 and 25-year cumulative-risk judgments made in 1988 and 1997 revealed: (a) Specialists beat generalists at predicting nuclear proliferation but not shifting nation-state boundaries; (b) Some counterfactual interventions—e.g., Iran gets the bomb before 2022—boosted experts’ edge but others—e.g., nuclear war before 2022—eliminated it; (c) accuracy fell faster on topics where expertise conferred no edge in shorter-range forecasts. To accelerate scientific progress, we propose adversarial collaborations in which clashing schools of thought strike Bayesian reputational bets on divisive issues and use Lakatosian scorecards to incentivize the honoring of bets.

  • Ben Powell, Ville Satopää, Niall MacKay, Philip Tetlock (2024), Skew-Adjusted Extremized-Mean: A Simple Method for Identifying and Learning From Contrarian Minorities in Groups of Forecasters, Decision, 11 (1), pp. 173-193.

    Abstract: Recent work in forecast aggregation has demonstrated that paying attention to contrarian minorities among larger groups of forecasters can improve aggregated probabilistic forecasts. In those papers, the minorities are identified using `meta-questions' that ask forecasters about their forecasting abilities or those of others. In the current paper, we explain how contrarian minorities can be identified without the meta-questions by inspecting the skewness of the distribution of the forecasts. Inspired by this observation, we introduce a new forecast aggregation tool called Skew-Adjusted Extremized-Mean and demonstrate its superior predictive power on a large set of geopolitical and general knowledge forecasting data.

  • Philip Tetlock and G. Mitchell, “Stretching the limits of science: Was the implicit-racism debate a ‘bridge too far’ for social psychology?”. In The Cambridge handbook of implicit bias and racism, edited by J. A. Krosnick, T. H. Stark & A. L. Scott, (Cambridge: Cambridge University Press, 2024), pp. 147-165

  • Philip Tetlock, Christopher Karvetski, Ville Satopää, Kevin Chen (2023), Exploring the limits on Meliorism: A commentary on Tetlock et al, Futures and Foresight Science.

  • Pavel Atanasov, J. Witkowski, Barbara Mellers, Philip Tetlock (Under Review), The person-situation debate revisited: Forecasting skill matters more than elicitation method.

  • Philip Tetlock, Lu Yunzi, Barbara Mellers (2022), False Dichotomy Alert: Improving Subjective-Probability Estimates vs. Raising Awareness of Systemic Risk, International Journal of Forecasting.

  • E. Karger, J.T Monrad, Barbara Mellers, Philip Tetlock (Under Review), Reciprocal scoring: A method for forecasting unanswerable questions.

  • Ike Silver, Barbara Mellers, Philip Tetlock (2021), Wise teamwork: Collective confidence calibration predicts the effectiveness of group discussion, Journal of Experimental Social Psychology .

  • Ville Satopää, Marat Salikhov, Philip Tetlock, Barbara Mellers (2021), Decomposing the Effects of Crowd-Wisdom Aggregators: The Bias-Information-Noise (BIN) Model, International Journal of Forecasting.

Teaching

All Courses

  • MGMT9330 - Psych Found MGMT

    This seminar-based course, with active discussion and analysis, is required of all first-year doctoral students in Management and open to other Penn students with instructor permission. The purpose of this course is to examine and understand basics in the theory and empirical research in the field of micro organizational behavior and to build an understanding of people's behavior in organizations and across organizations. The course covers a blend of classic and contemporary literature so that we can appreciate the prevailing theories and findings in various areas of organizational behavior. This course covers micro-organizational behavior, focused on topics such as influence/status, virtual teams, job design, organizational culture and socialization, identity in organizations and overall look on where the field of micro-organizational behavior is going.

  • PPE2355 - Political Psychology

    This course will explore psychological approaches to understanding political beliefs, attitudes, and actions at the levels of both individual citizens and national leaders. It will also explore the possibility that psychological science itself is not immune to the political debates swirling around it. Specific topics will include: the workings of belief systems (and their power to shape what we "see"), cognitive biases (and their power to cause miscalculations), sacred values and their role in stabilizing belief systems and social interaction, personality and ideology (the linkages between the personal and the political), and clashing conceptions of morality and distributive and corrective justice (striking variations among people in what they consider to be fair). We shall also explore some topics that have sparked controversy in the psychological research literature and that tend to polarize opinion along political lines, including work on intelligence and unconscious bias.

  • PSYC2355 - Political Psychology

    This course will explore psychological approaches to understanding political beliefs, attitudes, and actions at the levels of both individual citizens and national leaders. It will also explore the possibility that psychological science itself is not immune to the political debates swirling around it. Specific topics will include: the workings of belief systems (and their power to shape what we "see"), cognitive biases (and their power to cause miscalculations), sacred values and their role in stabilizing belief systems and social interaction, personality and ideology (the linkages between the personal and the political), and clashing conceptions of morality and distributive and corrective justice (striking variations among people in what they consider to be fair). We shall also explore some topics that have sparked controversy in the psychological research literature and that tend to polarize opinion along political lines, including work on intelligence and unconscious bias.

  • PSYC4998 - Mentored Research

    Mentored research involving data collection. Students do independent empirical work under the supervision of a faculty member, leading to a written paper. Normally taken in the junior or senior year.

  • PSYC4999 - Honors Mentored Research

    The Honors Program has been developed to recognize excellence in psychology among Penn undergraduates and to enhance skills related to psychological research. The 4998 credit signifies an Honors Independent Study, completed as part of the Honors Program. The honors program involves: (a) completing a year-long empirical research project in your senior year under the supervision of a faculty member (for a letter grade). This earns 2 cu's. (b) completing a second term of statistics (for a letter grade) before graduation. (c) participating in the year-long Senior Honors seminar (for a letter grade). This seminar is designed especially for Psychology Honors majors; this receives a total of 1 cu. (d) participating in the Undergraduate Psychology Research Fair in the Spring semester, at which honors students present a poster and give a 15-minute talk about their research. (e) a total of 15 cu's in psychology is required. Students will be selected to be part of the Honors Program in the Spring of their junior year (see application process online)

  • PSYC9999 - Independent Study

    Individual Study and Research

Awards and Honors

  • Fellow, American Philosophical Society, 2019
  • Thomas Schelling Award, Kennedy School of Government, Harvard University, 2017
  • Fellow, National Academy of Public Administration, 2016
  • Fellow, American Academy of Political and Social Science, 2016
  • Andrew Carnegie Fellow, 2015
  • Fellow, American Academy of Arts and Sciences, 2009
  • Harold Lasswell Award for Distinguished Scientific Contribution in the Field of Political Psychology, 2008
  • Grawemeyer World Order Prize, 2007
  • Woodrow Wilson Award for best book published on government, politics, or international affairs, 2006
  • Robert E. Lane Award for best book in political psychology, American Political Science Association, 2006
  • National Academy of Sciences Award for Behavioral Research Relevant to the Prevention of War, 1999
  • MacArthur Fellow in International Security and Conflict Resolution, 1999-2001
  • Nevitt Sanford Award for Distinguished Professional Contributions to Political Psychology, International Society of Political Psychology, 1997
  • Philip Converse Book Award for outstanding book in the field published five or more years ago, American Political Science Association (for co-authored book, Reasoning and choice: Explorations in political psychology, 1992
  • Woodrow Wilson Book Award, American Political Science Association (co-recipient with P. Sniderman & R. Brody, for Reasoning and choice: Explorations in political psychology), 1992
  • American Association for the Advancement of Science Prize for Behavioral Science Research, 1988
  • Fellow of Division 8 of the American Psychological Association, 1987
  • Fellow at the Center for Advanced Study in the Behavioral Sciences, 1987
  • Erik H. Erikson Award of the International Society of Political Psychology, 1987
  • MacArthur Fellow in International Security and Conflict Resolution, 1987-1989
  • Distinguished Scientific Award for Early Career Contribution to Social Psychology, American Psychological Association, 1986
  • Canada Council Doctoral Fellowship, 1977-1979
  • Yale University Fellowship, 1976-1977
  • Governor-General’s Gold Medal, Award for Undergraduate Academic Excellence, 1975
  • British Columbia Psychological Association Gold Medal, 1975

In the News

Knowledge at Wharton

Activity

Latest Research

Pavel Atanasov, Jens Witkowski, Barbara Mellers, Philip Tetlock (2024), Crowd prediction systems: Markets, polls, and elite forecasters, International Journal of Forecasting.
All Research

In the News

Can AI Predict the Future?

New research from Wharton's Philip Tetlock finds that combining predictions from large language models can achieve accuracy on par with human forecasters.…Read More

Knowledge at Wharton - 1/14/2025
All News

Wharton Magazine

How to See the Future (And Know It When You See It)

Our best pundits don’t have a solid track record. So how can the rest of us become better forecasters?

Wharton Magazine - 04/20/2016