Matheny on RAND’s Legacy and Future
Discover more from ChinaTalk
Matheny on RAND’s Legacy and Future
“We live in an extraordinary period. If we can navigate these risks and put up guardrails, the upside potential is phenomenal.”
Jason Matheny recently passed his first anniversary as CEO of the RAND Corporation, the legendary federally funded research organization founded after World War II that now has nearly 2,000 employees and over $350 million in annual revenue.
Previously, Matheny led the Biden White House’s policymaking on technology and national security at the National Security Council and Office of Science and Technology Policy. He also founded the Center for Security and Emerging Technology at Georgetown University and directed the Intelligence Advanced Research Projects Activity, which develops advanced technologies for the US Intelligence Community.
He is also great on the mic. We spent two hours together putting together one of my favorite conversations this year.
In the excerpt we’re running today, we discuss:
How RAND balances researcher freedom, long-term problem-solving, perennial budgetary constraints, and political imperatives.
Why Matheny looks for, of all things, kindness in hiring and promoting top-notch talent.
How RAND’s structure allows researchers to self-organize.
RAND’s sense of “optimistic urgency” and counterbalancing the weight of existential risk with existential hope for the future.
This interview is brought to you by the Andrew Marshall Foundation and the Hudson Institute’s Center for Defense Concepts and Technology. They’ve been generous enough to sponsor a series on the impact of technological change on bureaucracies and operational concepts.
Do check out the full podcast, I promise it will be worth your time.
The Flaring of Intellectual Outliers
Jordan Schneider: You pointed me to a 2014 paper published by the Andrew W. Marshall Foundation titled, “The Flaring of Intellectual Outliers: An Organizational Interpretation of the Generation of Novelty in the RAND Corporation.” This paper attempts to frame how RAND and other organizations like Bell Labs were able to do the seemingly impossible in generating novel research.
Albert Wohlstetter, one of RAND’s famous early social scientists, said he was attracted to the organization because of the enormous latitude it gave researchers, bridging practical and theoretical projects.
He was shocked to see RAND publishing on super random stuff like geometry, which was far from what the Air Force probably thought they were signing up to get when they just wanted better bombing targets and more efficient planes.
Today, RAND is mostly known for its practical policy research. What does research as a source of practical ideas mean to you?
Jason Matheny: There was a sense that to attract really smart researchers, RAND had to give them enough latitude so that they could do basic research on geometry or on causal inference. But you actually wanted them to work on practical problems that were going to be relevant to US policymakers.
In some ways, the freedom was a way to get people in the door.
A lot of the problems that had practical utility were also intellectually interesting and fascinating. Finding these attractants for really smart people is important. Self-determination is certainly one of those — the entrepreneurship that RAND has, where most of the researchers are picking their projects.
They’re self-organizing. They’re not given much top-down direction. That is an important part of the recipe for a place like RAND.
We’re pretty clear with folks when they’re applying to RAND that they’ll be spending their time working on policy analysis. They won’t be doing a ton of basic research, but you can do some. For people who come from academia and want to be working on hard problems that are actually going to matter, it’s great.
They didn’t just want to do publications for peer-reviewed journals that were making some methodological innovation that wouldn’t actually impact an important decision. They didn’t want to go through the whole tenure process in order to achieve institutional goals for a university.
What they really wanted to do was affect some consequential policy in national security or education or healthcare. There’s a selection effect.
The folks who want to join RAND might have come from an academic research background, but they’ve been dissatisfied with it because either it didn’t have enough practical relevance, or the mix of practical relevance versus research ambition, or because they just got frustrated with the incentive systems within academia that pulled them away from things with some policy impact.
One of the things that makes RAND unusual is our size. We have 2,000 people and we work across so many different disciplines.
Say a labor economist comes in because they want to work on labor displacement in the United States. But then they see a colleague who’s working on some really interesting analysis of labor and dependency ratios in China. Then they say, “Well, I’m interested in that project.” We do find a lot of people who come in the door because of one topic, but then quickly start working on a range of other topics.
It’s a great environment for informational omnivores, for people who are just broadly intellectually curious, even outside of the domain they specialized in, maybe in their earlier research career.
Jordan Schneider: Even in 1960 you had folks saying that going from 200 to 1,000 made multidisciplinary collaboration a lot harder. It became large enough that scholars from the same disciplines could get lunch together and not branch out. How do you keep multidisciplinarity alive even as the silos expand?
Jason Matheny: To what extent does size present opportunities versus costs? Early RAND wasn’t that small. We had about 500 employees with hundreds of projects at any given time. Bell Labs had about 15,000 employees in the mid-1960s.
You can have highly innovative organizations that are large and that draw on the infrastructure of large institutions.
For example, CSET is relatively small, but it’s within the much larger institution of Georgetown, which has thousands of people. It’s able to draw on that infrastructure.
You often see highly innovative groups within larger organizations. It makes sense to think about what makes those groups particularly good at doing research or analysis. There’s also a lot of bang for the buck in thinking about the mechanisms or processes that allow groups of different sizes to be effective in producing research and analysis.
One factor is having a sense of optimistic urgency — actually having a sense that the problems that you’re working on should be really important and can be really important, and you’re probably actually not duplicating other work.
We tend to overestimate the number of people who are working on the most challenging and important problems.
There was a guy at Bell Labs, Richard Hamming, who had a lecture on this topic that he would give. He was known for inviting himself to people’s lunch tables at Bell Labs and sitting down and asking people, “Are you all working on the most important problem? And if not, why?”
A lot of folks, just by inertia, aren’t necessarily working on the thing that they think is most important — just freeing up people’s sense that they could be working on the most important problem.
Living with Leviathan
Jason Matheny: We’re celebrating our 75th anniversary this year. One of the things I’ve been looking at are the periods in RAND’s history when we’ve had political backlash or policymakers who are unhappy with us.
I actually don’t think it’s been worse today in terms of the feedback or pressure we get politically than it has been historically. Part of the reason for that is the people who are generating most of the questions for RAND.
The folks within government are mostly civil servants. They’re folks who are not highly politicized. They’re folks who are not turning over from administration to administration. They’re people who have been spending 20 years working on a set of really hard problems related to defense policy or national intelligence or education policy or the other things in our portfolio.
Every once in a while, we’ll get some angry letters from folks about some analysis that we do, but we keep on working.
We have maintained our reputation and our credibility on both parts of the political spectrum pretty well over the last 75 years. We’re just generally viewed as being a set of nerds who are just really committed to getting the facts right.
Sometimes the facts are inconvenient. Sometimes folks don’t like the results. But that is not politically motivated.
One reason we’ve had this project on “Truth Decay” at RAND is because we realize this is not only an important issue for American society, it’s also an important issue for analytic institutions like RAND, whose entire business is organized around trying to figure out what the truth is.
Jordan Schneider: RAND initially had something of a blank check from the Air Force, and Open Philanthropy gave you a lot of latitude in the early years of CSET. But that’s not necessarily the same organizational structure that you’re living with today. What are your current constraints and how are you working around them?
Jason Matheny: As a percent of our total budget, it’s true that we’re more constrained than, say, early RAND or CSET. But in absolute terms, there’s as much or maybe even more unconstrained funding here right now. That’s all thanks to foundation funding or especially ambitious analytic efforts that we’re working on for places like the Office of Net Assessment. The work that comes out of that is some of the most interesting and effective work at RAND.
Mike Mazarr’s work on the social foundations of national competitiveness is as interesting as the work that Tom Schelling and others did at RAND.
I do think that the level of general government oversight over contracts has increased a lot since the 1950s. Much of that for the better, I should say. As a taxpayer, we probably did need more discipline on how we were spending federal dollars immediately after World War II, but that does have a cost for analytic organizations.
Now, there’re still a lot of open-ended projects at RAND, and we also have a lot of open-ended positions at RAND. We just recently put out a call for applications for a Technology and Security Policy Fellowship, which is open, and hope listeners who are interested in that topic will apply.
About 80 percent of the time is open to the fellows to decide how to spend it. It’s pretty great. There’re very few things like this in the research world. Then 20 percent of the time is spent working on policy analysis for actual policymakers.
You get the best of both worlds. You get the benefit of having some self-directed study, and the benefit of working on real projects that allow you to interact with an assistant secretary of defense or assistant secretary of state or other senior policymakers.
Jordan Schneider: This balance between independence and influence is another thing that Andy Marshall harped on as one of the keys to RAND’s success. How do you think about the trade-offs there?
Jason Matheny:
In some ways, RAND is probably more influential to more policymakers today than it was then.
RAND is known for pushing back on the questions we’re asked and saying, “You’re asking us the wrong question. The more important question is X, not Y.” We still do that.
When a federal decision maker comes to us with a question, and if we think it’s not framed well or there’s a bigger question that deserves to be asked, we’ll provide that feedback.
Now, ultimately, it’s the sponsor that’s paying us, and they own the question. We own the results. We don’t take direction on what the product of the analysis should be. That’s something on which we maintain a really fierce independence.
Maybe a third of the time we’re actually able to persuade the sponsor there’s a better framing for the question, that it can actually help them make a decision in a way that’s going to be more relevant.
There’s an Eisenhower line about this. Whenever you face a big problem, try to make it bigger.
If you can solve the more general class of problems, it not only can be more efficient analytically, it can also just prevent certain kinds of mental shortcuts. Whenever we can, we try to make the more general case on the topic of the analysis rather than the more tactical case.
The Best Time Was Yesterday
Jordan Schneider: There’s this great line in “The Flaring of the Intellectual Outliers” talking about the early years of RAND:
In some mysterious way, an urgent pressure for relevance became an urgent pressure for fundamental research and ideas that were relevant in the long term. It is as though a string quartet stranded in a winter snowstorm decided urgently to compose a new fugue rather than start shoveling.
How do you make sure that long-term focus is still part of RAND’s culture?
Jason Matheny: Andy had a very particular view. Not everybody at RAND felt that way about RAND.
Read Andrew May’s history of the early years of RAND. His dissertation is maybe the most complete history [of that period]. Or read, as I’ve been doing, memos from across the staff at RAND in the late 1940s and 1950s and even in the early 1960s. There were a lot of folks who felt like they were just turning the crank on analysis that wouldn’t be that relevant.
Certainly, the work done by Andy Marshall and other memorable figures at RAND had a more general approach to strategic analysis.
The sense of urgency was one that some RAND staff felt acutely. It was driven in part by the risks of the day that were collectively felt across the country.
There was a real risk of nuclear war. There were RAND staff who chose not to have a pension because they didn’t believe they would actually survive until retirement. They thought it was more likely that they would die in a nuclear blast.
It’s certainly hard to recreate that sense of urgency that was maybe felt broadly in the 1950s and 1960s.
But there are certainly parts of RAND that still feel that urgency. For instance, a lot of the folks here working on AI policy or biosecurity policy feel that sense of urgency. Folks who work on Taiwan analysis feel that sense of urgency.
It’s not to say that other folks are feeling sanguine about the portfolios that they’re working on. Education or healthcare policy are urgent in a different way. We’re not all going to die if we don’t get an education policy that’s more effective, but it’s a lower and slower existential risk if we don’t figure out those problems that are consequential for US policy and global policy.
There’s still a very high degree of mission focus and motivation at RAND. This is also a selection effect at RAND.
People who come here want to be working on problems that are consequential. Maybe they want to make a methodological contribution along the way. But the main reason that they’re here is because they want to solve a problem that’s important for humanity.
War Games, What Are They Good For?
Jason Matheny: We need to figure out how to design analytic tools and processes that are well-fit for the kinds of challenges we face in the 21st century. We had a set of tools and processes that we developed in the early years of RAND and continued to refine. Many of those are still just as relevant today.
I asked Andy Marshall once, “What are the most important methodological innovations that RAND made? Was it the advances in deterrence theory, game theory, rational choice, or some modeling and simulation work?”
He said, no, it wasn’t any of that.
The most important methodological contributions from RAND were in war gaming. He said the war games were the things that actually got decision makers to realize that they had made assumptions that weren’t realistic.
They allowed decision makers to run through a variety of scenarios, test different strategies, see where they broke down. That was a really interesting insight.
It’s not the fancy analytic work. The war gaming work does not tend to involve something that you could publish as a new research methodology paper. It’s more about getting real decision makers to a table and having them work through a bunch of different scenarios. A lot of that work ends up being highly classified because the details of the war games actually matter.
We probably conduct more classified war games than any other think tank or institution. The output of that is something that continues to be among the most valuable things that RAND does.
That is an example of a 20th century innovation that is just as relevant today.
There are also other things that we can do now that we couldn’t do in the 1950s or 1960s, like massively crowdsourced analysis. We just didn’t have the tools for that. We didn’t have the Internet. We didn’t have a way of distributing analytic work or questions to widely distributed expertise.
We also need to be thinking a lot about how to fully leverage advances in AI for analysis.
Personnel Is Policy
Jordan Schneider: Building CSET from scratch and leading a seventy-five-year-old research bureaucracy seem very different. What have you learned from your past year as CEO of RAND?
Jason Matheny: I’ve worked within organizations that are brand new, like CSET, organizations that are new but already have a ton of processes that you have to work within, like IARPA, and organizations that are pretty old and have even more processes, like the National Security Council.
RAND is in that final category. We have 75 years of history. It’s a mid-sized organization. You have to be very careful about making structural changes that are going to affect 2,000 people’s lives. You’re going to take more effort to change or undo change if you find out it was a mistake.
Something common across those organizations is that hiring and promotion are incredibly important. Among the most important decisions we can make are just about personnel. There’s the line “personnel is policy.” Personnel is at the heart of any organization’s likelihood of success. Most of the variance and outcome of a project is related to the people who are working on it.
Spending a lot of time thinking about the people who are on a project — how to get the right team working on a project — is essential. For RAND, CSET, and other places I’ve worked, it’s a combination of finding people who are not just brilliant and hardworking, but also kind.
Kindness is often underrated. Sometimes managers think, “Well, so-and-so is a jerk, but they’re so brilliant it’s worth hiring them or keeping them on.” In my experience, that very rarely works out.
The work that I’ve done has been predominantly team-based. No one wants to be on a team with a jerk. If you hire jerks, you lose the other talent on your team.
It makes sense to have organizations with a high tolerance for eccentricity and quirkiness. It’s one of the things that I love about the places I’ve worked — IARPA, CSET, and RAND. RAND has a ton of quirkiness that I adore, but there is no unkindness. I don’t think there’s room for that.
Interpersonal compassion is highly correlated with impersonal, intergenerational compassion — the sort that we’re going to need a lot of if we’re going to safely navigate the challenges ahead.
So far, I found that the changes we’ve made within RAND to make processes more efficient — to figure out how to take on these big challenges — they’re not changes that have required me to give orders from the mountaintop on stone tablets.
It’s actually something the researchers here have been really enthusiastic about — taking on these big problems and reducing the amount of process and bureaucracy. It’s going pretty well. I’m really optimistic about the things that we’re doing here.
Jordan Schneider: How do you test for kindness in an interview?
Jason Matheny: We tend to do a fair amount of interviewing at RAND. Our researchers come in and have several different interviews in different settings with different groups of researchers. One question we ask is to describe a situation where somebody you were working with was really struggling with a personal issue. How did you help them with it?
We also have questions about interpersonal compassion in some of the interview panels, particularly for some roles that focused on long-term issues. How do you think about compassion towards future generations that are going to be affected by a policy decision?
I don’t think we have any magic solution to figuring out how to make sure that folks that we hire are kind and compassionate. There’s some amount of self-selection.
They’re given a lot of clarity that the work they’re going to be doing is team-based. They’re going to be interacting a lot with other people.
The Self-Organization of Inquiry
Jason Matheny: Somebody who’s your boss on a project one day might be working for you the next day because the teams themselves shift. You’ve got principal investigators on one project who are team members on other projects.
It’s relatively flat. The researchers mix up the hierarchy constantly depending on the project. That reinforces the need for compassion. You don’t know who you’re going to be working for or whether you’re going to be the boss on the next project.
Jordan Schneider: RAND’s early hiring practices wouldn’t necessarily fly today. They seemed to rely on “old boy” networks and hiring people they thought were smart but weird. Somehow this led to thirty-two Nobel laureates passing through RAND over the years. Maybe this was due to intellectual competition. Is there room for that kind of competition at RAND today? Can it be done in a healthy way?
Jason Matheny: To what extent does competition within an organization motivate good work? Does it lead to a lot of wasted effort in jockeying for position? There is a balance between competition and collaboration that RAND tries to achieve.
One interesting feature of RAND is our internal labor market where people self-organize to pursue projects. It’s competitive, but people are competing to be on a team.
The overall emphasis on team-based, interdisciplinary work means that the people who succeed here are pretty pro-social. They’re people who are capable of collaboration. We don’t have many projects that are solo efforts.
We do need to figure out ways of aligning incentives with analytic work, including outside of RAND. To what extent can we set bounties or prizes for certain kinds of analysis? Can you award money for analysis that makes a decision better?
I would love to see more prediction markets — providing incentives for creating greater accuracy in judgments. We don’t have a policy that allows large, well-funded prediction markets to be used by society. That’s probably holding back. A lot of opportunities for creating these incentives for accuracy would be really useful for policy analysis.
Jordan Schneider: The early days of RAND had an aggressively hierarchical structure with giants like John von Neumann at the top. But the average age in the 1950s was 27. I don’t think that’s what RAND looks like today. What’s the optimal mix of junior and senior staff?
Jason Matheny:
For a lot of research areas, you want a mix of senior, mid-level, and junior researchers working on projects.
Senior researchers cost more. On a given project, it’s usually less of their time that’s spent on it compared to more junior researchers. Often, they’re playing the role of mentor, helping to point out some similarity to another project that we’ve done at RAND.
Also, because we have a graduate school at RAND, there’s a pedagogical component to this that’s just embedded in projects. Many of our researchers also teach at the graduate school.
They just feel a certain responsibility for making sure the next generation of policy analysts are equipped — that they’ve got the analytic tools and the dispassionate approach needed to pursue the truth relentlessly.
That is reflected in the way our projects are run. We aspire to have a pretty uniform distribution in seniority. The very early years of RAND certainly had that. It skewed younger, but then after about 15 years it was starting to become more uniform. We now have a bit of a bimodal age distribution.
There wasn’t much hiring during the pandemic. We’ve grown a lot in the last ten or so years. Right now, a lot of our hiring is at the junior and mid-level. Other things like the introduction of fellowships to RAND will help hiring in some areas like AI and synthetic biology.
Wonks of Change
Jordan Schneider: What kinds of change are you hoping to drive within RAND in the years to come?
Jason Matheny: RAND has an interesting matrix structure. We have research departments where the researchers are hired and mentored, and then research divisions where the projects occur. This allows the researchers to mix and match along projects in different divisions.
RAND also manages four federally funded research and development centers (FFRDCs), which are like think tanks for parts of the government.
We have FFRDCs for the Secretary of Defense, the Secretary of the Air Force, the Secretary of the Army, and the Secretary of Homeland Security. Those operate a little bit differently in that there’s just a steady stream of funding.
There’s a set of strategic priorities that RAND works on with the respective secretaries and their deputies and staff. Those are groups of a few hundred people at any given time working on different projects.
We need to be thinking about the things that truly cut across our divisions. Our China work is in that category. So is our work in technology, climate and energy, and some of our work on strengthening democracies and resilience against truth decay, disinformation, political polarization. Our work on inequality and inequity cuts across our domestic and global work.
We have seven offices at RAND. Three of them are overseas in the UK, Australia, and Brussels, supporting our work with allies. We need to figure out ways of leveraging our reach across geography, disciplines, classified and unclassified work, and our own graduate school talent pipeline.
I’m mainly interested in how we can find enough diversification of operating models within RAND that we can run lots of micro-experiments, and we can have people self-organize based on project type, so that the organizational design matches what the project is attempting to do. There’s a ton of room for experimentation.
For any listeners or readers who are really interested in organizational experiments, RAND is a great place to conduct those. Historically, we’ve had all sorts of different designs. Today, we have at least a dozen different kinds of business models and operating models that are running in parallel.
It’s a great place to run experiments like this. I’m a really big fan of Heidi Williams, an economist who thinks a lot about innovation and organization. There’s so much to be done in this general category of organizational and mechanism design on how we can produce better research and analysis.
“Optimistic Urgency”
Jordan Schneider: In what ways might you have surprised RAND’s board and its employees?
Jason Matheny: Folks pretty much knew what they were getting. I was somebody who came in knowing a little about a few areas of policy and not being an expert in a lot of the areas where RAND works. We have so many areas. I won’t ever hope to be able to be an expert in most of the things we work on.
But they also knew that I was going to be fully dedicated to making RAND the best workplace that I can make it and to make the work here easier.
That means reducing process spending and trying to make sure that we can get research results faster and supporting staff so their lives are happier — so they can accomplish more with their time and take on incredibly consequential things for the future of the country and the world.
Having a sense of “optimistic urgency,” as Andy Marshall put it, is a key part of that.
We don’t need to feel that the world is about to end to have that sense of urgency. We can also just feel that the opportunities are so great that there’s not only existential risk, there’s also existential hope.
We live in an extraordinary period where so much technological change is happening that can have incredible applications to improving human health and human prosperity.
If we can manage to navigate these risks and put in the guardrails, the upside potential ahead of us is phenomenal.
