Finding patterns in the noise

Using novel computational approaches, graduate student Shiyao Liu develops better tools for analyzing data

September 28, 2020 Leda Zimmerman MIT Political Science

When social scientists administer surveys and questionnaires, they cannot always count on the scrupulous cooperation of their respondents: It’s human nature to get distracted when faced with a form. So how can researchers sort through what may be unreliable data to identify statistically significant answers to their questions? That’s where Shiyao Liu comes in.

“I have designed a tool for reducing measurement errors when respondents don’t pay serious attention to online questions,” says Liu, a sixth-year PhD candidate in political science. Through statistical methods she has devised, Liu can detect and eliminate random-seeming answers that make for a noisy data set. “With cleaner data, it’s easier to discover patterns and generate meaningful results.”

Liu’s work on this computational tool earned the Best Graduate Student Poster Award from the Society of Political Methodology in 2019. It is one thrust of her dissertation research, which focuses on optimizing social science survey methods and data analysis.

Fast start

After arriving at MIT in 2015 from Peking University with undergraduate degrees both in statistics and in philosophy, politics and economics, Liu found herself in demand. During her first summer in Cambridge, she was engaged by Lily Tsai, faculty director of the MIT Governance Lab, as both a field researcher and methodologist.

“She was writing about public support for authoritarian regimes, and investigating the role played by retributive justice,” says Liu. “The idea is that when authoritarian leaders punish their lower-level officials for wrongdoing, public support rises for the leaders because people perceive that the regime is pursuing justice.”

Liu visited China twice, and with MIT colleagues and local Chinese collaborators, helped conduct 1,600 face-to-face interviews.

Some political science theories suggest that punishing corruption improves the image of authoritarian leaders because it makes them appear more competent. But with computational help from Liu, the research team learned that authoritarian image-building wasn’t just about making the trains run on time.

“By leveraging new surveying methods, I was able to show that increased support for top leadership also flowed from people’s belief that leaders were behaving in a moral way—that they knew the difference between right and wrong,” she says.

Liu, who co-authored a forthcoming paper with Tsai on the popularity of anticorruption punishment, believes this research provides a useful prism for examining governments today.

“When the Soviet Union collapsed in 1989, people thought democracy would be the only surviving political system in the world, but it wasn’t the case,” she says. “Regimes emerged that frequently used anticorruption campaigns as a way of building up their popularity among the people.” Think Duterte in the Philippines, and Bolsonaro in Brazil, says Liu. People are eager to prop up even the most non-democratic dictators, if they perceive them as pursuing justice.

Statistics for social sciences

Liu was born and grew up in Shanghai, the child of a businessman and a warehouse worker. She quickly discovered an affinity for history and political science, with a special fascination for the workings of different political systems and ideologies.

After Liu’s stellar results in China’s high-pressure college entrance exam (she scored at the very top of Shanghai’s cohort of 30,000 high schoolers), she received a full scholarship to Peking University. There she plunged into graduate-level courses on the social sciences and statistics. One of her undergraduate theses looked at whether mass shootings in different congressional districts in the U.S. had the effect of reducing votes for the representatives from those districts. She collected and analyzed ten years of historical data, geocoding the sites of mass shooting events.

By the end of college, says Liu, “I realized I wanted a career where I could basically use statistics to solve problems about society, in either politics or economics.” After serving in business consulting and research positions, she felt certain that his future lay in academics, and that a graduate degree in political science, at MIT in particular, was the right path.

Liu’s decision to study abroad coincided with a more open time in China. “My generation saw the country playing more according to international rules,” she says. “Demand was building from the government and Chinese business to give Chinese students international experience so we could help them communicate better with the rest of the world, to minimize confusion and misunderstanding.” This, says Liu, helped a large cohort of Chinese students to transform themselves.

Cleaning up messy data

At MIT, Liu was drawn to the Political Methodology Lab and the work of its director, associate professor Teppei Yamamoto. Under Yamamoto’s mentorship, Liu began to zero in on ways she might advance quantitative methods for political science research.

Liu turned toward the problem of clearing up statistical noise in social science studies: “I was doing online surveys myself, and realized that some people were spending 30 seconds, and others, 30 minutes,” he recalls. Liu came to adopt terms for these distinct types of respondents: “fast forwarders” are those who click through with little thought about the content, and the “wanderers” are those who take a longer time to answer questions because they are simultaneously looking at their mail or meandering through YouTube. “Both the fast forwarders and wanderers don’t pay attention, and their answers lead to great randomness in the data set.”

Liu embarked upon developing a statistical program that can identify both types of respondents in a data set. “It’s impossible to say precisely that one person is a wanderer and another is a fast-forwarder,” says Liu. “But my program uses probability to show that particular respondents are likely to be one of these types, and to kick them out of the dataset.”

Liu believes this tool could prove useful to many social scientists whose research deploys online surveys. “We could save researchers who are struggling to find a pattern in the noise of their data,” she says.

Forward through the pandemic

After departing Cambridge in March due to the Covid-19 pandemic, Liu has been preparing drafts of her working papers for publication, while residing in her family home in Shanghai. Her physical absence from MIT has posed a range of challenges: “I miss face-to-face check-ins with my advisors and colleagues, which inspired me, gave me new ideas about research, and created a sense of community,” she says. She also pines for the regular dinners, karaoke parties, and hikes with friends. The tense state of affairs between the United States and China, restricting travel, is “making Chinese students' lives difficult, including mine,” says Liu.

But there are consolations to being home, including ample support from friends and family, along with home-cooked meals. “The pandemic is an obstacle for everyone, and we’re all trying to overcome it in our own ways,” she says. “For me, it means pushing forward on my dissertation, and exploring faculty and post-doc positions all around the world.”