Peace Tech

As part of this effort to uncover peace speech, the Sustaining Peace Project has engaged subject-matter experts and local journalists in a series of workshops. The description of and findings from the workshops are detailed below.

If Peace Could Speak: A Report on a Series of Subject-Matter-Expert Workshops to Interpret Data-science Findings from News Accounts Originating from High versus Low-Peace Societies

Melissa Wild and Peter T. Coleman
Advanced Consortium of Cooperation, Conflict and Complexity (AC4),
Columbia University
Toyota Research Institute
October 2024

INTRODUCTION & RATIONALE
As a component of the Sustaining Peace Project, the Peace Speech Project, “If Peace Could Talk,” asks if “peace speech” exists, and if so, how it could be identified and supportive of sustaining peace. This stands in contrast to research and interventions targeting “hate speech.” The Peace Speech Project takes a data-driven approach to look beyond individual words or phrases that may express hate and go deeper and broader to understand what may be present – or absent – in language that serves to moderate or mitigate peaceful social processes.
As the Peace Speech Project research has found, through use of data science techniques such as Machine Learning and Natural Language Processing, there does appear to be significant differences in the language employed in some media originating from low- versus high-peace societies. These differences also appear to correspond to differences in social processes that scholars have identified as correlated with lesser and greater levels of peacefulness. To interrogate and better comprehend the findings uncovered by these data-science methods, we facilitated a series of workshop workshops with subject-matter and regional experts. The outcomes of these workshops seem to suggest that our findings through data analysis reflect particular realities on the ground as described by workshop participants. Additionally, the workshops have allowed for exploration and testing of ideas for how the findings could be useful for a variety of different stakeholders - from NGO workers in peacebuilding to journalists and news media consumers. Below, we outline the rationale behind the workshops, the methods used, and key findings from those sessions.
While our data-driven approach, using Machine Learning and Natural Language Processing techniques, did reveal there were significant differences between speech in more and less peaceful countries, the data alone could not help us interpret what we were seeing, nor understand what the classifications were telling us. We also wanted to check for biases in the data. Bringing human interpretation and analysis of the data allowed us to do so.

METHODS
Our team designed and conducted a series of “ground truthing” workshops on Zoom in the spring and summer of 2024, to hear from subject-matter experts and regional expert journalists on what they see in the data – the words generated by our machine learning and natural language processing techniques. Initially, two peacebuilding subject-matter expert workshops were held. The experts represented professional backgrounds in peace studies, social psychology, journalism, poetry, and anthropology. Following these two sessions, three additional workshops with journalists were held, two with journalists from the lower-peace countries we used to train our data set, and one with journalists from higher-peace countries used in the data. The countries selected to be represented were those from which the words generated from our data analysis were obtained, which would be the words presented in the workshops.
While 117 journalists and/or editors were contacted, with approximately 15 media experts/former journalists from 32 countries, 14 journalists in total across three sessions participated and are represented in the below findings from the following countries: India, Bangladesh, Jamaica, Kenya, Nigeria, Sri Lanka, UK, Canada.
The sessions involved introductions, a brief overview of the project, time for participants thoughts and reactions to the data (word clouds), some discussion of potential applications, and requests for permission to follow-up after the session. While each session was structured the same at the outset - with minor differences from subject-matter experts in peacebuilding to regional experts in news media and reporting - the facilitation was modified slightly in each to be responsive to the debriefing sessions the internal team held in follow-up to each. Specifically, this led to an increasingly greater amount of time left for discussion of applications per session.

The goals of the sessions were to:

1. Uncover, from a diverse group of interdisciplinary stakeholders, conceptual or theoretical distinctions related to the linguistic features among words from high-peace and low-peace countries, as categorized using data science techniques.
2. Discuss how to test if the theoretical distinctions identified in the data are indeed related to selected words
3. Discuss what guiding principles or specific ethical considerations must be included in the build-out of a live dashboard to test levels of peace in language

Our main research questions were:
• What are the primary distinctions you see in the word clouds from these different types of societies?
• What differences might these distinctions reflect in the underlying social dynamics of each type of society?
• To journalism or reporting?
• How might an application of these distinctions be useful in journalism/media in your society?
• To promoting higher levels of peacefulness?

MAIN FINDINGS
The workshops helped to confirm that there were significant linguistic distinctions between higher and lower peace societies that resonated with the subject-matter experts and the journalists. Workshop participants from lower-peace societies confirmed that language in their news media commonly emphasized order, stability, and authority, characterized by rigid, formal communication and hierarchical structures. Themes around feelings of threat, danger, and violence were emphasized, often described as shaped by state-sponsored media and the constant threat of arrest for journalists. Consequently, news reporting in these societies tends to emphasize less-peaceful and fear-inducing speech, reflecting the constraints under which journalists operate.

Furthermore, journalists from higher-peace societies confirmed that the more peaceful words reflected what they saw and tended to cover in their media. There was much more emphasis on creativity, openness, and diversity of topics. Here, subject matter experts and journalists mentioned themes around language being more informal, egalitarian, and collaborative, reflecting values centered on individual well-being, everyday life and community. While journalists in these contexts recognized their reporting allowed for a diversity of narratives that could foster a more positive discourse, they did also identify with the journalists from lower peace societies who discussed constraints in reporting. However, for journalists from more peaceful societies, their constraints were expressed more in relation to their needs to engage their audience.

Differences in the language representing higher- vs lower-peace societies resonated with journalist workshop participants, who acknowledged the insights from machine learning analysis as accurately representative of their experiences and as related to underlying social dynamics (be it state-control of media in lower-peace societies or greater individualism and concerns over audience engagement in higher-peace societies). For peace-related subject-matter expert participants, they discussed some of the theoretical distinctions they saw reflected in the words. Namely, Michele Gelfand’s theory on tightness and looseness in cultures, where lower-peace words were reflective of tighter societies (low tolerance of diversity, deviance and strict social norms) and higher-peace words reflective of looser societies (more relaxed social norms and higher tolerance of diversity). A suggestion was also made related to Darwin’s distinction between play-related versus aggression-related terms. This evolutionary perspective was seen in the lower peace words as more aggressive whereas higher peace words were more playful or related to recreational activities (including games and sports).

We then ran transcriptions of the workshop discussions through Perplexity.AI, and prompted it to help identify the different themes generated across the five focus groups. This analysis yielded similar results:

Peaceful Speech Characteristics
Peaceful societies' language tends to focus on:
• Everyday life, lifestyle, and entertainment
• Positive emotions, simplicity, and calmness
• Personal experiences, creativity, and imagination
• Action-oriented words like "time," "work," and "start"
• Education-related terms such as "school," "learn," and "experience"
Key words associated with peaceful societies include:
• "Love," "friendship," "easy," "play," "home," "child," and "good"

Nonpeaceful Speech Characteristics
Less peaceful societies' language emphasizes:
• Conflict, politics, and state/government affairs
• Aggression, authority, and power
• Formal institutions and rigorous structures
• Economics and foreign affairs
Common words in nonpeaceful societies include:
• "Fight," "action," "kill," "crisis," "conflict," "corruption," and "violence"

One valuable insight offered by Perplexity.AI was that words from higher-peace societies seem to point to actions, in particular, actions within people’s control versus words that are more structural and less action-oriented. As one journalist described, peaceful words as seeming to give a sense of “freedom”, things “in our own control.” On the other hand, when journalists would describe the words from lower-peace societies, their descriptions were often in a passive tense. In one instance, a journalist described how these words represent “what is being relayed to us.” Words related to government and state-level structures become distancing elements, where a person not only cannot relate as easily but also cannot control, while simultaneously being controlled by it. One journalist summed up the difference by sharing that peaceful words seem to suggest “getting something done”, and non-peaceful words as “outside home life.”
All workshop participants generally agreed that the distinctions and information from the higher vs lower-peace word clouds could be useful for journalists, readers and more broadly for people working in peacebuilding or for international organizations. Ideas of new ways of using the classifier that could be useful ranged from helping to train journalists, creating new incentives for editors and publishers, helping readers become more self-aware of what media they were consuming and their level of peacefulness, among others. Participants also noted that the linguistic distinctions may reflect the types of publications prevalent in higher- versus lower-peace societies, and it would be important to consider these differences in development of a tool. Additionally, the influence of the data-science research sole focus on English language (to date) raises concerns about a possible elite bias in the data that may overlook the complexities of reporting in various sociocultural contexts.

Overall, there was a strong consensus about the potential the findings offer for establishing a peace speech measurement metric and tool. Such an approach could help editors and journalists promote more optimal ratios of conflict and peace speech, reflect on their language choices, and thus foster more constructive and inclusive forms of communication.

LIMITATIONS and NEXT STEPS
Although the findings and interpretations derived from the 5 workshops we held were highly informative and mostly consistent with the initial conclusions of our research group, they are limited in their generalizability. Although 117 journalists or editors from 32 countries were initially contacted, only 14 journalists and 6 peace and conflict SMEs ultimately participated in the sessions, representing India, Bangladesh, Jamaica, Kenya, Nigeria, Sri Lanka, UK, and Canada. Future research will need to reach out more broadly into all the nations or regions studied, in order to assure we better understand the specific cultural and institutional variation of our findings.

Nevertheless, the insights drawn from the workshops have contributed to the research agenda and to the development of the UX/UI design of an initial pilot tool, MirrorMirror, where newspapers could receive a score of peacefulness in speech in their reporting and readers a score related to how much peaceful or non-peaceful news they consume. The idea would be to help share information with readers and journalists (or editors) what it is they are consuming or producing. Since we have conducted a series of workshops with journalists, with TRI’s support, the next workshops to be conducted are likely to be with consumers, so the MirrorMirror app can be informed by their insights.
Additional methodological considerations were addressed by journalists, and found in running analysis through Perplexity.AI, which included the importance of considering cultural context and language differences when interpreting linguistic patterns. A suggestion was brought up of analyzing speech at different societal levels (everyday speech vs. leaders' speeches, or regional versus state-level) as a way of ensuring linguistic analysis considers cultural context.

Finally, upon review of workshops, the team noticed participants mentioned that specific words – depending on by whom or where they were used – could be seen as inciting hate or being exclusionary. They also shared how stories in reporting could be written in different ways to be more or less positive but that reporting on negative incidents would still be important. This has since led to a revisiting of word and sentence classifications done through past data analysis (using generative AI and machine learning techniques), paying particular attention to both how something was written as well as what topics were used. Moving forward, the team will continue to pay close attention to tone versus word and try to understand what information is being used to create the classifications. This too will help inform the classifier that will power the MirrorMirror application.