Patricia Lucas | 23 May 2024

We've blogged about the potential for AI to help gather qualitative data. But what about the vast amount of data already collected? We know there's tons of it. Fantastically rich information from field notes, rapid assessment exercises, feedback sessions and past studies. But where is the time to sift through all this?

We can now use our AI-facilitated tool to analyse existing qualitative data, massively reducing the time needed for coding and managing the data. Once our coding framework is planned, production takes only seconds with insights and quotes available instantly in our interactive dashboard (more news on that soon in an upcoming blog).

We used a set of interviews and fieldnotes made available on the excellent UK Data Service. These interviews ask about a microcredit intervention, providing loans for toilet construction in rural Maharashtra (Attanasio, 2022). Sanitation, Microcredit and Awareness - A Qualitative Analysis, 2019-2021. [data collection]. UK Data Service. SN: 855387, DOI: 10.5255/UKDA-SN-855387).

We are often asked how we can demonstrate the quality of our AI-supported analysis. It was therefore reassuring to see that our approach highlighted many areas which aligned with the findings of the original team led by the French Institute of Pondichéry.

Both analyses showed a shift in social norms. Many people interviewed wanted and expected toilets in their home compound, feeling shame and embarrassment when they didn't have one:

"she used to [go] out in [the] field. That made her feel uncomfortable and shamed"

Most of those interviewed had access to toilets, financed through a mix of savings, government loans, microcredit agency, and money lenders. But many of the toilets were non-functional, in poor repair, or unpleasant to use:

"The toilet is dirty, not clean, it smells bad"

What came through strongly in both analyses was the particular importance of women in decision making. The original team noted that toilets were a way to secure the safety and dignity of young women. They also highlighted that toilets expressed wealth and status, and were therefore built at times such as marriages and celebrations.

Our AI-supported analysis also identified that the demand for toilets was often driven by women and that concerns about women's safety and dignity were often at the forefront of decisions to construct toilets. As a loan officer observed:

"Ladies are the ones who demand toilets first!"

We viewed the link between marriages and toilet construction as an example of emerging social norms about gender and sanitation. While the need to build a toilet for marriageable sons could be viewed as primarily a display of wealth, we noted that participants also mention the needs of future daughters-in-law. In addition, we found that the birth of a daughter was also a catalyst for toilet construction. These findings suggests that these toilets were not being constructed as a demonstration of wealth, but in the response to the particular needs of young women:

"Now before marriages, toilets are built so that the daughter-in-law will come and be peaceful."

The beliefs about women's safety may reinforce gender stereotypes in unhelpful ways. However, in the context of shifting norms around sanitation, it may also be that there are significant status gains from providing for the dignity of young women.

An analysis of quantitative data from the same trial showed that toilet building was more likely in households where women highly valued toilets and in households where women were involved in decision making. The qualitative work helps explain these findings. Both our analysis and the original analysis find similar explanations; that the need for toilets is driven by women. But by utilising AI to automate some elements, we were able to reach our results quickly. By efficiently looking at all interviews in a different way we were also able to identify examples that suggested a more subtle relationship between gender, social norms, status, and demand for sanitation. Ladies do demand toilets, but it is perhaps more accurate to say that families demand toilets for their young women.

You can read our findings in full here.

Using our AI-assisted analysis to this existing dataset revealed its vast potential. If you have an untapped treasure trove of data, contact us at hello@colectiv.tech to unlock it. We are grateful to the researchers who collected, transcribed and archived these interviews for future use, and to the participants who shared their voices and experiences.