In May 2024, the Centro para Puerto Rico (CPPR) of the Sila María Calderón Foundation requested collaboration with PR PASS Workshop to learn about the impact that the entrepreneurial training efforts it has led over the past 15 years have had.
Background
In our review of studies conducted on the CPPR’s Entrepreneurial Capacity Building initiatives, we found information on the population it serves and insights that directed at quality assurance and program development.. From the data collected from participants, it emerges that
- 71% of them, when starting the program, reports incomes below the poverty level.
- 44% have some form of employment.
- 65% report receiving public assistance.
- 72% have households with 3 or more members.
- According to a survey conducted between November of 2019 and March of 2020, which included the participation of 761 graduates from the different programs, it is documented that 42% of the graduates have a business in operation.
What is an impact study?
The concept is often confused with an accountability report that includes quantitative data such as number of people served or businesses created. However, an impact study goes beyond talking about results, it seeks to better understand how the programs and services have contributed to changes in the lives, work, and communities of its participants. When reviewing the survey questions, participants will notice that questions focus on whether or not there was change, while other questions ask about the causes or if the change can be attributed to some degree to the program. To contribute to understanding the relationships between the changes experienced and the program, the survey also invites participants to share more details and stories that explain the changes observed. The present impact study will not only rely on the impact survey, but rather the survey itself and the interpretation of the data are part of a larger participatory methodology in which participants in the program are at the center of creating knowledge about how the program and its services contributed to changes in their lives.
Who defined the changes and metrics used?
For programs developed in complex contexts, where it is recognized that not all variables that influence generating change are known, a participatory approach that includes participants in defining change and how to measure it is recommended (Hertz et al 2021, Douthwaite & Hoffecker 2017). To learn about the changes experienced by participants, PR PASS Workshop held 4 workshops (2 in the metro area and 2 in Mayagüez) to identify them. In these workshops, participants shared their most significant change stories. Using these stories as a reference point, they proceeded to identify the changes mentioned in them, and 4 interviews were conducted with the work team and members of the Board of Directors to identify additional topics of interest. Based on this information, and with the help of participants, possible metrics were identified that would help address the change and the role of Entrepreneurial Training in it.
Where are we in the development of the impact study?
After programming the impact survey and conducting tests that validated its clarity and relevance, during 2025 the Sila María Calderón Foundation, through the Centro para Puerto Rico and the Centro Empresarial para la Mujer, will begin disseminating the invitation to participate in the study among its thousands of graduates. PR PASS Workshop will be storing the data as an external entity responsible for protecting the privacy and confidentiality of the study’s participants. Upon completion of data collection, PR PASS Workshop will proceed with a preliminary analysis before moving on to a participatory analysis process known as "data parties," which will be held with participants and members of the work team in late 2025. If everything progresses as we hope, the final report will be ready by the first quarter of 2026.
Comparable impact studies and possible contribution to the field
In a review of 159 studies conducted in 2017, found that most focus on short-term results, which often include subjective indicators of entrepreneurial aptitude and entrepreneurial intentions (Nabi et al 2017). A more recent review of 80 studies grouped the indicators used into 4 domains: subjective, objective, institutional, and national (Mensah-Williams and Derera 2023).
- “Subjective” indicators include measuring changes in an individual’s self-management, entrepreneurial aptitude, and entrepreneurial competencies. This description coincides with the set of metrics described by the study led by Nabi, as they are also indicators measured through a student’s perception and demonstration of knowledge.
- By “objective” indicators, Mensah-Williams and Derera include measurements of the business sector. Examples of these are the number of businesses created, the number of business practices used, the number of sales, or changes in business activity. Although it was not contemplated in the studies, the authors suggest that entrepreneurship within an existing job could be an objective indicator not yet used. In the study led by Nabi, these metrics were scarce and were considered medium- and long-term indicators.
- Institutional indicators refer to knowledge products generated by universities and agencies that document the development of knowledge put into action.
- The review of impact evaluations by Mensah-Williams and Derera does not include examples of studies that link the impact of entrepreneurial training programs to national indicators of economic development and entrepreneurial culture. This field is recognized in theory, but no studies were found that explored this relationship. The Global Entrepreneurship Monitor (GEM) studies provide reports with systematized and uniform indicators that shed light on policy, activity, and entrepreneurial culture development. One limitation of these reports is that they do not include qualitative data collection, nor do they address cultural changes or particular transformations within a society.
According to the list above, we find that existing data on the CPPR entrepreneurship capacity building programs focused on objective indicators, but lacked short-term or subjective indicators, as well as narratives that help to better understand the processes and factors that contribute to the change observed. The impact study being developed for the Sila María Calderón Foundation stands to provide a case study that will present an analysis of short- and medium-term change, comprised of both objective and subjective indicators. The participatory methodology that helped identify and interpret the reach of the changes studied will be novel contribution to the field that is expected to provide rich examples of the relationships between entrepreneurial capacity building and economic and social development. Finally, the fact that Puerto Rico has experienced a cascading variety of disasters—given hurricanes, earthquakes, COVID-19, and political and economic instability—is expected to shed light on the how external factors affect the anticipated outcomes and the the sustainability of the identified changes.