This article examines how effective education institutions improve educational achievement, regardless of students’ socioeconomic backgrounds. The paper utilizes a multilevel model using data from the Third Regional Comparative and Explanatory Study (TERCE) for Ecuadorian students in reading and mathematics. Findings show that between 31 and 52% of the variation in test results is due to differences between schools, thus underlining the importance of the compensatory effect of schools in promoting equality of opportunity.

It is widely accepted that education functions as a strategic mechanism for narrowing income gaps and ensuring equal opportunities (Aguado et al., 2009; Walker and Unterhalter 2007). The relationship between years of study and income, put forward in human capital theories, is not fully supported empirically (Fix, 2018). Furthermore, it has been suggested that the term human capital should be replaced by skilled labor (Milanovic, 2015), and that other factors contributing to improved academic performance- such as health or nutrition (Hanushek, 2015)- should be considered. However, academics agree that education represents a core competency for achieving socio-economic well-being, by promoting a person’s capabilities and freedom to achieve (Sen, 1979).

Mere access to education does not guarantee potential personal and social benefits, as the quality of education available is also key in ensuring that people are able to attain higher levels in diverse skills (Hanushek and Woessmann, 2007). It is important to understand that higher levels of performance do not correspond solely to a person’s individual traits but are connected to their social context; a person’s abilities are interwoven with their environment (Fix, 2021).

Given this, the quality of the educational processes available is fundamental, and equal opportunities for accessing education are not sufficient. Consequently, it is important to gain an understanding of the external and internal inequalities present within the system, as these then lead to widely variable educational ecosystems that combine peoples’ characteristics and personal situations with their social environment.

External inequalities can be ascribed to the student’s characteristics and conditions such as sex, age, family income, cultural capital, support, and family expectations. These are considered external because the education system has limited influence on them, and few opportunities exist to modify them in the short term without an intersectoral implementation of other social and economic policies (State of the Nation Program, 2015). Studies show that individual learning outcomes are closely related to a student’s socioeconomic level, ranking above other factors such as gender, immigrations status or previous number of years spent in education (Cervini, 2006; Rumberger and Palardy, 2005; Van der Silk et al., 2006).

In contrast, internal inequalities pertain to factors that are directly linked to the education system, the school, and the management of available resources such as teachers, infrastructure, funding, and educational materials (State of the Nation Program, 2015). These factors can be modified by guidelines, policies and management styles within educational centers and are key when comparing opportunities and in contributing to the improvement of students’ performances (Ehrenberg and Brewer, 1994, 1995; Hanushek, 1986, 2002). Research in the field of education, therefore, increasingly focuses on the school and what happens within its walls.

If the benefits of education are to translate into social improvement, the offering must endeavor to be equitable and high in quality. Thus, as well as evaluating people’s incomes, attention must be paid to factors such as school infrastructure, evaluation methods, supplies and educational materials, student achievements and indicators of students remaining in education (Guzmán, 2014; Martinez Rizo, 2002).

One proxy variable used for measuring quality in learning outcomes, is the score obtained on international standardized tests. The level of knowledge acquired, reflected in the scores obtained, depends on a variety of factors that form the education process and in which schools play a fundamental role.

Related studies recognize that even though students’ test results are largely determined by their motivation and effort, as put forward by Arias et al., (2016), when looking at Costa Rica, the educational establishment’s condition, amenities and the availability of human and financial resources are also vitally important factors in the development of teaching and learning outcomes (State of the Nation Program 2017). For example, an Australian study showed that outcomes in math increased in a quiet and orderly classroom environment, where students are keen to learn (Thomson et al., 2004). The data for the Program for International Student Assessment (PISA) for Costa Rica shows that there is no single factor that explains why some schools or countries obtain better results than others. High rates of success can be attributed to a correlation of factors that include school resources and policies and practices implemented in both the school and classroom (Montero et al., 2014).

In the same vein, this article focuses on students’ academic results as a proxy of quality and the school as a determining factor in achieving these. The school effect can be understood as the percentage variation in student grades and can be explained by differences that exist between schools (Rodríguez and Murillo, 2011). Two types of effects can be found in studies in this field, gross and net effects. The first takes no variable control into account, neither for the student nor the school. The second takes these variables into account and includes the school’s properties as well as the student and family’s sociodemographic profile in their estimates. Borman and Dowling (2010) found a 5% net school effect, when looking at the United States.

Findings from European studies show that the net school effect oscillates between 10 and 15% (Brandsma and Knuver, 1989; Fitz-Gibbon, 1991; Murillo, 2005; Raudenbush and Willms, 1995; Smith and Tomlinson, 1990).

When looking at Latin America, evidence shows a higher school effect than for other regions. Using data obtained from UNESCO’s 2006 Second Comparative and Explanatory Regional Study (SERCE), Cervini (2012) shows that the gross school effect for Latin American countries is in fact between 30 and 40%, depending on the grade analyzed and the subject evaluated.

Evidence such as this fuels the discussion on effective schools and allows it to remain a key factor for actions undertaken to foment the region’s education system.

Furthermore, evidence shows that there are normally two categories of countries within Latin America, according to the school effect identified: those that have a low net value and, therefore, have more standard or equitable models of education systems, and others where the net effect is relatively higher (Woitschach et al., 2017). Therefore, the education center is important and students’ results in the education system are not equitable. Once the impact of students’ socioeconomic background is added, greater implications of inequity in education systems could be found (Cervini, 2011). According to evidence found in regional studies such as Woitschach et al. (2017), Ecuador could be in the second category of countries, which makes the analysis of this particular case even more relevant.

In the presence of ample empirical evidence on the school effect, this study has two main objectives. The first is to quantify the school effect for Ecuador and determine how it fluctuates as control variables capturing characteristics of students, families and schools are included. The second, is to analyze the variation between gross and net school effects when a variable quantifying the family’s socioeconomic status is included. The objective is to show that the school’s role is central to students’ educational achievements, even if socioeconomic factors are included in the analysis. The article is organized in the following manner. After the introduction, the methodology used to measure the school effect is presented, using a multilevel model, and the main results outlined. Finally, the conclusions and implications for educational policy, derived from this study, are presented.

The Coleman report (1966) was groundbreaking in establishing the relationship between results obtained by students and the school that they attended. Coleman notes that schools influence grades, but that the effect is limited. The study sparked a discussion about the education process as an input-output structure and showed that this type of analysis is relevant when creating public policy for education. Furthermore, it brought about a series of studies that replicated that logic. Those with an education economics approach proposed converting the input-output focus into an educational production function (Hanushek, 1979). The proposal was not mere semantics, because from that perspective, the educational production function reflected how the use of these inputs could increase the result of the output. Thus, the development came from the need to find a variable to describe the degree of knowledge a group of people had, beyond the number of years they attended school. In other words, it was born of the need to explain how quality in the education system can be improved and how the improvement is determined by a series of individual and social interactions, in which the students’ learning process takes place.

Later, Mayeske et al. (1972) showed that even if the effect that the school could have on students’ outcomes was small, the influence existed. Therefore, an analysis of determinants that involved aspects of the school, beyond physical resources was necessary.

Thus, using microeconomic theory, Hanushek (1979) proposes a first approach to the measurable or observable factors that influence students’ cognitive and non-cognitive achievements (A_it) and organizes them according to four determinants: i) the student’s background (B_it) ii) influence of classmates (P_it) iii) the school’s resources (S_it) and iv) the innate capabilities of the student (I_i).

(1)

Measuring the education process’s output is complex. According to Mancebón (1999), understanding the education process requires a proper definition of the inputs to be considered. Despite its limitations, the conceptual proposal from the 1980s is still current and is used in many studies around the world (Cervini, 2012; Rodriguez and Murillo, 2011; Román and Murillo, 2011; Woitschach et al., 2017).

This study is based on the theoretical approach of the education production function, described previously (see equation 1) and uses data from the 2013 (UNESCO, 2013) Third Regional Comparative and Explanatory Study (TERCE). TERCE also provides information on learner achievement in Latin American and Caribbean countries and identifies the associated factors. The study collected data on three levels; students, classroom, and school for students in third and sixth grade^TN1, focusing on four subjects; mathematics, reading, science and writing.

For the present study, we worked specifically with students from the third and sixth grades, only evaluating math and reading and excluding science and writing. Science was excluded due to a lack of data for students in third grade, which limited our ability to compare between grades. Writing was omitted due to the nature of the subject and the type of learning evaluated, as it presented difficulties in the scale and form of the evaluation which would not allow for a strict comparison with results from the other two areas analyzed (UNESCO, 2016). These grades and subjects were chosen to be able to compare results with studies undertaken in other countries such as Argentina (Cervini, 2010), Colombia (Rodríguez and Murillo, 2011), Costa Rica, Cuba, Chile, El Salvador, Guatemala, Nicaragua, and others (Murillo and Román, 2011).

The proposal put forward here is based on surveys by Cervini (2010), Rodríguez and Murillo (2011) and Woitschach et al. (2017). The sample used is representative, as it was comprised of 194 schools (public and private) and 4,730 students across Ecuador. Sample weights are considered in the methodology to compensate for unequal selection probabilities of the observations (Yansaneh, 2003).

These types of surveys tend to provide information that is ‘nested’ or ‘hierarchical’. This means that whilst differences in student learning achievements in the survey are expected, one can find similarities in the case of students that attend the same school (Murillo, 2008). Considering the data structure, a multilevel model was used as a method to estimate the school effect. The model is widely used in health and education fields (Bryk and Raudenbush, 1987; Cebolla, 2013; Draper, 1995: Goldstein, 1995; Hox, 2002).

The specification of the model estimated, detailed in equation (2), provides the analytical framework needed to show that the result of learning is the consequence of a combined interaction of elements that are organized in the Education Production Function’s conceptual proposal. The proposal considers student, family, and school properties to explain the grade obtained by the students on the standardized TERCE tests. Annex 1 provides a glossary of the variables included in the model.

(2)

Where:

y_ij:: corresponds to the score obtained by the student i from the school j in each of the TERCE tests,

: is the interception that reflects the global score obtained in the test,

: student and environment characteristics vector that includes variables for gender, child labor, repeating a grade, early education, and the Socioeconomic Index (SEI),

: school characteristics vector, considering the variables of whether the school is public or private; area (urban or rural); possession of reading book, math book; education center infrastructure; classroom environment; presence of violence in the school and teaching practices (this last was only available for sixth grade),

: error term that represents the variance y showing how far the school j average is from the global school average,

: which corrects the effect in each group that the variables have on y_ij and the term e_ij is the variance of the error term and captures the existing differences between schools, showing how far the student’s grade is from the school’s average.

The school effect is no more than the percentage of the total variance of students’ grades explained by the differences between schools. Therefore, the calculation formula is given by equation (3). This last equation is known in related studies as the variable percentage or the intraclass correlation coefficient and represents the “school effect” (Woitschach et. Al., 2017).

(3)

To quantify the effect on students’ performances and identify how its explanatory weight varies using school and family characteristics as variables, six models were created for each of the subjects and the levels studied (see table 1). The first specification is known as a null model (equation 4) as it does not include any type of explanatory variable. The objective of the calculation is to determine the scale of the gross school effect, that is to say, how much the school impacts student performance without controlling for student or school characteristics (Rodríguez y Murillo, 2011). The second specification (equation 5) starts from the null model and includes the SEI. The specification aims to show variations in the school effect once characteristics relating to the students’ families and social background are included. The third specification (equation 6) is proposed with the objective of controlling for student characteristics. The fourth specification also includes a vector of the school’s characteristics. It is important to note that the equation includes both individual and group characteristics (the students’ and the school’s respectively). It was therefore necessary to introduce random slopes in the effects of the independent variables for individuals. Given the importance of socioeconomic status as a predictor for student performance, the variable is included with a random slope (Cebolla, 2013). The final calculation gives us the net school effect.

In the fifth specification, (equations 8a and 8b, for third and sixth graders respectively), the main objective is to determine which other factors can explain the school effect, calculated using the previous specifications. According to prior studies, these variables can include classroom environment, teachers’ practices, and violence in the school, among others. This specification also includes the SEI random slope.

The sixth specification (equations 9a and 9b, for third and sixth graders respectively) aims to only measure the school effect in relation to academic and non-academic characteristics of the educational center. In these equations¹1, socioeconomic factors relating to the student as well as the average of said variable for the school, are excluded. It is important to note that given that the final specification consists of independent variables reflecting group and not individual characteristics, it does not include the SEI random slope, included in the two previous specifications.

The aim of calculating the six previous specifications is to identify if the school effect is really an associated factor to learning gains and if it can be reduced or removed once factors pertaining to schools and students are considered. To that end, statistical software Stata 16 was used, and 24 models were calculated (6 specifications for each course and subject evaluated) (see tables 2-5). The main result shows that, independent of the student or school profile and context, the school effect persists, although it is reduced when controlled by the aforementioned factors.

The results of the null model show that there is a significant gross school effect that oscillates between 31 and 52%, depending on the grade and subject. The results are consistent with the available empirical evidence in which similar results are obtained in other Latin American countries (Woitschach et al., 2017). Additionally, these are in accordance with results published by Scheerens (2000) who establishes that the gross school effect in Latin American countries tends to be higher than that in developed countries, which have a school effect of about 10 to15%. However, the effect tends to decrease as student and school control variables are incorporated. Calculations of the following specifications (2 to 5) show this, even though the decrease is not homogenous in all the cases analyzed. The results are summarized in figure 1, where a greater decrease in reading versus math can be found, in both the grades evaluated. Differences depending on the grade analyzed are also present, for example in math the gross effect is 42.26% in third grade and 28.43% in sixth grade. Once characteristics for students (model 3) and school (models 4 and 5) are included, the effect drops to 37.42 and 15.28 % respectively (specification 5). When looking at reading, the gross effect is 31.14% in third grade and 51.89% in sixth grade, and the effect changes to 11.51 and 13.92% respectively once characteristics specified in models 4 and 5 are considered.

These results show that the school effect persists even when other characteristics pertaining to the student, their family and the school are included. Thus, factors pertaining to the school and students allow for a greater decrease in the school effect for reading. Finally, we see that in non-observable characteristics²2, that the effect picks up, and is higher in math especially in third grade.

Studies undertaken in other countries show that the gross school effect in Argentina is approximately 42% in math and 36.5% in reading. If adjustment variables such as student, family and school characteristics are included, the value decreases to 18.4 and 16.7% respectively (Cervini, 2010). When looking at Colombia, Rodríguez and Murillo (2011) found that the gross effect was 27% for math and 40% for reading. However, the net effect decreased to 15 and 6 % respectively.

In Spain, Murillo (2015) also found that the school effect varied between subjects. The author notes that the school effect was 9.26% for math and 3.7% for reading. In Scotland, Raudenbush and Willms (1995) reached the same results, estimating a school effect of 15.5% for math and 13.6% for reading. Some authors explain these differences from a pedagogical perspective, suggesting that students have a negative predisposition for subjects such as math and science. This would mean that the roles played by both the teacher and classmates could impact performance (Cervini, 2010, 2012; Juidías and Rodríguez, 2007; Rodríguez, and Murillo, 2011; Woitschach et al., 2017). Valverde and Näslund-Hadley (2011) also note that educators’ teaching practices are fundamental to learning achievement, when it comes to this type of subject. They emphasize that classroom practices should incentivize the development of critical thinking, so that students are more engaged and thus learn more easily. It is important to note that learning to read is linked to learning the mother tongue, and that the family can provide important support, whilst support from specialized staff is fundamental for mathematics.

One of the advantages of analyzing the net school effect, gradually controlling student and school characteristics, is being able to identify the minor impact that each of these has on learning achievement. Thus, the specifications used allow for the identification of factors associated to the learning gains of each case. With regards to student characteristics, the scale and importance of SEI is noteworthy on all the cases analyzed. This variable has a positive connection with learning achievements, consistent with available empirical evidence. Attending preschool also showed a positive and significant connection to learning achievement in all cases, although the scale varies depending on the grade and subjects evaluated. Finally, children having to work or repeat a school grade were both observed to have a negative impact on learning achievements, across all analyzed cases.

In the case of third grade students, factors relating to schools that played a statistically important role in learning achievement, both for math and reading, were indicators of violence in the school setting, the classroom environment, infrastructure, and whether the student had a book for the subject being evaluated. Factors such as the school’s location and whether it was public or private did not have a significant impact. The results for sixth grade vary only with regards to the importance of the index of violence in the school surroundings.

These findings are backed up by empirical evidence. Boissiere (2004) suggests that the positive connection between having a subject book and learning achievements can be explained; the books support the teacher’s classroom work and make it easier for the student to learn certain subjects.

A suitable classroom environment increases the scores obtained by students, by reducing factors not related to the learning process. The importance of handling bullying correctly is highlighted. The presence of bullying prevents students concentrating and creates a violent environment that limits academic learning and relationships between peers (Andreou and Metallidou, 2004; Román and Murillo, 2011).

In Costa Rica, one of the countries that ranks highest in the region according to TERCE tests, results for factors associated to student performance show that the teacher’s role and the resources available for studying (notebook, computer, and classroom climate) are fundamental to better outcomes. Furthermore, increased availability of financial resources and better infrastructure contribute to higher student scores (State of the Nation Program, 2015).

These results confirm the central role that schools play in student performance. They highlight action points for education centers, which are strategic actors in providing equitable learning opportunities, even when faced with students’ diverse socioeconomic backgrounds. Analyzing the relationship between learning achievements and socioeconomic status is important, as it is one of the most used context variables in educational research (Sirin, 2005). A family’s socioeconomic status not only has a simultaneous impact but is also a precedent for students’ cognitive abilities. In other words, the impact of a family’s socioeconomic status on a student’s development from early childhood through childhood, can limit brain development and learning achievements (Institute of Medicine National Research Council, 2000; Jednoróg et al., 2012).

The results of this study show that on including families’ SEI in the analysis of the gross school effect (the difference between specifications (1) and (2)), the decrease in the school effect is significant, particularly in reading. Figure 1 shows that in the case of third grade reading, the school effect decreases from 31.14 to 24.65%. Thus, socioeconomic status reduces the school effect by approximately 6.5 percentage points. When looking at the same subject for the sixth grade, the school effect is reduced by 9.5 points. But, when looking at third and sixth grade math the effect decreases by 1.2 and 4.8 percentage points, respectively. The results show that, school-wide, the family has greater influence on reading that it does on math. These results are consistent with those found in other studies (Cervini, 2011; Cervini et al.,2016; Murillo and Román, 2011; Woitschach et al., 2017).

The next step is to compare the decrease in the school effect when socioeconomic status is used as the only control variable (model 2) and when the school’s academic and non-academic characteristics are considered, without adding variables relating to the individual (model 6). In figure 2, the difference between these models can be observed with respect to the null model. Thus, gap 1 is defined as the difference in school effect observed between models 1 and 2. Gap 2 is defined as the difference in the school effect found between models 1 and 6. In all cases, it can be observed that gap 1 is smaller than gap 2. Thus, the decrease in the school effect is greater when only the school characteristics are controlled, in comparison to only controlling the grade studied with the SEI.

This shows that even though the SEI is a relevant predictor of performance, school quality is also important. However, the scale of this effect can vary depending on the levels of analyses added to the study. Educational literature suggests including a third level³3 (classroom), with the aim of establishing the classroom effect, which allows for capturing the influence of both peers and teachers on student academic performance (Cervini, 2006). The inclusion of this level is relevant, as evidence exists that omitting an intermediate level, in this case the classroom level, can result in the higher level (school level) being overestimated (Opdenakker and Van Damme, 2000). This would imply that the school effect obtained includes the incidence of classroom characteristics in student performance, which is why it is necessary to include some of these properties in order to achieve greater precision.

This last result is relevant as it reflects the importance of school policies focusing on improving school amenities and their impact on equity in the education system. According to Cervini (2011), the association between the average student attainment of a school and the socioeconomic status of its students can be an indicator of inequity within the system. Thus, the smaller the association between these variables, the greater the ability of the education system to improve quality of life for a student. The school, therefore, functions as a means to compensate for inequalities.

A better education contributes to reducing inequities in the education system. Alongside other social and economic factors (Fix, 2018), this can lead to a greater uptake in opportunities to increase income, reduce the possibility of falling into poverty and increase life span through improved health practices and less physically strenuous work (Agudo et al., 2009; Huesca, 2009; Lámelas and Aguayo, 2009).

The school’s margin for action (teachers and classroom) continues to be strategic. In spite of a school having an unfavorable socioeconomic context, student performance can be similar to schools with more favorable socioeconomic circumstances, if and when the principal has good leadership skills, there is a positive school environment, teachers and principals have high expectations for their students’ academic future and when students are constantly evaluated to monitor learning gains (Ellis, 1075; Phi Delta Kappa, 1981; Webber, 1971).

This study has two objectives; the first to quantify the school effect in Ecuador and determine how it changes as control variables capturing characteristics specific to students, their families, and schools are added. The second, is to show whether the school - even with the associated impact of socioeconomic factors- continues to play a central role in student academic achievement.

Both objectives were applied to Ecuadorian students of third and sixth grades, in math and reading, using TERCE data. An Education Production Function was used as an analytic framework, as well as a multilevel model using two levels (student and school), as a strategy model. There were six specifications including student, family, and school environment variables, implemented successively to identify the change in scale of the school effect. The main results show that, despite being smaller than the gross effect found, the net effect is relevant when student and school characteristics are included. Thus, even if there are non-observed characteristics in the model, Ecuadorian education centers play an important role in student performance.

The SEI is an important predictor for student grades in Ecuador. Its significance is highlighted as the scale of the school effect decreases significantly when this variable is included in the null model. In addition, the effect of this index on student grades is higher for reading than math. Therefore, the family environment is relevant to subjects related to literature, while schools appear to have greater impact regarding math.

Even though the SEI is an important predictor for learning gains, once the grade obtained was controlled by the vector for school characteristics, the school effect decreased. This confirms the school’s importance in student attainment for third and sixth grade students in math and reading, in Ecuador.

The findings, organized by the subject and grade evaluated, provide the space to broaden the study of the complex interaction that associated factors have on the learning process of subjects requiring different skills, as well as of students in different stages of their development.

This represents a challenge for developing public policy so that investment made translates into an increase in quality. Furthermore, it is important to strengthen and create continuity in national standardized testing systems, as they provide important information for reproducing, discussing, and contrasting the results of this study, focusing on different school subjects and grades studied. Simultaneously, the results highlight opportunities for improving amenities and processes of schools working to improve quality in the Ecuadorian education sector, or to evaluate the effectiveness of investment programs that have already been implemented.

Finally, when looking at the connection between the SEI and student results, these have been conditioned by a variable that is exogenous to the education system. This presents a challenge to schools in terms of their role and relevance in achieving equal opportunities. If socioeconomic conditions determine students’ success to a large extent, improving the quality of this system would create access to more and better jobs, in the future. Therefore, the current investment in Ecuadorian schools, aimed at increasing their quality, in terms of infrastructure, study programs, staff training, strengthening teacher and principal leadership roles, designing a feedback loop between students and teachers (and vis versa), access to educational materials (such as books and notebooks) and the implementation of strategies to improve the use of class time could translate into greater success for future generations in the education system.