Method

Sampling

To capture the way schools and school context can influence progress and outcomes, it is important to have a large enough sample of schools to obtain reliable estimates of the impact of school context and quality of learning.

The desired sample in each city is 4000 students in the modal grade level for 15 year-olds, across at least 30 schools. Actual sample sizes may vary across cities, as smaller cities may not have 4,000 students in the entire 10th Grade cohort. Some cities will take a census approach, while others will survey students and teachers in a sample of schools.  Larger cities may prefer to have more schools and students to derive estimates based on different districts, regions or types of schools.

The basic sample design is a two-stage stratified cluster sample design. The first stage consists of the sample of schools, which may be stratified; the second stage consists of a sample of students from the target cohort selected in sampled schools. It is important that the sample of schools in each city reflects the range of school provision and the social geography of the school system. Large enough samples of students within individual schools are also needed to identify and compare program participation and attainment, and to allow for attrition over the life of the project.

Tracking students over time

Students will be contacted for a first time in 10th Grade and once yearly over the next four years.

The first part of the study takes a snapshot of students’ family background, school experiences, and hopes and plans for the future. Information will be gathered on the quality of students’ instructional experience, their academic self-esteem, the learning climate in their classes, and the quality of relationships with teachers and other students, as well as an assessment of non-cognitive skills. Recent evidence suggests that such skills can be important in promoting individual success and can be influenced by the work of teachers and schools.

While a major role of the survey is to assess the level of student engagement in school as a learning environment, an equally important aim is to assess how well young people are engaged as active participants in the various facets of community life, and the impact of schooling on this engagement. The survey therefore also explores social integration— students’ attitudes and perceptions relating to life outside of school, how they see their social world, and their confidence in the workings of the economy and the political system.

One of the aims of the project is to compare the “starting positions” and the “trajectories” of young people of similar measured ability. Controlling for achievement enables us to investigate the impact of different institutional arrangements on student outcomes. It also helps identify “similar” students in terms of achievement for international comparisons. This requires tests designed to transcend the particular structures of national systems. For this reason, student achievement will be measured using a test adapted from the OECD Programme for International Student Assessment (PISA).

The aim of the longitudinal study is to follow the directions the participants take throughout their subsequent schooling and on to further study and employment. Repeat contacts will be made with participants over the next four years. At each contact, information will be gathered on the place within the school curriculum or in vocational training or employment that young people occupy. They will be asked about turning points and decisions, about changes in circumstances, and about how well they are coping with the demands made upon them as learners, apprentices and workers.

Questions to be considered at this stage of the research include:

  • To what extent, over the four follow-up years, do their views change—views about economic chances, study and training opportunities, social and community involvement?
  • How well are they advanced in terms of economic integration—the nature of the jobs they have, their attachment to their jobs, including working on their own account or in partnership with others?
  • Student achievement and skill levels will also be assessed at some point in the follow-up period to measure growth. Which students and school systems better prepare students for further study and careers beyond secondary school?

The teacher perspective

The study also generates an international perspective of schools and education systems from the standpoint of teachers. The teacher survey canvasses views about:

  • school context, including intake and policies
  • student characteristics
  • pedagogical and behavioural challenges
  • teaching practice and professional aspirations.

It is designed for a sample of teachers in each school, representative of the teachers in touch with the relevant group of students. This information will help the researchers see each school in the context of other schools serving the city, and from a classroom rather than school administrator’s perspective.

The school perspective

In the first year of the study, school principals (or their nominees) will complete a survey designed to build up a picture of their school. The survey gathers information about:

  • the structure and organisation of the school (e.g. grade levels in the school, main sources of funding)
  • the student body (e.g. family background)
  • the teachers (e.g. qualifications) and the school’s resources (e.g. access to qualified teachers and instructional resources)
  • curriculum and assessment practices (e.g. specific measures which schools take to assist weaker learners or to challenge and extend stronger ones)
  • the school climate, decision making practices and school policies (e.g. for admitting students)
  • school initiatives and strategies such as those designed to raise student achievement, or address the needs of students at risk of failing or dropping out of school.

The project will use this information to consider the impact of school policies and processes on student outcomes and pathways.

Other data to be collected

The project will examine other available data to explore the links between the social, economic and cultural context of schools and school communities, and student pathways and outcomes. Most countries administer a census which can provide important social, demographic, economic and cultural data on local areas and neighbourhoods, making it possible to study different school neighbourhoods in participating cities, particularly their social, economic, organisational, political, and cultural structures. Other government and non-government survey data may also be used, such as community and labour force surveys as well as the school census.

In some cities, there is also the opportunity to link the student data with administrative data. For example, in some cities, the researchers aim to connect the student data records to administrative data in order to obtain educational histories of young people to better understand origins and destinations. Where feasible, this will also promote richer comparisons across cities.