Texto en español: PISA para "países en desarrollo" (PISA-D)
Some of the concerns raised around OECD's PISA tests (reading, mathematics and science, applied every three years to 15 year olds) refer to their inadequacy for "developing countries".
Participating Latin American countries systematically occupy some of the lowest places in PISA rankings, far from "developed" OECD countries (in the past few years, Mexico, Chile and Colombia have been accepted as OECD countries; they are also at the bottom).
PISA tests were developed by and for OECD countries. Later, non-OECD countries have joined PISA, 10 of them from Latin America and the Caribbean: Argentina, Brazil, Colombia, Costa Rica, Dominican Republic, Chile, Mexico, Panama, Peru, Trinidad & Tobago, and Uruguay. (See: list of countries that have participated in PISA)
Issues related to the inadequacy or usefulness of PISA for non-OECD countries include the following:
» the need to contextualize the tests, responding to the great heterogeneity (socio-economic, cultural, etc.) of countries, within the same generic category of "developing";
» many 15 year olds are out of school because they never enrolled or because they dropped out (drop out rates are high in may countries, especially in secondary education);
» there is no technical capacity in the majority of countries to administer a complex and massive standardized test such as PISA;
» many countries participate in international tests (such as UNESCO's LLECE tests in the case of Latin America);
» devastating domestic effects in countries getting low results and rankings in PISA tests;
» the enormous attention dedicated by governments to improve scores and rankings in the next PISA test, distracting time and resources from critical structural issues and from learning as such.
Some of these concerns have been aired in open letters addressed to OECD, such as the one sent in 2013 by Ministers of Education in Latin America (Los Ministros de Educación del MERCOSUR y la prueba PISA) or the one sent in 2014 by 92 academics from the US and other OECD countries (Stop PISA!). Concerns have also been raised in Chile (Bárbara Figueroa critica la Prueba PISA porque mide asuntos 'ajenos a la realidad educativa chilena') and in Ecuador vis a vis this country's decision to join PISA (PISA ¿para qué? ¿El Ecuador en PISA?). (See: Critical Voices of PISA in Latin America).
Responding to these and other concerns, the OECD proposed PISA for Development (PISA-D), an initiative addressed to middle and low income countries. The idea is to expand the participation of non-OECD countries in PISA. Nine countries - from Africa, Asia, and Latin America and the Caribbean - expressed their interest to participate in the pilot project: Bhutan, Cambodia, Ecuador, Guatemala, Honduras, Panama, Paraguay, Senegal, and Zambia. (See: Technical meeting, June 2013); OECD Call for Tender 100000990, August 2014).
"PISA for Development aims to increase the policy relevance of PISA for developing countries through enhanced PISA survey instruments that are more relevant for the contexts found in developing countries but which produce scores that are on the same scales as the main PISA assessment. The project will also develop an approach and methodology for including out of school children in the surveys. The project’s objectives will be achieved over a 36 month period through a three-way partnership involving the OECD, concerned development partners (DAC members plus the World Bank, UNESCO and other UN bodies and other regional organisations) and partner countries from the developing world".
- In: OECD Post-2015, Element 2, Paper 1: The OECD's contribution on education to the post-2015 framework: PISA for Development.
The OECD sees the following advantages of PISA for Development:
PISA-D is an OECD strategy for the post-2015 period. 2015 marked the deadline for two major world initiatives: Education for All (EFA, 1990-2000-2015) and the Millennium Development Goals (MDG, 2000-2015). In 2015, the Sustainable Development Goals (SDG) and the 2030 Agenda were approved. • "A single reference against which to rigorously gauge the degree of progress
made towards targets for educational quality and equity.
• A comparable and robust measure of progress to allow all countries – regardless of their starting point – to establish themselves on an improvement trajectory to achieve targets referenced to common international goals.
• Credible and comparable results: PISA requires participating countries to follow common technical, institutional and administrative standards for the assessment.
• An opportunity to help build institutional capacity. Countries are responsible for overseeing PISA implementation; therefore, participation in PISA can also drive improvements in institutions. This capacity building could be implemented directly with development partners in a way that creates spill-over benefits to other parts of the educational sector."
International awareness around a "global learning crisis" - millions of children worldwide not learning to read, write and calculate after attending school for four years or more - acknowledges for the first time the precariousness of learning in primary schools in most "developing countries" and is contributing to finally place learning as an explicit and fundamental goal. Unfortunately, that goes also with an emphasis on learning assessment starting now in pre-primary education. In that context, OECD/PISA appear as key global partners.
"Developing a universal measure of educational success" is one of eleven areas in which OECD plans to contribute to the 2030 Agenda (Beyond the MDGs: Towards an OECD contribution to the post-2015 agenda). In other words: the aim is to establish ONE definition of 'educational success' and ONE way to measure it worldwide, in the North and in the South. PISA-D is the strategy and the instrument to incorporate that "other part of the world" still absent from the global education evaluation race.
A big player and evaluation enthusiast such as the World Bank blesses global learning benchmarks and recommends them especially for "developing countries."
"In a global economy the primary benchmark for success is no longer improvement by national standards, but the best-performing education systems internationally. (Having said that, it’s also important for countries to set and measure learning goals that reflect their own national priorities and values.) This usually means participating in one of the many international assessment programs that test the math, science, problem solving or other competencies of students at the same grade or age level in different education systems around the world. Countries – particularly developing and emerging economies – may feel at a disadvantage in this global benchmarking, but should keep in mind that steady improvement over time is the important thing." (Education: Measuring for Success in Today’s World, 9 May 2014).
Issues that may need more focused attention and discussion
The following issues were mentioned in "The PISA for Development initiative moves forward: Have my wishes been fulfilled?" (2 Feb. 2015):
- A test with questions that 15-year-olds in emerging and developing economies can actually answer.
The OECD’s original plan was to draw solely on their existing pool of 337 PISA questions to create the PISA for Development test. One and a half year later, the OECD started to explore using questions from other regional and international assessments to supplement the PISA questions. The idea is to make sure 15-year-olds in emerging and developing countries can actually answer.
- A test that emerging and developing economies can afford.
The PISA for Development pilot is about twice as expensive as the regular PISA exercise, since it involves lots of developmental work, in an effort to adapt the questionnaires to the contexts of these countries and to develop a methodology to include out-of-school students. Donors such as the World Bank have provided financial support to facilitate countries' participation. It is essential, however, to take a hard look at the long-term sustainability of PISA for Development, if countries are expected to cover the costs on their own.
- A test that contributes to learning for all.
The pilot faced the challenge of collecting learning data on the entire 15-year-old cohort in a country, including those who are out of school. This may include youth who never went to school or who are semi-literate.
PISA-D is administered both in and outside of school. Eight
countries – Bhutan, Cambodia, Ecuador, Guatemala, Honduras, Paraguay, Senegal
and Zambia – participated in the
school-based implementation of PISA-D, which was carried out from 2015 to 2018.
Six countries administered the out-of-school assessment:
Guatemala, Honduras, Panama, Paraguay, Senegal and Zambia.
PISA-D countries can compare their results to the more than 80 countries participating in PISA.
PISA-D results
PISA-D countries can compare their results to the more than 80 countries participating in PISA.
PISA-D results
Ecuador
Guatemala
Honduras
Paraguay
Senegal
Zambia
oe.cd/il/PISA4Dev
Related texts in this blog
Sobre evaluación en educación | On Evaluation in Education
Artículos sobre PISA | Articles on PISA
25 Years of Education for All | 25 años de Educación para Todos
