An Insight into Elementary School Pupils’ Achievements in the Content Area Data

Ability to work with data, as a component of mathematical literacy, is one of the necessary segments of the general literacy of citizens of modern society. Within the mathematics curriculum it was introduced much later than the traditionally studied content areas. Since its introduction on national level in 1997 until 2012, the only assessment of pupils’ achievements in mathematics at the end of elementary education was done in 2001. Worldwide, reference data are part of the international study TIMSS. Th e research reported in this paper is focused on evaluation of pupils’ achievements in the content area data at the end of elementary school. An empirical research was conducted using a test consisting of selected released items from TIMSS 2003 and the National Assessment of pupils’ achievements in mathematics in 2001. Th e test was administered to a sample of 404 grade 5 pupils in the school year 2011/2012 (25 classes from 9 schools). Th e sample is not representative of the entire population hence no general conclusions may be derived. Th e results reveal diff erences in the achievements in the content area data obtained by pupils in schools from diff erent areas of urbanity. Th e lowest results were achieved by pupils in a suburban school with a signifi cant percent of Roma population. Th e practice of pupils reading assignments incompletely appears repeatedly as a hindering factor. Th e authors recommend that pupils are trained for independent work, from reading the assignments and their requirements to performing each step separately, which also preserves the cognitive level of tasks. Th is goal needs to be emphasized within the national documents that determine the quality of education.


Introduction
Professionals in any fi eld of work make daily decisions based on the information provided by diff erent sources, and oft en collected by the users themselves.Correctness of those decisions primarily depends on the ability for properly collecting, selecting, processing and interpreting data.Children, as well, meet daily with a multitude of data from the world that surrounds them, which raises the necessity for introducing data contents into the elementary mathematics curriculum along with ways for enabling children to collect data, to represent data using tables and graphs, to read, to interpret and to use data correctly and purposefully.
Th e content area data was introduced in the Macedonian national mathematics curriculum for elementary school children with the changes introduced in 1997.It was introduced with 15 class hours per year in each of the fi rst four grades of elementary school.In 2007, the number of hours assigned to the content area data has been reduced to 8 and 9 class hours per year in grade 2 and 3, respectively, and the number of hours for the content area data has not been specifi ed in grades IV and V. Th e curriculum goals encompassed enabling pupils to gather, classify and compare simple data as well as read and represent data in tables, bar charts and line graphs.
Th e scarcity of published evidence on the development of young learners' competence working with data 15 years aft er the introduction of systematic instruction in the content area necessitated this study.Th e main objective of the study was to examine pupils' achievement in the content area data at the end of elementary school, i.e. the end of grade 5.
Th e only research on mathematics achievements of pupils at the end of elementary school in the Republic of Macedonia was done in 2001 by the Department of Assessment at the Bureau for Development of Education (Mickovska, Naceva, Aleksova, 2002).Th e results showed that approximately three quarters of the pupils know how to read data from a table or a bar chart, yet have diffi culties in selecting data according to a given criterion; half of the pupils have no diffi culties in reading and representing data on a bar chart, but can't interpret the data; only a quarter of the pupils are able to read and represent data on a bar chart and interpret the data, yet have diffi culties in classifying data and interpreting data obtained using number operations; and only about 10 % of the pupils are able to perform correctly all of the above, yet have problems in interpreting data on more complex tables or charts involving number operations.Other research on mathematics achievements of elementary school pupils have not been undertaken at national level, because of which it is not possible to compare and evaluate the conclusions drawn from the National Assessment of pupils' achievement in elementary school in 2001.
Apart from the above mentioned national study on mathematics achievements of pupils at the end of elementary school there are no other public records on this issue.It is not known whether there are diff erences in pupils' mathematics achievements on national, regional, municipality or school level although the national mathematics curriculum is mandatory for all schools.
Th e international study TIMSS (Trends in Mathematics and Science Study) conducted in 1995, 1999, 2003, 2007, 2011, examined trends in grade 4 and grade 8 pupils' mathematics achievements in approximately 40 countries worldwide.Macedonia participated only in TIMSS 1999, TIMSS 2003, TIMSS 2011, with a representative sample of the population of eighth graders.In each of these studies the pupils from Macedonia demonstrated lower levels of achievement compared to all European countries in the study (Mullis, Martin, Gonzalez, & Chrostowski, 2004;Mullis, Martin, Foy, & Arora, 2012), among which countries from former Yugoslavia sharing similar traditions in elementary education.For example, the international average percent correct in TIMSS 2012 achieved by grade 4 pupils in the content area data display -58 % is higher than for the other content areas (number -47 %, geometric shapes and measures -49 %).Similarly, the international average percent correct in TIMSS 2012 achieved by grade 8 pupils in the content area data and chance -45 % is higher than for the other content areas (number -43 %, algebra -37 %, geometry -39 %).Th e average percent correct in TIMSS 2012 achieved by grade 8 pupils in Macedonia in the content area data and chance -33 % is the same as for the other content areas (number -32 %, algebra -32 %, geometry -33 %) and lower than the international average.Th e authors of the study remarked having reservations about the reliability of average achievement of Macedonian pupils because the percentage of students with achievement too low for estimation does not exceed 25 %, but exceeds 15 %.
It is important to note that Macedonian 10-11 years old elementary school pupils were not included in any of the tests conducted under TIMSS and therefore there is no possibility for comparison of their achievements with the achievements of grade 4 or grade 5 pupils from other countries nor to monitor trends in mathematics achievement of 10-11 year old pupils in the country over the years.
Given that, worldwide, the content area data was introduced in the mathematics curricula for elementary education signifi cantly later than other content areas, such as Numbers and Operations, Algebra, Geometry, Measurement, TIMSS is one of the few cross-national studies which provide research data on student achievement in this relatively new school mathematics content area.

Method
An empirical research has been conducted with the aim of collecting evidence for the achievements of 5 th grade pupils in the content area data.Released items from the National assessment in Macedonia in 2001 and from TIMSS 2003 have been used as the test instrument (two of the items are given in the Appendix).
Th e main research questions to be addressed in the paper are: 1. What kind of knowledge, understanding and skills do grade 5 pupils have at the end of the school year in the content area data? 2. Are there diff erences in the level of achievements in the content area data among pupils from schools in urban, rural and suburban areas? 3. On which test items there are diff erences in the students' achievements among pupils from schools in diff erent areas? 4. Which errors appear most oft en in pupils' responses to the test items?Th e research population consisted of grade 5 pupils at the end of 2011/2012 school year.Th e sample consisted of 404 pupils in 25 classrooms from nine schools in rural environments (2 schools, 3 classrooms, 44 respondents), suburban environments (1 school, 4 classrooms, 67 respondents) and urban environments (6 schools, 16 classrooms, 303 respondents) in 4 diff erent regions (out of 8 regions) in the Republic of Macedonia.
Data were analyzed with the statistical package SPSS 20 using the chi -square test.A comparison was made of the success in solving the test by pupils from various schools, by pupils from schools in diff erent areas of urbanity, and a comparison was made of the success on each item separately by pupils from diff erent schools and diff erent areas of urbanity.

Results
Statistical indicators of the results obtained on the test conducted within the research in the country in 2012 are shown in Table 1.
Th e most frequently obtained result is 12 correctly solved items out of the given 13 items (16.3% of pupils), and the average number of correct responses on the items is approximately 10. Th e distribution of the results is shown on the histogram in Figure 2. From the value of the skewness and the value of the kurtosis as well as the visual image obtained by the histogram of the distribution, it can be concluded that the distribution of results is not normal and the curve is shift ed to the right.

Figure 2. The distribution of the test results
Th e achievements on each test item are represented on the bar chart in Figure 3.   Th e chi -square test shows that there is a statistically signifi cant diff erence in the success (categorized as low with 0-4 correct responses, medium with 5-9 correct responses and high with 10-13 correct responses on the test) in solving the test among pupils from schools in diff erent environments (central urban areas, suburban area and rural areas), with moderate level of association ( 2 (4,n = 404) = 42.76,p<.000, Cramer's V = 0.23).
Th e Kruskal-Wallis test shows that there is a statistically signifi cant diff erence in the overall success in solving the test among pupils from schools in diff erent environments (central urban areas n=303, suburban area n=67 and rural areas n=34),  2 (2,n = 404) = 35.10,p<.000.Th e median result achieved by pupils in central urban schools is 11, in rural schools is 9 and in the suburban school is 8.5.Th e Mann-Whitney test revealed signifi cant diff erences between the results of pupils from the central urban schools and the results of pupils in the suburban school (U=6005, z=5.26, p=0.000, r=0.27), and between the results of pupils from the central urban Th e internal homogeneity of the test is relatively acceptable (Cronbach's alpha is 0.675) having into consideration that the test consists of test items selected from two distinct sources.
Th e chi-square test shows that there is a statistically signifi cant diff erence in the success in responding correctly to the test items in the content area data among pupils from diff erent schools,  2 (16, n = 404) = 51,012, p <.000 Yet, the chi-square test for two schools in the same urban area (the state capital) shows no signifi cant diff erences on each test item.Th e same is true for the two rural schools in the sample and for the two urban schools in a same town from another region except for the results on Item 9 (Cognitive level -Reasoning),  2 (2,n = 55) = 6.35, p <.05, Cramer's V = 0.34, and for three urban schools in two diff erent regions except for the results on Item 7 (Cognitive level -Solving Routine Problems),  2 (1,n = 103) = 6.04, p <.05, Phi = 0.24.

Th e school area as a factor
Since the best results on the test were achieved by pupils in urban schools, and lower scores by pupils in suburban and rural schools, the impact of the school environment on the pupils' performance on the test was examined.
Th e survey included two schools from rural areas (3 classrooms, 44 respondents), six schools from central urban areas (16 classrooms, 303 respondents) and one school from a suburban area (4 classrooms, 67 respondents).No general conclusions may be derived from the comparisons of the results of the pupils from schools in diff erent areas; they may only serve the purpose of opening paths for further investigations.schools and the results of pupils in the rural schools (U=3325, z=3.41, p=0.001, r=0.19).No signifi cant diff erence was found between the results of pupils from the rural schools and the results of pupils in the suburban school (U=1042, z=0.70, p=0.483, r=0.07).
Th ere are signifi cant diff erences among pupils from diff erent areas in responding correctly to each test item (Table 6), yet the level of association is very weak or moderate.In general, pupils from central city schools show greater success in solving the items on the test in comparison to pupils in schools from suburban areas and rural areas.Pupils from suburban areas showed the least success.
Particularly striking is the low level of success achieved on the test by the pupils of a suburban school in the country's capital city.It is a school with pupils from ethnically mixed community with a large percentage of Roma children (20 %).On more than 60 % of the test items, the average achievements of the pupils of this school are the lowest compared with the average achievements of pupils from all other schools surveyed.TIMSS 2011 (Mullis et al., 2012) emphasizes the fact that the most successful schools are usually schools that have pupils who come from economically better standing families, whose native language is the language of instruction and who enter elementary education with developed numerical skills.Th ese may be the primary factors infl uencing the results obtained on the test.
With the aim of exploring the contribution of the low test results achieved by Roma children to the overall test achievements in the school, the Kruskal-Wallis test was applied to the results achieved by the other pupils in the school when the results obtained by the Roma pupils were excluded from the analysis.It shows that there is a statistically signifi cant diff erence in the overall success in solving the test among pupils from schools in diff erent environments (central urban areas, rural areas and a suburban area -the school results without Roma pupils' results),  2 (2,n = 391) = 19.45,p<.000.Th e median result achieved by pupils in central urban schools (11) is higher than the median results (both 9) obtained by pupils in rural schools and in the suburban school.Again, the Mann-Whitney test revealed signifi cant diff erences between the results of pupils from the central urban schools and the results of pupils in the suburban school (U=5972, z=3.18, p=0.001, r=0.17), with weaker infl uence than when the results of the Roma pupils were not excluded.As previously, no signifi cant diff erence was found between the results of pupils from the rural schools and the results of pupils in the suburban school when the results of the Roma pupils were excluded (U=798, z=1.03, p=0.302, r=0.11).
It appears that the low results achieved by the Roma pupils (13 out of 67 pupils) in the suburban school contribute to the signifi cance of the diff erence in the success of responding correctly to almost half of the test items (Table 7).It is interesting to note that, although pupils from rural areas generally showed lower results in solving the test items, the best results on Item 9 were achieved by pupils from a rural school.A possible explanation for this deviation could be sought in the cognitive level of Item 9 (Reasoning) or in what the task measures, which is the pupils' ability to display results of addition and subtraction needed to solve a word problem, on a bar chart.It is possible that the teachers from this school practice to a greater extent with their pupils solving word problems which are reduced to solving a numerical expression with two operations, resulting in a greater pupils' success in setting a strategy for solving this task.However, this assumption remains in the domain of unverifi ed assumptions until appropriate measurement is performed.

Interpretation of pupils' errors in responding to the test items and recommendations
When solving tasks in the content area data pupils face diffi culties arising from several sources.
Item 1 measured pupils' ability to read data presented on a bar chart.Th e cognitive level of the task is defi ned as Solving routine problems.Th e most common mistake that pupils have made in solving Item 1 originates from pupils' unpreparedness to locate a number on the number line.
Item 2 also measured pupils' ability to read data presented on a bar diagram.Th e cognitive level of the task is defi ned as Solving routine problems.Th e most common error that occurred when solving Item 2 was pupils giving answers to questions that they assumed had been asked (Which day the most tickets were sold?) instead of the question actually posed on the test (Which day were sold 500 tickets?).
Item 3 measured pupils' ability to read data presented on a bar chart and to order data.Th e cognitive level of the task is Solving routine problems.As on Item 1, the most common mistake that pupils made in solving Item 3 relates to pupils not knowing how to assess the position of the number on the number line.
Item 4 measured pupils' ability to read data represented symbolically in a table and to represent data on a bar chart.Th e cognitive level of the task is Reasoning.Analysis of the pupils' responses on Item 4 reveals that pupils do not know how to read a table of symbolic representations (instead of considering that an asterisk represents fi ve votes, they used it as only one vote).
Item 5 measured pupils' ability to read data given in a table.Th e cognitive level of the task is Solving routine problems.Th e most common mistake that pupils made in solving Item 5 was not responding to the question Which class has read the most books?, the answer of which was to be found by comparing numbers in the summary column.Instead, they determined whether boys or girls read more books (comparison of numbers in the summary row), that is, pupils did not respond to the question posed.
Item 6 measured pupils' ability to operate with data presented in a bar chart.Th e cognitive level of the task is Solving routine problems.Again, the most common error that occurred when solving Item 6 was pupils addressing a question which they assumed had been set (How many pupils received the mark of excellence?) instead of the question posed (How many pupils have been tested?).
Item 7 measured pupils' ability to read the data presented on a bar chart.Th e cognitive level of the task is Solving routine problems.As in Item 2, Item 5 and Item 6, the most common error that occurred when solving Item 7 was again pupils addressing a question which they assumed had been set (Which class collected the most bottles?) instead of the question posed (Which class collected almost 45 bottles?).
Item 8 measured pupils' profi ciency in reading data presented on a bar diagram.Th e cognitive level of the task is Solving routine problems.Among those pupils who incorrectly answered the question there were pupils who correctly entered the names of the children under the columns in the chart according to their height, but they either did not answer the question or they answered a question which was not asked (e.g.Who is the highest student?instead of How tall is Sara?).
Item 9 measured pupils' profi ciency to represent the results of addition and subtraction used to solve a word problem, on a bar chart.Th e cognitive level of the item is Reasoning.Th is item is an example of an item which most likely was solved correctly by all pupils who have achieved international middle threshold (475 out of 800 units) in TIMSS 2003 (Mullis et al., 2004).Some pupils who did not respond correctly to the question posed, had a problem with setting up a numerical expression with two operations, and another part of the pupils were confused by the need to identify the location of the number 5 on the number line, which in itself is a possible indicator that the number line as a representation is not suffi ciently present in the elementary mathematics instruction.
Item 10 measured the ability of pupils to read data represented symbolically in a table and to compare data.Th e cognitive level of the task is Solving routine problems.Th e incorrect answers refl ect a misinterpretation of the symbols in the table.In fact, some of the pupils treated the crossed slashes as symbols for items sold and the rest as representing unsold items.
Item 11 measured the ability of pupils to read data presented in a table.Th e cognitive level of the task is Solving routine problems.Th e most common errors can be classifi ed as partial solution of the task, lack of attempts to solve the problem or computation of a sort instead of comparison.
Item 12 measured the ability of pupils to read data represented symbolically in a table and to represent data on a bar chart.Th e cognitive level of the task is Applying.Again, problems arise in the misinterpretation of the symbols in the table or failure to accede to locate the number 9 on the number line, or to perform a proper assessment of the height of the column in relation to the assigned numbers.
Item 13 measured the ability of pupils to read data represented on a line graph.Th e cognitive level of the task is Solving routine problems.

Discussion of the results
Th e area where the school is set appears as a signifi cant factor in the level of development of pupils' skills for working with data.Th e pupils from central urban areas show greater success in solving the tasks on the test, compared to schools from suburban areas and rural areas.Pupils from suburban areas achieved the lowest success in responding to the questions on the test.Th e results achieved by the pupils in a suburban school were signifi cantly lower than the results obtained by the pupils in the central urban schools.On almost half of the test items the diff erence originated from the contribution of the extremely low results achieved by the Roma pupils who represent 20 % of the grade 5 school population.Pupils from the Roma population are considered as a socially vulnerable group whose educational needs are not being addressed properly despite the widely publicized eff orts of the governmental and non-governmental sector during the last decade.
Finding the reasons that infl uenced the formation of the resulting image is a problem, which is not being addressed within the framework of this paper.One of the legitimate questions that might be asked is whether teachers who teach classes in different areas have adequate training and/or continuing support for working with pupils in the content area data, as well as what importance do teachers assign to this content area.In this respect, the answer could be sought in comparing and analyzing data that would be obtained from: -Th e annual, thematic and daily planning made by teachers (intended curriculum); -Th e implemented instructional activities in the content area data (implemented curriculum); -Teachers' attitudes towards working with pupils in the content area data; -Initial pre-service training of teachers for work in the content area data; -In-service training of teachers for working in the content area data.
Gaining insight into the teaching resources that are used in a variety of environments, ranging from textbooks, classroom materials prepared by teachers themselves and the application of information and communication technologies would be important.According to the fi ndings obtained in many studies, including TIMSS 2011 (Mullis et al., 2012) , successful schools are schools with better working conditions, in terms of space and in terms of teaching materials such as books, computers, technology support and the like.Another possible factor in-fl uencing pupils' achievements is the home environment -the family support and the learning resources available to pupils in their homes, which is outside the scope of this research.
Among pupils from schools in diff erent areas there are statistically signifi cant diff erences in the success in solving tasks on all cognitive levels: Applying (application of concepts), Reasoning, as well as Solving routine problems.
Th e most frequent error originates from pupils' habit of incomplete reading of the tasks.Pupils are not trained consistently to approach problems by reading and understanding the requirements posed by the task.Instead, they immediately scan the problem visually, if a graph is given, without completely fi nishing reading the question which is posed, consequently giving an incorrect answer.Th is practice is probably a consequence of possible teachers' tendency to read aloud and fully explain the requirements of the tasks before inviting pupils to work on the tasks.Th is practice carries a double risk: lowering the cognitive level of the task as well as strengthening the habit of not reading the requirements of the task.
During the test, pupils demonstrated insecurity when facing the challenge of how to solve a test with 'so many' items.Some of the respondents were waiting for help from the teacher or the examinerpossibly as a consequence of the established practice of assistance provided by teachers whenever the pupils face diffi culties.
What can be noticed is that pupils make mistakes in solving tasks the solutions of which do not consist of a single step.Th is fi nding is most likely due to the repeated interventions of teachers at each step of solving the tasks with directions for pupils how to proceed, which signifi cantly decreases the cognitive level of the problems.In this way, opportunities to train pupils to solve problems autonomously and to create optimal conditions for the development of strategic thinking of pupils are not used.
A possible factor in producing mistakes in solving the test is the lack of practice of external evaluation of elementary school pupils (the External evaluation on national level has been introduced in 2013), and the fear of a new type of activity that naturally occurs in children.
Th e authors of this paper recommend that all pupils are trained for independent work in solving the tasks from reading the task and explaining the task requirements to performing each step separately.Th us, multiple positive eff ects on the development of mathematical competence of pupils would be achieved, on the development and strengthening of positive character traits such as perseverance, persistence, patience, ability to invest focused eff orts in achieving the goal set as well as a commitment to quality.
Also, greater attention to solving tasks with higher cognitive demands is required, instead of solving a multitude of routine problems.Th is objective should be clearly defi ned and adopted in the national documents that determine the quality of education in the country, as well as by all stakeholders that infl uence education policy making.Comprehensive review of the textbooks for elementary mathematics instruction can reveal the depth of the problem that teachers consciously or unconsciously deepen through the choice of many drilling tasks with low cognitive levels, that is, routine problems aimed at practicing formal procedures, and by avoiding to work on tasks at higher cognitive levels (scarcely present in textbooks as it is), which promote conceptual understanding applied in problem solving.
Mathematics curricula and textbooks used in school mathematics instruction are just some of the components that need to be evaluated and, if necessary, redesigned in view of changing the focus of education in the direction of adopting quality knowledge and development of skills for solving problems.Th ere is a need to redefi ne the goals of mathematics teaching and of didactics of mathematics within the initial teacher education for elementary school teachers and a need to standardize the necessary elements that make up the university education in terms of compulsory and optional mathematics courses (CBMS, 2012).Th ese recommendations are a part of the recommendations found within numerous international studies that deal with the issue of the quality of teachers' professional knowledge as a factor in the development of mathematical competence of pupils (see for example Hill, Rowan & Ball, 2005) and recommendations for initial teacher education (CBMS, 2001;CBMS, 2012).

Figure 3 .
Figure 3.The achievements on each test item

Figure 5 .
Figure 5.Comparison of the success in responding correctly to test items among pupils from different areas of urbanity

Item 4 (
Item 8 of the National Assessment 2001) Pupils voted for the most favorite sport.Th e resulting data are given in the table.Represent the data in a bar chart.Item 10 of the National Assessment 2001)Th e table shows data on books read in four classes in a school.In which class have the pupils read the smallest number of books?

Table 1 .
Statistical indicators of the results in 2012

Table 4 .
Th e cognitive level of the tasks and the task description of

the items Item Th e cognitive level of the task Task description
Reading data presented in a bar chart and interpretation (comparison) of data 4 ReasoningReading data froma table, symbolically represented, and their presentation on a bar chart 5 Solving routine problems Reading data from a table 6 Solving routine problems Reading data presented in a bar chart 7 Solving routine problems Reading data presented ina bar chart 8 Solving routine problems Presenting data on a bar chart 9 Reasoning Displaying results of operations for solving word problem on a bar chart 10 Solving routine problems Reading and comparing data symbolically represented in a table 11 Solving routine problems Reading data presented in a table 12 Applying Reading data froma table, symbolically represented, and their presentation on a bar chart 13 Solving routine problems Reading data presented on a line graph

Table 6 .
Statistical signifi cance of the diff erence of the results achieved by pupils from schools in diff erent areas on each test item