2. Spatial thinking and relational thinking

Spatial thinking is a kind of thinking that has been receiving more and more attention from educators in the past tenyears. It refers to the

knowledge, skills, and habits of mind related to the use of: concepts of space; tools of spatial representation; and processes of spatial

reasoning (National Research Council, 2006). According tothe authors of the influential book “Learning tothink spatially”, spatial thinking is

universal kind of thinking that is useful in a wide variety of academic disciplines and everyday problem-solving situations (National

Research Council, 2006). Huynh and Sharpe (2013, p. 3) argue that spatial thinking is “core to the theoretical and practical un-

derpinnings of geography”. Spatial thinking within the geography domain is somewhat different from spatial thinking in other domains.

Geographers typically use a geospatial reference frame. They study problems that are connected to locations on Earth, rather than problems

in abstract thinking spaces. In other words: for geographers, the place on Earth matters.

Spatial thinking consists of several distinct but interrelated abilities. It is somewhat broader than spatial ability, although the two are

related to each other (Lee & Bednarz, 2009). Cognitive psychologists (e.g. McGee,1979) and geographers (e.g. Huynh & Sharpe, 2013; Lee &

Bednarz, 2009) generallyagree that two keycomponents are ‘spatial visualization’ and ‘spatial orientation’. Spatial visualization is the ability

to mentally manipulate, rotate, twist, or invert individual objects (McGee, 1979), while spatial orientation is the ability to determine the

relative position of objects, to change the point of view, and to maintain a sense of direction when moving through an environment.

However, there is some disagreement about whether a third component, ‘spatial relations’, is also part of spatial ability (Gilmartin & Patton,

1984; Golledge & Stimson, 1997; Lohman, 1979; Montello, Lovelace, Golledge, & Self, 1999). Many geographers argue that the ability to

reason about spatial distributions and patterns, spatial interactions, and spatial relations is an important part of spatial thinking (Albert &

Golledge, 1999; Bednarz, 2004; Gilmartin & Patton, 1984; Golledge & Stimson, 1997; Huynh & Sharpe, 2013; Lee & Bednarz, 2009; Self &

Golledge, 1994). The spatial relations component is the most often addressed component in geography courses (Bednarz, 2004), and

Huynh and Sharpe (2013) argue that it’s the geospatial reference frame and the focus on spatial relations what make geospatial thinking

different from the general kind of spatial thinking.

Although manygeographers argue that the spatial relations component is the most important componentof geospatial thinking, it is also

the least clearly defined component (Lee & Bednarz, 2009). According to Golledge and Stimson (1997, p. 158), the spatial relations

component includes the ability “to recognize spatial distributions and spatial patterns, to connect locations, to associate and correlate

spatially distributed phenomena, tocomprehend and use spatial hierarchies, to regionalize, to orientate to real-world frames of reference, to

imagine maps from verbal descriptions, to sketch maps, to compare maps, and to overlay and dissolve maps”. Although this description of

the spatial relations component has formed the basis for several geospatial thinking tests (among others: Huynh & Sharpe, 2013; Lee &

Bednarz, 2009), it has some shortcomings.

First, many of the subcomponents in Golledge and Stimson’s definition of the spatial relations component are not directly linked with

spatial relations. For example, recognizing spatial patterns can be a prerequisite for recognizing spatial associations, as it requires one to

compare two spatial patterns to see if there is any match. However, recognizing spatial patterns can also be a goal in itself.

Second, the terms contain different verbs, such as: ‘recognizing’, ‘connecting’, ‘comprehending’. Most refer to the construction of new

knowledge. This is a somewhat limited view on spatial thinking, as it hardlyaddresses spatial reasoning skills. For example, students should

not only be able to recognize spatial distributions and spatial patterns, but should also be able to explain them, and to reason how they

might change in the future due to autonomous processes or as a result of human interventions in the system. This kind of reasoning requires

them to structure and apply their knowledge about spatial associations or causal relations. These spatial reasoning processes are an

important part of spatial thinking, as they are needed to understand, for example: how the Arab Spring is related to the heat wave in Russia

in 2010; why some glaciers in South Norway are advancing, while they are retreating in the rest of the world; and why the construction of

the Erie Canal resulted in an economic boom in New York, while the construction of the Beaver Canal failed to do the same for Philadelphia.

For the reasons outlined above, it might be better to change the name of the component from ‘spatial relations’ to ‘spatial analysis and

spatial reasoning’, and to distinguish a subcomponent called ‘geospatial relational thinking’. It is clear now that this subcomponent goes

much further than recognizing spatial associations alone, as it also includes geospatial relational reasoning processes. The subcomponent

connects to systems thinking, which is a holistic approach that focuses on how the constituent parts of a system are related to each other,

how such systems respond to changes, and how systems work within the context of larger systems. It aims to provide insight into how

things influence one another within a whole system over time, based on the belief that the components of a system can best be understood

in the context of relations with each other and with other systems, rather than in isolation. Systems thinking often focuses on cyclical cause

and effect. Attention to feedback mechanisms, with their reinforcing effects (positive feedbacks loops) or balancing effects (negative

feedback loops), is an essential component of systems thinking.

Systems thinking is well anchored in the earth sciences tradition, which is one of the four traditions in geography distinguished by

Pattison (1964). An earth systems thinking approach views the world as a system that consists of four central subsystems (Mayer,1995): the

geosphere, the hydrosphere, the atmosphere, and the biosphere (including humans). Orion’s (2002) didactic model for science education

based on the earth systems approach emphasizes the study of geochemical and biogeochemical cycles in the different subsystems (e.g. the

rock cycle, the water cycle, the food chain, the carbon cycle, and the energy cycles), and the interrelations between those subsystems

through the transfer of matter and energy from one subsystem to another. In order to be able to reason about relations in such systems,

students should develop a dynamic, cyclic, and systematic representation of these systems, thereby taking feedback mechanisms, stocks,

and time lags into account (Kali, Orion, & Elon, 2003).

Although earth systems thinking and geospatial relational thinking both aim to structure the complexity of systems, there are also

several differences. The first difference is that earth systems thinking solely focuses on physical and chemical processes, while geospatial

relational thinking focuses on the natural system, the human system, and the relations between these two systems. Van der Schee (2000)

distinguishes two types of relations: vertical relations; and horizontal relations. Vertical relations are relations within regions, such as the

relations between different physical geographic properties in a region (e.g. soil, climate, hydrology, ecology) and relations between different

human geographic properties (e.g. demography, sociology, economies, politics) in a region, and human–nature relations. As regions are

situated in networks, and as flows may occur between regions, a change in a property of one region may result in a change in a property of

another region. This is called a horizontal relation. Because of these horizontal relations, the interlinked natural system and human system

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 226

can be seen as a spatial system. This system provides opportunities and challenges fordifferent land-use functions (e.g. residential, transport,

agriculture, recreation, nature). In turn, land-use functions can affect the system via environmental pollution, groundwater extraction,

fertilizing, etc. People can apply planning measures to change the system, and in such a way optimize the opportunities and decrease the

challenges.

The second difference lies in the analysis level. In geospatial relational thinking, the analysis units are concrete regions (e.g. ‘the Dutch

Delta Region’), while in earth systems thinking, the analysis units are usually more abstract system components (e.g. ‘the atmosphere’). For

example, geographers can reason about the effects of sea level rise on salinisation in the polders of the Dutch Delta Region, thereby taking

the flow of salt groundwater from the estuaries to the polders into account. In contrast, earth scientists can reason about the effects of CO 2

emissions on climate, thereby taking flows of carbon between the atmosphere, hydrosphere, biosphere, and geosphere into account. So, in

contrast to earth systems thinking, the relations in geospatial relational thinking are connected to concrete geospatial units: regions, places,

etc.

The third difference lies in the kind of representations that are used tostudy relations. Kim (1990) argues that systems thinkers can use at

least ten different kinds of tools. Maps are not included in Kim’s list, while they are one of the most important sources of information about

geospatial relations, as they can show the match between two spatial distributions or spatial patterns or show the flow of people, products,

money, etc. between regions. Besides maps, geographers also often use time charts, scatterplots and conceptual frameworks of causal

relations, just as earth system thinkers.

Conclusively, we can say that geospatial relational thinking is a deep kind of thinking which has some links with systems thinking.

Geospatial relational thinking is an important part of geospatial thinking, which is a valuable way of thinking to analyse and reason about

the big challenges in the world around us.

Geospatial technologies seem to hold promising opportunities for stimulating students’ geospatial relational thinking and other aspects

of geospatial thinking. The next section discusses how geospatial technologies can be used to stimulate the development of geospatial

(relational) thinking, and what is known about the learning effects of lessons series with geospatial technologies on students’ geospatial

(relational) thinking.

3. Geospatial technologies

One of the main innovations in geography teaching in the past few decades has been the introduction and diffusion of geospatial

technologies, which are technologies that offer access to geospatial information via digital representations (such as digital maps), and tools

for interaction with those representations. Geospatial technologies allow us to work with large quantities of detailed, authentic and up-to-

date geospatial information in a fast, flexible, and user-friendly way. Well-known types of geospatial technologies are global positioning

systems (GPS) and virtual globes such as Google Earth. Another type of Geospatial technologies is desktop GIS (Geographic Information

Systems), which is professional software for visualising, creating, manipulating, querying, analysing and presenting digital maps. Desktop

GIS is widely used in business, government and science for studying spatial problems and exploring solutions for these problems. Recently, a

large range of WebGIS applications have become available: websites that contain a interactive windows for viewing maps and other kinds of

geospatial information. They generally offer far less tools and an easier interface than desktop GIS (Kim, Kim, & Lee, 2013; Milson & Earle,

2007). Some WebGIS applications are specifically developed for use in secondary education, and can be seen as a sort of ‘web-based atlas’. A

special category of geospatial technologies are serious geogames, which are games based on a schematic map and a computer model. They

are not only entertaining, but also aim to educate students about specific geographic problems.

Geospatial technologies offer many opportunities for designing interesting and challenging geography lessons. Bednarz and Van der

Schee (2006) argue that they can be used to raise students’ understanding of local to global issues. They can also be used to train impor-

tant skills, such as: analytical skills; critical thinking skills; communication skills; and inquiry skills (e.g. Audet & Ludwig, 2000; Demirci,

Karaburun, & Unlu, 2013; Favier & Van der Schee, 2012; Kerski, 2003; Milson, Demirci, & Kerski, 2012). Besides this, many authors argue

that geospatial technologies offer many opportunities for stimulating geographic thinking skills, in particular: geospatial thinking skills

(Bednarz, 2004; Huynh & Sharpe, 2013; Lee & Bednarz, 2009; National Research Council, 2006; Nielsen, Oberle, & Sugumaran, 2011).

Regarding the development of students’ geospatial relational thinking, we can identify seven major advantages of geospatial technologies

over traditional means (e.g. paper maps):

1) Many geospatial technologies offer enormous amounts of map layers about interrelated phenomena. As the map layers cover the same

regions and have the same aggregation levels, students can easily explore them to see if there are any associations (Bednarz & Van der

Schee, 2006).

2) Most geospatial technologies allow students to stack map layers on top of each other, to switch them on and off, and to make them

transparent. Some applications also offer the possibility to put map layers next to each other. These functionalities make it easier for

students to identify spatial associations (Bednarz & Van der Schee, 2006).

3) Some applications offer additional textual, audio, or video information about regions and phenomena. See, for example, the Wikipedia

and YouTube layer in Google Earth. This can provide additional information about the geographic system, which can help students

construct knowledge about cause-and-effect.

4) Some geospatial technologies allow students to investigate geodata in coupled representations: maps; tables; and scatterplots. When

students select a region on the map, the region also blinks in the table, and in the scatterplot. This also makes it easier for them to

identify spatial associations. Furthermore, working with such coupled representations often generate questions about why a region has

specific property values, why the property value is an exception to the rule, etc. In other words, it stimulates them to think about the

relations that underlie the geodata.

5) Some geospatial technologies allow students to calculate the strength of associations (correlation coefficients). This makes it possible to

follow a more quantitative approach to studying relations (Yano, 2000).

6) Many geospatial technologies offer the possibility to display temporal geodata in animated maps, histograms, temporal charts, and

scatterplots (Rosling, 2007). Students can explore these animations, in order to identify associations in the geodata.

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 227

7) Geospatial technologies such asserious geogamesallowstudents toevaluate theeffects of planning measures onthe geographic system,

and therefore provide insight into the different relations in the system. Research has shown that well-designed educational games

embedded inproper instruction can be motivational for students and can aid tothe development of higherorder thinking skills (Hainey,

Connolly, Stansfield, & Boyle, 2011).

The characteristics of geospatial technologies outlined above enable teachers to develop instruction methods that aim to stimulate

geospatial relational thinking skills that are often difficult to address. This connects to Palladino and Goodchild’s (1993) argument that

geospatial technologies make it easier to engage students in ‘higher order thinking activities that are often so hard to come up with.’ The

design of geography lessons with geospatial technologies is also often very different from the design of traditional geography lessons.

Lessons with geospatial technologies are often learner-centred, and their designs connect to (socio)constructivistic learning theories. In

such away, geospatial technologies have the potential to fundamentally alter the manner of teaching and learning in the classroom (Baker &

White, 2003; Kerski, 2003; Sinton & Lund, 2007).

Until the beginning of the 21st century, the adoption of geospatial technologies in schools proceeded slowly (Aladag, 2009; Cremer,

Richter, & Schäfer, 2004; Doering & Veletsianos, 2007; Kerski, 2003; Lam, Lai, & Wong, 2009; Shin, 2006; Zink & Scheffer, 2009). Often-

cited barriers include (Milson & Kerski, 2012): the lack of hardware, software, and instruction materials; teachers’ limited competencies

in teaching with geospatial technologies; and the absence of clear educational standards for geospatial technologies. However, in the past

fiveyears or so, notonly have computerfacilities at schools improved, but also awide rangeof freeand user-friendly web-based applications

and instruction materials have become available (Kim et al., 2013). Furthermore, geospatial technologies have now been included in the

curricula of more countries. Hence, many teachers have started making a first step (Milson et al., 2012). Still, there are some important

challenges to the further diffusion of geospatial technologies in schools. One of the frequently cited challenges is that little is known about

the effectiveness of lessons with geospatial technologies compared with lessons using analogue materials (i.e. paper maps). Some teachers

argue that integrating geospatial technologies might not be worth the extra effort. In a survey on the implementation of geospatial tech-

nologies in secondary education, one teacher expressed concerns about the effects of geospatial technologies on learning (Kerski, 2003: p.

133):

‘I personally have been troubled with the question of whether students are learning geographic inquiry strategies or merely learning to use a

very powerful tool without much thinking about the underlying questions under consideration.’

The lack of experimental research in the field of geography education and geospatial technologies was first noticed by Baker and Bednarz

in 2003, and has been underlined by other authors since (Kerski, 2008). Even today, scientific papers that report about the outcomes of

experimental research on the effectiveness of teaching with geospatial technologies are scarce (Demirci et al., 2013; Kim et al., 2013),

especially when it concerns the effects on specific thinking skills. There are some studies that showed significant higher achievement for

lessons with GIS (e.g. Baker & White, 2003; Goldstein & Alibrandi, 2013; Lee & Bednarz, 2009). However, geospatial relational reasoning is

not addressed in any of these studies. Existing standardized tests, spatial thinking tests, and spatial analysis tests do not fully test the skills

that students need to reason about the big challenges in the world around us, nor do they test the full width of geospatial relational thinking

skills that can be addressed in instruction methods with geospatial technologies. The problem is that in the spatial thinking literature, the

focus is often on learning to use (complex) geospatial technologies and its geospatial analysis methods, while the focus should be more on

learning to analyse and reason about the challenges in the world around us. Each of these studies made use of desktop GIS applications, a

kind of geospatial technology that has been known for its complexity (Kim et al., 2013). Therefore, a large number of lessons were spent on

‘learning how to handle the technology’ rather than ‘learning how to analyse and reason about the challenges in the world around us’. The

complexityof desktop GIS softwareand the time needed tomaster the software are barriers for incorporating GIS intothe classroom (Bodzin

& Anastasio, 2006; Kim et al., 2013; Milson & Earle, 2007). Today, young people are familiar with technology, but they are also easily

frustrated when technology is not intuitive (Nielsen et al., 2011). Therefore, several authors (e.g. Golledge, Marsh, & Battersby, 2007; Kim

et al., 2013) have argued that educators should use simple geospatial technologies, so called ‘minimal GIS’. Such geospatial technologies

can play a critical role in realizing the pedagogical potential of geospatial technologies.

So, in summary, it is evident that although many educators have the feeling that teaching and learning with geospatial technologies can

contribute to higher achievement in geospatial thinking, there is still little concrete evidence for this assumption, especially as it concerns

geospatial relational thinking.

4. Research questions

This paper attempts to contribute to knowledge development in the field of geography education with geospatial technologies, by

investigating the effects of a geography lesson series with geospatial technologies. The main questions of the research are: (1) ‘What are the

effects of a lesson series with geospatial technologies on high school students’ achievement in geospatial relational thinking, compared with

a conventional lesson series which focuses on the same content?’; and (2) ‘What are students’ perceptions about the learning effects of a

lesson series with geospatial technologies, compared with a conventional lesson series?’.

5. Methods

5.1. Research design

A quasi-experimental approach was followed to get more insight into the relation between the type of instruction (independent vari-

able), the scores on a geospatial relational thinking test (dependent variable 1), and students’ perception of the learning effects (dependent

variable 2). The research consisted of an experimental group who followed a lesson series with geospatial technologies, and a control group

who followed a conventional lesson series. Students in both test groups did a pre-test (30 min); then followed a series of three lessons

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 228

(3 ? 45 min); and, finally, did a post-test (30 min). In order to investigate the differences in students’ perceptions about the learning effects

of the lesson series, a survey was included at the end of the pre-test and post-test.

5.2. Test groups

Tests were conducted at five different schools in the Netherlands, in 3rd grade HAVO (General Secondary Education) and 3rd grade VWO

(Pre-university Secondary Education). This is comparable to 9th grade high school classes in the USA. The students were 14–15 years old.

Table 1 presents the characteristics of the test groups.

Classes were assigned tothe two conditions. Each teacher had two classes: one that followed a lesson series with geospatial technologies

(the experimental class); and one that followed the conventional lesson series. Over 300 students participated in the research, but only

those students who completed the entire lesson series and both tests were included in the data analysis: 139 students in the experimental

condition, and 148 students in the control condition.

In many respects, the five schools were quite similar. They were all located in small to mid-sized towns of 30,000–300,000 inhabitants in

the Western part of the Netherlands. They had 800 to 1400 students, with a low (<5%) population of students with a non-European

background. All five schools scored 0.1 to 0.2 point higher than the national average on national exams (SchoolVO, 2013).

5.3. Design of the lesson series

The lesson series focused on water-related spatial planning issues in two regions in the Netherlands: the Delta Region; and the River

Region. In this research, the main requirement of the lesson series was that the lesson series of both the control group and the experimental

group should be realistic for the present educational context. The lesson series should fit with the geography curriculum, and should fit with

the skills of the average teacher (no special training should be needed, and teachers should be able to conduct the lesson series without

assistance).

The control group followed a conventional lesson series that was based on the schoolbook ‘Buitenland’. Most lessons started with a

discussion of the texts, pictures and maps in the textbook. Some teachers used additional sources of information about water-related

planning issues, such as YouTube videos and newspaper articles. After this introduction, students subsequently worked on assignments

in the workbook. These assignments mainly focused on the properties and structures of the water system in the Delta Region and River

Region in the Netherlands, and, to a lesser extent, on the functioning of the system. Spatial planning measures were explicitly listed in the

textbook. Most teachers organized a classroom discussion at the end of each lesson, in which they discussed part of the assignments.

Students in the experimental group followed a lesson series with geospatial technologies. In the first two lessons, students followed a

module of The Water Manager, a serious geogame for secondary education. The game consists of two modules: one about the Delta Region,

and one about the River Region. The game uses schematic maps of the two regions (Fig. 1). In the game, students take the role of water

manager. They have tofind an optimal solution for water-related challenges (increased flood risk) in schematic maps of the Delta Region and

River Region. They can select different kinds of measures, and drag these measures to different places in the schematic map (Fig. 1). A

simulation subsequently shows the effects on the system, and on the different land-use functions. Students can see that the effects of

measures depend on the local circumstances, and that measures also often have effects in other regions (i.e. upstream or downstream). In

order to obtain a maximum number of points, students have to carefully weigh the effects on the different planning measures. After gaming,

students work on assignments inwhich they have to summarise the system rules: they have to draw arrows in a conceptual framework that

represent the relations between measures and land-use functions (Fig. 2). Research by Hwang, Yang, and Wang (2013) showed that inte-

grating concept mapping in game-based instruction can significantly improve the learning outcomes, as it helps them organize what they

have learning during the game-based learning process.

After the lesson with the game, students then switch to another application: EduGIS, a webatlas for secondary education in the

Netherlands. Students have to investigate which measures were taken in the Delta Region and River Region in the context of the ‘Delta

Works programme’ and ‘Room for the Rivers programme’, and why these measures were taken at these locations. They have to study digital

maps, like a land-use map, a map of the population density and a water system map, in order to answer these questions. In addition, they

have to apply the system knowledge they have developed when playing the game.

In the final lesson, students also investigated the safety norms of different polders in the Netherlands with the help of EduGIS maps.

Safety norms are set by the Dutch government, and the height of dykes around polders is based on the safety norm of those polders. The

safety norm depends on the number of inhabitants and economic value of the polder, the type of water threat (sea, river or lake), and the

water depth in case of a dyke breach, In the EduGIS assignment, the students had to investigate these relations by studying map layers of

these properties of polders (Fig. 3). They are thereby stimulated to vary one variable while keeping the others constant. So the assignment

Table 1

Characteristics of the test groups.

School Teacher Condition

Experimental group N (male) N (female) Control group N (male) N (female)

A A1 3rd grade VWO 12 8 3rd grade VWO 10 15

B B1 3rd grade VWO 16 10 3rd grade VWO 11 10

C C1 3rd grade HAVO 14 12 3rd grade HAVO 12 16

C2 3rd grade HAVO 12 18 3rd grade VWO 15 14

D D1 3rd grade VWO 9 10 3rd grade HAVO 7 14

E E1 3rd grade VWO 11 7 3rd grade VWO 12 12

Total 74 65 Total 67 81

Total 139 Total 148

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 229

can be seen as a ‘control of variables strategy’ task. They also had to make the causal relations explicit, and draw them in a partially

completed conceptual framework.

In summary, the lesson series in the experiment groupand control group covered exactlythe same topics and issues. However, the lesson

series with geospatial technologies placed more emphasis on systematic thinking about geospatial relations. As the lesson series with

geospatial technologies was very short, only two applications could be used. It was, therefore, not possible to make use of all seven ad-

vantages of geospatial technologies for stimulating students’ geospatial relational thinking (see Section 3). Only advantages 2 and 7 were

implemented.

In this research, existing educational geospatial technologies were used, without making adaptations to the applications. EduGIS and the

Watermanager are readily available for teachers via the Internet without costs (www.edugis.nl and www.watereducatie.nl). Both appli-

cations are very intuitive and user-friendly. They connect with the idea of ‘minimal GIS’ (Kim et al., 2013). EduGIS has been developed since

2005, by a consortium of, among others: VU University Amsterdam, Geodan (a geospatial technology and data consultancy), and Kadaster

(the Dutch Ordnance Survey). The application is used by more than 50% of the teachers in The Netherlands. The Watermanager has been

developed bythe VU University, commissioned by the National Water Education Board. The number of teachers who use The Watermanager

is probably less than 5%, as it is a relatively recent application.

Fig. 1. A screenshot of The Water Manager. The figure shows the sites where students can build the measure: “dam”.

Fig. 2. Assignment in which students had to complete a framework of causal relationships. The figure shows the effects of the measure “dam”.

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 230

5.4. Design of the test

As existing spatial thinking tests (e.g. Huynh & Sharpe, 2013; Lee & Bednarz, 2009) cover only a small part of geospatial relational

thinking, a new test was developed by the authors of this paper, building on the revised Bloom Taxonomy (Anderson & Krathwohl, 2001),

the literature about spatial thinking (see Section 2), earth systems thinking (see Section 2), map skills (e.g. Van der Schee, Van Dijk, & Van

Westrhenen,1992), and geographic thinking (e.g. KNAG, 2003). The test was discussed and validated with teachers. The test consisted of six

testitems (Table 2). The first and second items were‘control of variables strategy’ (CVS) testitems (Chen & Klahr,1999), inwhich students had

to identify relations in representations. The first dealt about a non-geographic topic. Students had to analyse a figure that contained eight

different paper planes, and the test results for each plane (the distance covered). Students were asked which factors determine how far a

plane can fly. In order to answer the question, students had to compare planes that are similar except for one property. In the second test

item, students had to analyse a water-related problem that was not covered by the lesson series, either in the experimental or in the control

group: the salinisation of the polders in the Westernpart of the Netherlands. Students were shown eight schematic maps and cross-sections

from the sea to a polder, and were asked to identify the factors that determine the degree of salinisation in a polder. In order to answer the

question, students had to compare cross-sections that are similar except for one property. Both test items aimed to provide insight into the

degree to which students were able to identify relations in representations, but the representations in item 2 were more difficult to analyse

than the representations in item 1.

The third and fourth test items were ‘conceptual framework completion tasks’, in which students had to fill out a partially completed

conceptual framework of causal relations. In the third test item, they were asked to draw arrows between boxes with the causes and effects

of the 1953 floods in the southwest of the Netherlands. The content was not very difficult, and students were expected to possess the

required knowledge. In the fourth item, students had to fill out a partially completed conceptual framework about the factors that are

related to the water level in a lake. Students were asked to draw arrows between different variables: water level; inflow; outflow; pre-

cipitation; evaporation; and temperature. The content of this test item is more difficult to structure than the content of the third test item,

but again, students were expected to possess the required knowledge. The two test items aimed to provide insight into the degree to which

students were able to organize their knowledge about relations in a conceptual framework. Test item 4 contained more relations than test

item 3, and the relations were also more complex.

Fig. 3. Screenshots of EduGIS. The figure shows the elevation map.

Table 2

Characteristics of the test items of the pre-test and post-test.

Nr Content Type Tested processes

1 Aeroplanes CVS task Identifying non-spatial relationships in representations

2 Salinisation CVS task Identifying geospatial relationships in geospatial representations

3 Floods of 1953 Conceptual framework completion task Organizing geospatial relationships

4 Water level in a lake Conceptual framework completion task Organizing geospatial relationships

5 Fictive delta Design task Evaluating measures and creating solutions

6 Real river Design task Evaluating measures and creating solutions

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 231

The fifth and sixth test items were semi-open design tasks. In the fifth test item, students had to plan measures in a schematic map of a

fictive delta system in order to prevent the delta from flooding and to make sure that there is enough fresh water available for agriculture,

while making sure that other land-use functions (mussel farming and freight shipping) can continue. This fictive delta system had different

properties from the Delta Region in the Netherlands, while the measures that students could choose from were similar to the measures

discussed in both the experimental and the conventional lesson series. The sixth test item focused on rivers. This item used a real topo-

graphic map instead of a schematic map, and required more complex geospatial relational thinking. Students had to plan measures in a

Dutch river (that was not covered by either one of the lesson series) in order to decrease the flood chance. The two items provided insight

into the degree to which students understood the functioning of the water system, and the degree to which they were able to evaluate the

effects of measures, and create an optimal solution for the problems in the two regions.

5.5. Design of the survey

In order to investigate the differences in students’ perceptions about the learningeffects of the lesson series, a survey was conducted that

contained three questions: ‘How much knowledge do you have about the characteristics of the water system in the Delta Region and the

River Region?’; ‘How much knowledge do you have about the functioning of the water system in the Delta Region and the River Region’; and

‘How much knowledge do you have about the water-related problems in the Delta Region and the River Region’. Students were asked to rate

their knowledge on a 1-to-5 point Likert scale. The survey was included at the end of the pre-test and post-test, which enabled us to

investigate in which degree students think their knowledge has increased.

5.6. Data analysis

The whole project (two times 30 min for the tests, and three times 45 min for the intervention) was conducted over the course of three

weeks. The post-test was the same as the pre-test. The pre-tests and post-tests delivered qualitative and quantitative data. The qualitative

data were coded by a research assistant, with the help of a coding scheme designed by the authors of this paper. In order to assess the

interrater reliability, 40 tests were also coded by the first author of this paper. Cohen’s Kappa was 0.96, which indicates an almost perfect

match.

The effect size was computed for each test item and for each condition, by subtracting the average pre-test score from the average post-

test score, and dividing it bythe pooled standard deviation (Cohen,1988). Only thosestudents whoconducted the pre-test, intervention, and

post-test were included in the data analysis.

6. Results

The experiment went well in both test groups. The teachers who conducted the lesson series with geospatial technologies spend just as

much time on the topic as teachers who conducted the conventional lesson series: three lessons.

No technical problems occurred in the experimental group. Students played the game attentively and enthusiastically, and completed

each module in about 20 min. However, there were some within-group differences when the students worked on the assignments after the

game. While some students neatly made every assignment, other students had more problems staying focused. Differences between

students were also noticed during the EduGIS lessons. While some students followed the assignments step-by-step as intended, others

deviated from the planned route and explored the digital map by themselves. At the end of each lesson, about half of the students had

completed all assignments. The teachers argued that this was quite normal, and that similar within-group differences regarding students’

engagement in the tasks were noticed in the control group.

The pre-test data showed a slightly higher average score for the experimental group (X ¼ 31.8, on a 0 to 100 scale), than for the control

group (X ¼ 31.3). The Effect Size (ES) was just t0.03, and a t-test showed that this difference was not significant (p ¼ 0.29). Also, there were

no differences between boys and girls on the pre-test scores (p ¼ 0.55, ES ¼ 0.00). However, the VWO classes (Pre-university Secondary

Education) scored significantly higher on the pre-test than the HAVO classes (General Secondary Education), in both the experimental

condition (p ¼ 0.0009, ES ¼ t0.43), as well as the control condition (p ¼ 0.0007, ES ¼ t0.41). As both conditions had the same number of

VWO classes and HAVO classes, the difference between the two types of classes did not affect the outcomes of the study.

Analysis of the pre-test and post-test data showed that the lesson series with geospatial technologies had positive effects on students’

achievement. In the experimental group, the average scores changed from 31.8 (on a 0 to 100 scale) in the pre-test, to 38.7 in the post-test

(Table 3). The overall effect size was t0.38. This falls in the ‘low to moderate practical significance’ range according to Cohen’s (1988)

convention for interpreting effect sizes. The effect size was higher for the lower complexity assignments (on average t0.52) than for the

higher complexity assignments (on average t0.25). Students especially improved their scores on task 5, which was a lower complexity

Table 3

Average standardised pre-test scores (X1), post-test scores (X2), and Cohen’s D (effect size) for the experimental condition: the lesson series with geo-ICT (N ¼ 139).

Item Skill Com-plexity Pre-test score Post-test score D

X1 Stdev1 X2 Stdev2

1 Identifying non-spatial relationships in representations Low 38.3 23.3 47.4 23.3 t0.39

2 Identifying geospatial relationships in geospatial representations High 11.3 11.5 15.0 12.9 t0.31

3 Organizing geospatial relationships Low 62.9 18.1 66.8 18.3 t0.22

4 Organizing geospatial relationships High 28.1 19.0 32.5 19.9 t0.23

5 Evaluating measures and creating solutions Low 37.1 16.3 53.4 18.2 t0.94

6 Evaluating measures and creating solutions High 13.1 17.4 17.1 20.4 t0.21

Total 31.8 17.6 38.7 18.9 t0.38

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 232

design task that assessed skills in the category ‘evaluating measures and creating solutions’ (d ¼ t0.94). For the other test items, the effect

size ranged between t0.21 and 0.39. The effect size was slightly higher (ES ¼ t0.05) for boys (N ¼ 71) than for girls (N ¼ 68), and slightly

higher (ES ¼ t0.04) for VWO classes (N ¼ 83) than for HAVO classes (N ¼ 56). However, these effects were not significant (p < 0.01).

In contrast to the lesson series with geospatial technologies, the contribution of the conventional lesson series to students’ relational

geographic thinking was minimal: the overall effect size was just t0.04 (Table 4). However, the data did show a small learning effect for the

lower complexity CVS task with the paper planes (ES ¼ t0.24), which suggest a learning effect from the test. The data of the control group

also showed a small learning effect for the lower complexity design task in which students had to evaluate measures and create a solution

for water-related challenges in a fictive delta (ES ¼ t0.20), although the learning effect was much smaller than in the experimental group.

The small learning effect does indicate that students had learnt about the functioning of the water system in deltas from the lesson series

with the schoolbook. However, the data showed no learning effects for higher complexity relational thinking tasks (ES ¼ ?0.01 to ?0.08).

A t-test conducted on the change scores showed that there was a statistically significant difference between the experimental group and

the control group (p < 0.01). The difference in effect size between the experimental group and the control group was t0.34. This suggests

that the lesson series with geospatial technologies contributed more to the development of geospatial relational thinking than the con-

ventional lesson series.

Not only did the lesson series with geospatial technologies result in higher achievement, but students were also more positive about the

effects on the learning outcomes. The survey data showed that students’ perceptions about their knowledge about the water system

increased in both conditions (Tables 5 and 6), but that the effect size was higher for the students in the experimental group (d ¼ t0.58 to

t0.68) than for students in the control group (d ¼ t0.26 to t0.49). Although the two lesson series covered the same content, students who

followed the lesson series with geospatial technologies had the feeling that they constructed more knowledge than students who followed

the conventional lesson series. The teachers were also more positive about the lesson series. Four of the six teachers who participated in the

research agreed that the learning goals and design of the lesson series with geospatial technologies were more closely aligned with the key

tenets of modern geography education than the conventional lesson series (Table 7).

7. Conclusion and discussion

The findings of this study suggest that geographyeducationwith geospatial technologies can have positive effects on students’ geospatial

relational thinking. Students who followed a lesson series about water-related spatial planning issues with the Watermanager and EduGIS

showed higher achievement on a geospatial relational thinking test than students who followed a conventional lesson series with a school

book that focused on the same content. The effect size of the experimental condition (N ¼ 139) was 0.34 higher than the effect size of the

control condition (N ¼ 148). This indicates a ‘low to moderate’ positive effect, which is quite promising, given the fact that the lesson series

consisted of only three lessons of 45 min. Students who followed the lesson series with geospatial technologies were also more positive

about the effects on the learning outcomes. This is good news for geography teachers who are thinking about introducing geospatial

technologies in their classes, as the study suggests that at least some of the learning goals of geography education can be reached by using

geospatial technologies.

Indeed, it mayseem obvious that the post-test scores were higher for students in the experimental group than for students in the control

group, as the lesson series with geospatial technologies paid more attention to systematic geospatial relational thinking. This is because it is

difficult to develop lesson series for control groups and experimental groups that meet all of the following criteria: (1) the lesson series of

the experimental group make use of the benefits of geospatial technologies as much as possible; (2) the lesson series of both test groups are

realistic for the educational context; and (3) the lesson series of both test groups are comparable regarding the knowledge and skills that are

trained. In this research, it turned out to be impossible to develop realistic analogue versions of the tasks in which students used the

interactive technologies. Therefore, wedecided touse an existing schoolbook lesson series in the control groupthat coveredthe same issues.

Still, this study showed that the lesson series with the Watermanager and EduGIS provided only a first step in realising deep geospatial

relational thinking. In the experimental group, the average pre-test scores increased from 31.7 points to 38.7 points, which is still far below

the maximum possible score of 100 points. Students identified only a part of the relations in the representations. Also, they were able to

structure their knowledge about geospatial systems to a limited extent, and they took only a part of the relevant factors into account when

theyevaluated measures and sought solutions for spatial challenges. Why did their skills not increase more? First of all, geospatial relational

thinking is a complex way of thinking. Research by Assaraf and Orion (2004) on earth systems thinking has shown that earth systems

thinking skills can be taught, but that it takes a considerable amount of time: several tens of lessons. The same probablyapplies togeospatial

relational thinking. Although it takes a considerable amount of time, we think that geospatial relational thinking is a goal worth pursuing, as

it is such an important way of thinking for studying the challenges in the world around us. The second explanation for the lower-than-

desired learning outcomes might be related to the design of the lesson series. During the tests, it seemed as if the tasks worked out well

for a large part of the students, but not for all students. Some students followed the tasks better than other students. It seems likely that the

Table 4

Average standardised pre-test scores (X1), post-test scores (X2), and Cohen’s D (effect size) for the control condition: the conventional lesson series (N ¼ 148).

Item Skill Com-plexity Pre-test score Post-test score D

X1 Stdev1 X2 Stdev2

1 Identifying non-spatial relationships in representations Low 34.3 20.1 39.3 21.9 t0.24

2 Identifying geospatial relationships in geospatial representations High 10.3 9.7 10.2 10.0 ?0.01

3 Organizing geospatial relationships Low 63.5 19.5 60.6 19.9 ?0.15

4 Organizing geospatial relationships High 28.9 18.2 27.9 17.8 ?0.08

5 Evaluating measures and creating solutions Low 37.0 15.4 40.6 19.6 t0.20

6 Evaluating measures and creating solutions High 13.4 17.9 12.6 15.9 ?0.05

Total 31.3 16.8 31.9 17.5 t0.04

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 233

effectivenessdepends on students’ learning style. If this is the case, wemight need toredesign the tasks, or to differentiatethe tasks, in order

to optimize the learning outcomes. However, little is known about how students learn when they work on lesson series with geospatial

technologies that aim to stimulate geospatial relational thinking. As far as we know, no process-oriented research has been conducted that

connects learning styles, design of lesson series with geospatial technologies, and learning outcomes for particular geographic thinking

skills. Therefore, we recommend investigating this issue further.

The potential of geospatial technologies has beenwidely recognised in the literature (e.g. Baker & White, 2003; Hall-Wallace & McAuliffe,

2002; Kerski, 2003; Milson et al., 2012; Sinton & Lund, 2007; West, 2003), but concrete evidence is still needed, especially when it concerns

the effects on thinking skills, such as geospatial relational thinking. This study adds to the knowledge body about the effects of geography

educationwith geospatial technologies. The difficulty with this kind of research is, however, that higherachievement cannot be attributed to

the geospatial technologies alone. It is not the technology itself that produces learning, but the complex whole of clear and appropriate

learning goals, solid educational technologies, well-designed tasks, and high-quality instruction, coaching, and reflection provided by the

teacher. In this research, we have tried to optimize the output of the lesson series with geospatial technologies in several ways. First, we set

clear learning goals: relational geographic thinking. Second, we chose two kinds of interactive technologies that are especially suited for

exploring relations. Third, we designed tasks based on a theoretical framework for relational geographic thinking, in which students were

stimulated to make relations explicit, both verbally (in the form of generalizations and rules) and visually (in the form of a conceptual

framework). Fourth, and finally, wetried tostimulate teachers toset up an evaluative classroom discussion at the end of the lessons inwhich

students had to summarise what they had learnt regarding the content and problem-solving strategy, although the degree into which they

actually did so varied greatly among the teachers who participated in the project. As all components (learning goals, software, tasks,

coaching, etc) are connected to each other, it would not make sense to try to attribute the positive learning effect to these components

individually.

Lesson series without geospatial technologies that meet the same criteria (clear and appropriate learning goals, well-designed tasks, and

high-quality instruction, coaching, and reflection) can also result in higher learning outcomes. However, without technologies such as the

Watermanager and EduGIS, it would be much more difficult to develop a good lesson series about water-related challenges that aims to

stimulate geospatial relational thinking. As the applications provide access to huge amounts of digital geographic information and contain

interactive tools for investigating geographic relations, students can explore water-related problems deeply, in a way that is not possible

with analogue materials. We therefore underline Palladino and Goodchild’s (1993) argument that geospatial technologies make it easier to

engage students in ‘higher order thinking activities that are often so hard to come up with’, and we think that geospatial technologies hold

many possibilities to make a step towards education that focuses on deep learning. Unfortunately, in many countries, geography education

focuses on learning facts and concepts and can be described as an ocean wide but just a few centimetres deep. Today, inquiry projects and

systems thinking do not get as much attention as they should. Like Hattie (2009, p. 28), we think that a balance of surface and deep learning

helps students to construct defensible theories of knowing and reality more successfully.

Table 5

Average values for the survey question: ‘How do you perceive your knowledge about the water system in the Delta Region and River Region (on a 1 to 5 scale)?’ in the

experimental condition.

Knowledge about the: Pre-test score Post-test score

N X1 Stdev X2 Stdev d

characteristics of the water system 139 2.9 1.1 3.6 0.9 t0.68

functioning of the water system 139 2.7 1.1 3.4 1.0 t0.65

problems of the water system 139 2.8 1.0 3.3 1.0 t0.58

the water system (total) 139 2.8 1.0 3.3 1.0 t0.64

Table 6

Average values for the survey question: ‘How do you perceive your knowledge about the water system in the Delta Region and River Region (on a 1 to 5 scale)?’ in the control

condition.

Knowledge about the: Pre-test score Post-test score

N X1 Stdev X2 Stdev d

characteristics of the water system 148 2.9 1.2 3.2 1.2 t0.26

functioning of the water system 148 2.7 1.2 3.2 1.1 t0.49

problems of the water system 148 2.9 1.1 3.2 1.1 t0.26

the water system (total) 148 2.8 1.0 3.3 1.0 t0.34

Table 7

Scores for the survey question ‘Compare the lesson series with geo-ICT with the lesson series with the

school book. How do the learning goals of the lesson series fit with the key tenets of modern ge-

ography education?’

Frequency

The lesson series with the school book fits better 0

The lesson series with the school book fits slightly better 0

The two lesson series fit equally 2

The lesson series with geo-ICT fits slightly better 3

The lesson series with geo-ICT fits better 1

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 234

Nevertheless, using geospatial technologies is not always easy for students. The huge amount of geographic information and the

interactive tools can only be an advantage for students whose actions are guided by good research questions. Also, students have to be able

to investigate the challenges in the world around us in a systematic way, and be able to use the tools in a sensible way. Geography education

with geospatial technologies should therefore follow a step-by-step approach, giving attention to the development of subject knowledge,

inquiry skills, thinking skills and motivation. Teachers should provide feedback on the learning process, and organize whole class discus-

sions on the content (Favier & Van der Schee, 2012).

The introduction of geospatial technologies in classrooms often requires extra effort for teachers (Kerski, 2003). However, the lesson

series that was tested in this study did not require many technical skills, as simple applications were used. The teachers who participated in

the study were able to use the applications without any technical training. However, in order to provide good feedback on the learning

process, and in order to organize a good whole-class discussion at the end of the lessons, the teachers did need a considerable amount of

knowledge about the water systems in the two regions, and did need to be able to help students structure their geographic thinking. So, in

order to conduct the lesson series with The Water Manager and EduGIS in an optimal way, teachers especially need to increase their Content

Knowledge and Pedagogical Content Knowledge, and not so much their Technological Knowledge and Technological Content Knowledge. As

it was the first time that the group of teachers had used these applications and coached students in undertaking tasks that aim to structure

students’ geospatial relational thinking, it is no surprise that some teachers were better able to provide good feedback on the learning

process and organize a good whole-class discussion than others. It would therefore be worthwhile to repeat the experiment with more

experienced teachers, and seewhat the effect is. As the lesson series with geospatial technologieswasveryshort, it coveredonlya partof the

wide range of geospatial relational thinking skills, and did not practice these skills extensively. Furthermore, the lesson series could make

use of only a part of the several advantages of geospatial technologies for stimulating students’ geospatial relational thinking. Nevertheless,

the results are very promising, and we recommend that this issue should be investigated further in a large-scale test, with longer lesson

series, perhaps spread out over several years, that cover the entire range of geospatial relational thinking skills and practice these skills

extensively, therebymaking use of everyadvantage provided bygeospatial technologies. Also, following Lee and Bednarz (2009), wesuggest

that much more research should be done on the learning effects of geography instruction on other components of geospatial thinking, and

on how proficiency the different components of spatial thinking are related to teach other. Therefore, this study al least justifies the further

pursuit of research in this field.

Acknowledgements

The authors of this paper would like to thank Anneke Kleeven, research assistant, for coding 600 tests.

References

Aladag, E. (2009). The opinions of Turkish social studies pre-service teachers about using GIS in primary education. Learning with Geo-Information, VI, 39–49.

Albert, W. S., & Golledge, R. G. (1999). The use of spatial cognitive abilities in geographic information systems: the map overlay operation. Transactions in GIS, 3(1), 7–21.

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of educational objectives. Boston, MA: Allyn & Bacon.

Assaraf, O. B. Z., & Orion, N. (2004). Development of systems thinking skills in the context of earth system education. Journal of Research in Science Teaching, 42(5), 518–560.

Audet, R. H., & Ludwig, G. S. (2000). GIS in schools. Redlands, CA: ESRI Press.

Baker, R. T., & White, S. H. (2003). The effects of GIS on students’ attitudes, self-efficacy and achievement in middle school science classrooms. Journal of Geography, 102, 243–

254.

Bednarz, S. W. (2004). Geographic information systems: a tool to support geography and environmental education? GeoJournal, 60, 191–199.

Bednarz, S. W., & Van der Schee, J. A. (2006). Europe and the United States: the implementation geographical information systems in secondary education in two contexts.Technology, Pedagogy and Education, 15, 191–205.

Bodzin, A. M., & Anastasio, D. (2006). Using web-based GIS for earth and environmental systems education. Journal of Geoscience Education, 54(3), 295–300.

Chen, Z., & Klahr, D. (1999). All other things being equal: acquisition and transfer of the control of variables strategy. Child Development, 70(5), 1098–1120.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

Cremer, P., Richter, B., & Schäfer, D. (2004). GIS im Geographieunterricht – Einführung und überblick. Praxis Geographie, 2, 4–7.

Demirci, A., Karaburun, A., & ünlü, M. (2013). Implementation and effectiveness of GIS-based projects in secondary schools. Journal of Geography, 112(5), 214–228.

Doering, A., & Veletsianos, G. (2007). An investigation of the use of real-time, authentic geospatial data in the K12 Classroom. Journal of Geography, 106, 217–225.

Favier, T. T., & Van der Schee, J. A. (2012). Exploring the characteristics of an optimal design for inquiry-based geography education with geographic information systems.Computers and Education, 58, 666–667.

Gilmartin, P. P., & Patton, J. C. (1984). Comparing the sexes on spatial abilities: map-use skills. Annals of the Association of American Geographers, 74(4), 605–619.

Goldstein, D., & Alibrandi, M. (2013). Integrating GIS in the middle school curriculum: impacts on diverse students' standardized test scores. Journal of Geography, 112(2), 68–74.

Golledge, R., Marsh, M., & Battersby, S. (2007). Geospatial concept understanding and recognition in G6-college students: a preliminary argument for minimal GIS. Annals ofthe Association of American Geographers, 98(2), 285–308.

Golledge, R. G., & Stimson, R. J. (1997). Spatial behavior: A geographic perspective. New York: Guilford Press.

Hainey, T., Connolly, T. M., Stansfield, M. H., & Boyle, E. A. (2011). Evaluation of games to teach requirements collection and analysis in software engineering at tertiaryeducation level. Computers and Education, 56(1), 21–35.

Hall-Wallace, M. K., & McAuliffe, C. M. (2002). Design, implementation, and evaluation of GIS-based learning materials in an introductory geoscience course. Journal of

Geoscience Education, 50, 1–14.

Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.

Huynh, N. T., & Sharpe, B. (2013). An assessment instrument to measure geospatial thinking expertise. Journal of Geography, 112, 3–17.

Hwang, G.-J., Yang, L.-H., & Wang, S.-Y. (2013). A concept map-embedded educational computer game for improving students’ learning performance in natural science

courses. Computers and Education, 69, 121–130.

Kali, Y., Orion, N., & Elon, B. (2003). The effect of knowledge integration activities on students’ perception of the earth’s crust as a cyclic system. Journal of Research in Science

Teaching, 40, 545–565.

Kerski, J. J. (2003). The implementation and effectiveness of geographic information systems technology and methods in secondary education. Journal of Geography,102,128–

137.

Kerski, J. J. (2008). The role of GIS in Digital Earth education. International Journal of Digital Earth, 1(4), 326–346.

Kim, D. H. (1990). A palette of systems thinking tools. The Systems Thinker, 1(3), 5–6.

Kim, M., Kim, K., & Lee, S.-I. (2013). Pedagogical potential of a web-based GIS application for migration data: a preliminary investigation in the context of South Korea. Journal

of Geography, 112, 97–107.

KNAG. (2003). Gebieden in perspectief: natuur en samenleving, nabij en veraf. Utrecht, Nederland: KNAG.

Lam, C. C., Lai, E., & Wong, J. (2009). Implementation of geographic information systems (GIS) in the secondary geography curriculum in Hong Kong: current situations and

future directions. International Research in Geographicand Environmental Education, 18, 57–74.

T.T. Favier, J.A. van der Schee / Computers & Education 76 (2014) 225–236 235

Lee, J., & Bednarz, R. (2009). Effect of GIS learning on spatial thinking. Journal of Geography in Higher Education, 33(2), 183–198.

Lohman, D. F. (1979). Spatial ability: A review and reanalysis of the correlational literature. Stanford, CA: School of Education, Stanford University.

Mayer, V. J. (1995). Using the earth system for integrating the science curriculum. Science Education, 79, 375–391.

McGee, M. G. (1979). Human spatial abilities: psychometric studies and environmental, genetic, hormonal, and neurological influences. Psychological Bulletin, 86(5), 889–918.

Milson, A. J., Demirci, A., & Kerski, J. J. (2012). International perspectives on teaching and learning with GIS in secondary schools. New York, NY: Springer.

Milson, A. J., & Earle, B. D. (2007). Internet-based GIS in an inductive learning environment: a case study of ninth-grade geography students. Journal of Geography,106(6), 227–

237.

Milson, A. J., & Kerski, J. J. (2012). Around the world with geospatial technologies. Social Education, 76(2), 105–108.

Montello, D. R., Lovelace, L. L., Golledge, R. G., & Self, C. M. (1999). Sex-related differences and similarities in geographic and environmental spatial abilities. Annals of the

Association of American Geographers, 89, 515–534.

National Research Council. (2006). Learning to think spatially: GIS as a support system in the K–12 curriculum. Washington, D.C.: National Academy Press.

Nielsen, C. P., Oberle, A., & Sugumaran, R. (2011). Implementing a high school level geospatial technologies and spatial thinking course. Journal of Geography, 110(2), 60–69.

Orion, N. (2002). An earth systems curriculum development model. In V. J. Mayer (Ed.), Global science literacy (pp. 159–168). Dordrecht, The Netherlands: Kluwer Academic.

Palladino, S. D., & Goodchild, M. F. (1993). A place for GIS in the secondary schools? Geo Info Systems, 3(4), 45–49.

Pattison, W. D. (1964). The four traditions of geography. Journal of Geography, 63, 211–216.

Rosling, H. A. (2007). Visual technology unveils the beauty of statistics and swaps policy from dissemination to access. Statistical Journal of the IAOS, 24, 1–2.

SchoolVO. (2013). www.schoolvo.nl.

Self, C. M., & Golledge, R. (1994). Sex-related differences in spatial ability: what every geography educator should know. Journal of Geography, 93(5), 234–243.

Shin, E. K. (2006). Using Geographic Information Systems (GIS) to improve fourth graders’ geographic content knowledge and map skills. Journal of Geography, 105, 109–120.

Sinton, D. S., & Lund, J. J. (2007). Understanding place. Redlands, CA: ESRI Press.

Van der Schee, J. A. (2000). Helping children to analyse a changing world: looking for patterns and relationships in space. In M. Robertson, & R. Gerber (Eds.), The child’s world:

Triggers for learning (pp. 214–231). Melbourne, Australia: Australian Council for Educational Research.

Van der Schee, J. A., Van Dijk, H., & Van Westrhenen, J. (1992). Geographic procedural knowledge and map skills. In H. Schrettenbrunner, & J. van Westrhenen (Eds.), Empirical

research and geography teaching, NSG 142. Utrecht, The Netherlands: KNAG.

West, B. A. (2003). Student attitudes and the impact of GIS on thinking skills and motivation. Journal of Geography, 102, 267–274.

Yano, K. (2000). GIS and quantitative geography. GeoJournal, 52(3), 173–180.

Zink, R., & Scheffer, J. (2009). GIS auf neuen Wegen in das Klassenzimmer? Learning with Geo-information, VI, 121–130