Abstract

Many faculty are using web course building tools that use a database to store information and track students. The courses could be improved if faculty were able to understand the underlying structure of the information in their course and use this understanding to build layers of support for student learning. There are six steps to modify a course; 1) specifiy the learning objectives, 2) build a concept map, 3) describe the learning activities, 4) describe the cognitive processes, 5) build assessments of both knowledge and process, and 6) design tests that reflect the importance of the process of understanding. The layers of support and teaching strategies suggest multiple formats and timing of student learning that fits best with flexible format delivery (partial face-to-face and partially mediated via technology). Some central concepts that students will see repeatedly are best addressed with reusable learning objects. These digital learning objects can be introduced from different assignment and lead to separate assessment. The overall goal of using technology in this manner is to substantially shift the relationship between the student and information, helping the student become a thoughtful and deliberate user and producer of information and ideas.

1. Introduction

Because of both the demands for technology skills and the benefits of using technology to help students learn, we are rushing to build and deliver web course material. We are mostly using web course tools to contruct these resources. These tools have powerful functions and are built on top of a database. These tools are relatively easy to use and help us meet the present demands. The problem is that many of us end up teaching a course that is being delivered via some type of database and yet we don't understand the basis for using a database for this purpose. More than that, we don't understand how information in our course is structured and therefore would have trouble putting that course in an intentional way. This paper describes how we should approach structuring the information in a course so that it can improve teaching and learning.

There are three major reasons for using structured information in teaching:

1. The goal is toward a learning environment that changes the relationship between students and information.

2. As teachers, we should construct layers of support for learning in between discipline oriented information domain and the level of the student.

3. To make this effort feasible in the short term and extendable, we need to be able to reuse resources that you have made or are available elsehere.

Each of these points will be developed below, followed by an explanation of how to get started with this approach.

2. Constructing Support for Student Learning

Whether we are planning lectures, assignments, web-pages or email interactions; we are trying to make conditons to support learning. There are many ways of helping students learn and almost as many ways to plan. I have described one such method of planning that not only meets the current demand of students, but also in a way that will help build resources that can provide the basis for future improvements of your course.

There are six steps:

1. specify the learning objectives or other student outcomes
2. relate these outcomes in a concept map
3. describe the activities that students will perform to understand the material
4. describe the steps that students can grow through to learn this information
5. assess the student performance in progress
6. design tests for overall performance

I will descibe all of these steps in the context of a course that has units or modules that are about two weeks of class (4 credit hours per week).

2.1 Specify Learning Objectives

You should begin by specifying the learning objectives for the unit. Of course these will vary by the type, level and intensity of the course. In a course for non-majors in Biology, I will specify between 20 and 30 vocabulary words, approximately 10 concepts that need to be described and specify the types of applications and problem solving approaches that I expect them to be able to perform at the end of each two week unit. By comparison, for a 300 level majors course, I specify over 50 specific learning objectives for each two week unit that are above the level of vocabulary. The course has general problem solving, numercy and information literacy objectives that apply to all units. The number of specific learning objectives does not determine the difficulty of the course. For many students in the 100 level course, those are 20 completely new vocabulary words and 10 new concepts that they have never had to describe exactly before. This entirely new context may make the course more difficult for those beginning students than a course for upper division students who are essentially seeing an extension of concepts in a well defined context.

The student outcome objectives should be classified in some scheme that helps connect these objectives to a larger context for learning. For learning objectives, I have used Bloom's taxonomy of cognitive domains (knowledge, concepts, application, analysis, synthesis and evaluation).Vocabulary is at the lower end, simple knowledge where as some forms of problem solving require analysis, synthesis and sometimes evaluation. These classifications or hierarchies will greatly aid in understanding what strategies might be appropriate for students to meet those objectives.

2.2 Build a Concept Map

These individual vocabulary words and concepts can be related by use of a concept map. The items in the concept map and their arrangement may be derived from your own discipline based view of the structure of information in the teaching unit. Figure 1 gives an example of a concept map that is organized around dispcipline based understanding, i.e. a "Biology discipline" view of the information rather than a student's view of the information.

Figure 1. Concept map for a unit on food webs and predator-prey interactions. This map is shown without links between the individual concepts. Those will be developed later.

2.3 Describe the Learning Activities

The links between the items in the unit should be based on how students would move from one concept, and build on that knowledge to include more concepts and information. I have seen some concept maps in which every concept is related to every other concept, this approach is not particularly useful, unless it has an order or a weighting system applied to the links. Table 1 provides an example of cognitive operations that a student might have to perform to connect one or more items in the map with other items in the map.

2.4 Describe the Steps

Taken together, the cognitive domain of each learning objective and the layout of the map should suggest some pathways for students to build their understanding of the information in the unit. You can visualize these paths as layers in their learning of the material, starting with just a few pieces of information on layer 1 and progressively building new layers until the entire map has been constructed. Figure 2 provides an illustration of the first two layers of such a process for the concept map given in figure 1.

Figure 2. Layers that build toward a concept map. Layer 1 was association and recall from their general knowledge that coyotes eat deer and that deer eat grass. Layer 2 is attaching biological concept names to those organisms and introducing the concept of decomposer. These are all linked to the description of a community and a simple figure that shows nutrient cycling and energy flow in the book.

You may need to develop multiple layers of builds to support the student's understanding. Each layers indicate a particular activitity that you will be doing with your students or having them perform on their own. Table 2 develops this example further, showing multiple layers with the information items that become included in the building process, what cognitive process was employed and what activites were designed.

It is important to note that these layers build from what students know (from their background understanding or recall) to simple levels to more sophisticated levels. The entire concept map (which represents the discipline view of the structure of that information) is not presented all at once. This is the essence of the difference between the students view and the discipline view of the structure of information. Discipline-based concept maps focus on the general concepts and then "drill in" to see the details, whereas the layers of learning start with the specifics and build to the more general.

Table 2. Layers that could be used to build (from the bottom up) an understanding of food web dynamics. The right most column indicates activites that might be best done outside of a normal class time constraint.

Figure 3. A simple schema for energy flow and nutrient cycling in a community.

The order and time required for each layer becomes a major constraint if all the processes are going to happen in class. Using the Web in combination with lecture and, more specifically, flexible format courses have a distinct advantage in structuring these activities. For example, some activities need to preceed class discussions and other activities might be more valuable as a follow-up to a class discussion. Flexible format courses are sometimes much more structured than their traditional counterparts. The students are guided through a series of activities in a particular manner and time sequence. The flexibility with regard to the time schedule allows optimization of the learning processes.

2.5 Assess Student Knowledge and Understanding

Based on this framework, it should be easy to see how you could build assesments to test both the knowledge of the students but also their ability to perform a subset of these cognitive operations.

The class might have been designed to hit several points in layer 1, then build that into a concept in layer 2, extend that concept in layer 3 and then find a counter example in the final layer. This example path is shown in red dots and paths in Figure 3. The point of this path is not to be exhaustive, but to model for students how information is related in this unit.

You could choose to assess whether students are ready to undertake a similar path on their own. You could design a short assessment of whether they can recall the information represented by the two green triangles in layer 1. If they can't do that, they won't be able to build on that.

You can also assess if students can perform a similar set of knowledge building operations. You can supply them with information and see if they can go through the same concept forming processes and exceptions. The closer the information that you give them to the example you used, the easier it will be. You can make the assessment more difficult by eitehr making the information more different or by addign extraneous concepts that they need to evaluate first. This type of assessment is looking at student performance and their ability to use the process, not just the facts.

Figure 3. Assessments of information items or paths. The path of the class is shown in red and possible points for assessment shown in green.

2.6 Design Graded Exercises

The same approach that was used for assessments can be applied to grading methods. If you want your students to learn how to solve problems and build their understanding of the information, your exams and grading should reflect that. There are some issues with time and security that need to be addressed.

One way to really see how the students perform is to watch them solve problems. Laurillard (ref) calls this the phenomenographic approach and an important part of this effort is to look for misconceptions that the students hold that will interfere with their further progress. Another way to do this is to have students perform tasks on the tests or quizzes. Their performance of closely and loosely associcated problems will help you understand whta they understand. This could be combined with making the students explain how they got particular answers and, if they weren't able to solve the problem, where they got stuck. This approach also will help students develop metacognition skills for self analysis (ref Nickerson?). These approaches are different than simple answer questions and are more difficult to grade (both in terms of being objective and in terms of the amount of time that they will take). It will be important to construct multiple choice or short answer questions that probe for knowledge at higher cognitive levels (such as in Bloom's taxonomy) and performance of tasks that show that they understand the material.

I have used several approaches with varying success. In a small class taught in a studio environment (ample computers and desk space), I gave the students "challenges" that built upon the course material and I watched them solve these problems. It was instructive to see which students would open Excel when the problem obviously called for multiple calculations (and which students didn't). Similarly, I could observe which students started answering a question by outlining their answer and which ones started flipping through their course notes. In a larger course (60 to 80) students, I scheduled a portion of the students for interviews just before the quiz. Each student was given a piece of paper with the problem on it and I watched as they solved the problem. It took each student about 15 to 20 minutes for this process. From this type of assessment I was able to learn quite about how they solve problems and where they get stuck. After each set of interview, I posted my lumped comments to a web page and discussed them with the class so that all students got some benefit from the interviews. In a large course (> 100 students) none of these approaches seemed feasible so I restructured the entire course to put the stories that I normally would have told in class onto web pages, and scheduled a face-to-face discussion session with groups of 20 students each. The flexible format of this course was scheduled to help students to be more prepared for these discussions.

3. Building Learning Objects that are Reusable

Learning Objects are any artifact that we might construct or use to support student learning (ref - ). After looking at the structure of the information in the unit of interest or the course as a whole we might want to help students learn some key concepts or themes that are threaded through the entire course. For example, in General Biology the concepts associated with metabolism come up in almost all of the units. Each time it comes up we could refer back to previous reading or lecture notes, or we could create a set of web pages that helps students learn these concepts and then refer to this learning object from different perspectives. Digital information technology (and in particular the internet) allows us to create learning object documents which have three important qualitites; 1) they can contain metadata which describes the subject area, teaching/learning approach, appropriate audience as well as other characteristics, 2) the same document can have multiple respresentations, depending on the context of its use, 3) the document can be reused easily in another context.

3.1 The Value of Metadata

Negroponte (ref) calls metadata "bits on bits", the ability of a document to carry information about itself. Just as it is useful for an audio CD to carry information on the number and length each track, it would be useful for all digitial documents to include more complete information - within the document, not as a separate document. The Dublin Core is a set of information that has been agreed would be minimal. Tables 3 and 4 show the core information that I use to describe web pages and graphics respectively.

Table 3. Metadata for this document. It should indicate the version or draft status and any upgrades. The "relation" element should be a link to the full document. This iinformation should be included in the meta tag in the header of the html (but it isn't).

The description element in the graphic's metadata allows me to include a full description of the image that may be helpful both for sight-impaired uses and for searching for images in my collection.

Metadata standards are being developed by IMS and Educause. There are two areas where these standards will help us; subject area and teaching/learning approach. In my web pages that deal with biology I use subject area descriptors that I have derived from Biosis. I am not trying to put each element into a specified hierarchy (such as you would get from the Library of Congress Subject Headings) but rather that the key terms in my metadata subject element will allow it to be found with search strategies. The IMS is also developing Learning Object Metadata that provides elements that describe the target audience, level, time for the exercise and several other elements. A major goal of IMS is to provide a large selection of learning objects that can be searched and used by teachers and learners, and by doing so create a market for these learning objects. Given that many of these learning objects are being created for industry training and certification programs, the goal to create a market is not out of place. However, for academic use we might want to consider the metaphor of a library for these learning objects so that they can be shared.

3.2 Multiple representations

One of the limitations with our current use of desktop applications is that we think of each document as if it only has one representation, i.e. a MS Word document is a text document, a Dreamweaver document is a web page, a Excel document is a spreadsheet, etc. These applications can present different representations but sometimes it is awkward. For example, each MS Word document shows up with a screen representation, a disk represenation and can be easily printed with a postscript file representation of that same document. With very little messing around, you can "export" a separate html version that has nearly the same fomatting as the printed version. You can also change the style sheets for any particular document to get a different look. With an Adobe application you can create a pdf document of that same page.

Applications that can project multiple representations of each document will become more common, and instead of focusing on the formatting and limiting the potential "clients" to being the screen, disk and printer; we will be able to create documents that carry information about how we want that document displayed in different contexts or to different audiences. For example, you can already make documents that will be viewed differently if they are called from a 3.0 browser compared to a 4.0 or higher browser. We should soon be seeing desktop applications for education that will allow us to display the document differently to different users. These documents will probably be built using metadata and XML (Extensible Markup Language).

The reason that being able to project multiple representations is important is that it will allow us to create one document or one set of information resources that will be context and user sensitive. We could present the same information to freshman and seniors in different ways, or we could present the "senior" level version to a well-prepared freshman. Building on the predator-prey example; a learning resource could contain definitions of the biological terms that the student would automatically see in their first visit to the site, but be hidden in subsequent visits. In order to create these resources however, we will have to structure and describe the information more carefully.

3.3 Reuse and Sharing

Creating resources to support learning is very time consuming. We should be using technology, support structures and social interactions of some sort to help us get the maximum value from the resources that already exist and those we are investing our time creating. When I build a website for an individual course, it contains hundreds of documents in different formats (graphics, text and spreadsheet). I have found that I don't have a plan for reusing components that I have created, let alone being able to find resources that someone else created. It is often easier for me to create a new picture instead of trying to find one that I made before. I blame part of this inability on the technology for authoring that doesn't include (at this time) tools for categorizing and searching. I am told that the tools exist and will move to incorporate those in the near future.

If I have this problem with my own material, it is totally understandable that I haven't incorporated elements from other authors into my websites. Unlike the familiar textbooks and journal articles, websites are difficult for me to scan, explore and incorporate only certain portions.

This probelm (of the lack of reusability) is being addressed by the IMS. Their goal is to provide a standard set of metadata elements that would describe the content and learning attributes of these learning resources. Using these proscribed (?) elements will help me learn to search the growing body of resources.

Just having metadata on a learning object doesn't necessarily mean that everybody will want to use that in the same manner. Systems are being developed that would allow you to learn how other people used a particular learning object and to keep track of how you used it. For example, the author might categorize the learning object as suitable for sophomore level, biology major work, but you find that it works well in a non-majors biology class as long as it is preceded by chemistry. (Dietinger et al. 1999)

All of these larger projects are based on the ability to transfer leaning objects and thus involve issues of intellectual property rights. Part of the goal of IMS is to create a market for these objects and allow people to incorporate these into their curriculum. I am assuming that this will include some compensation or reciprocity. One form of reciprocity that is currently being used is when your institution joins a developers' group, pays dues to support that group and in doing so has access to resources and tools created by the group. To participate in a collaboration, individuals need to have a clear understanding of whether they can share or sell the materials and tools they create. This issue is surrounded by a great deal of uncertainty in the university environment.

4. Changing the Relationship Between Students and Information

We have previously described levels of students' ability to work with technology and related these skills to the process of assessment (See Perrin and Rueter, Title3 project, The Assessment & Information Literacy Process, http://web.pdx.edu/~rueterj/ail/ail_process.htm).

"information competency" - as a minimum set of defined competencies in using software applications

"information literacy" - the ability to use information technology within the curriculum

"information fluency" - the ability to choose which technology and information skills to use, and the the reliance on these skills without prompting

We have used a variety of descriptions for learning outcomes such as Blooms, cognitive flexibility, and cognitive complexity to help think more clearly about the use of technology in both teaching/learning strategies and assessment (Perrin and Rueter - LLOTA). These outcome measures are the first step in recognizing what we want our students to be able to do. The next, and very important step, is that we need to find a way to have students take action. Our students need to understand the information and then take action. They need to produce information and share that with each other. We talk about having a "course on the web", but what we should really be aiming for is to have our "students on the web". They are already there in many ways; searching, using chats, sharing music with Napster, and other social and asocial behaviors.

The goal for our students should be that they be active participants in creating, sharing, and modifying larning resources and that they use these objects as part of their mode of communication. For example, when a student needs to explain to another student how predator-prey interactions works, they should incorporate figures from the learning object or refer to a particular page in that site.

5. Steps to Get Started

Pick a unit and a concept that would tie to other courses you teach.

Describe the learning objectives and strategies help students understand that information and be able to do something important (creative or authentic).

Create a set of layered resources for understanding that unit and in particular that central concept.

Put extra effort into the central learning object that they can't get in books or through discussion, tie your other resources together with that learning object (find one or create one).

Design a graded exercise that tests how well students understand the material that will be graded on their process and their knowledge.

Put this into your course and prioritize it in your schedule so that you will be able to study and learn from how the students did.

6. Conclusions

Education is often compared to other industries to show how we are lagging behind in transforming ourselves through the use of the internet. I think that these claims are largely unfair. It may be true that we aren't loosing quite as much money per month as some e-businesses or we aren't completely changing our business model to take advantage of the latest opportunities for profit. We are seeing a substantial change in how we are dealing with students. We are restructuring our courses to be more flexible and we are demanding specific learning outcomes both from our students and from our academic programs.

Also to be fair to universities, this is the second phase of our involvement with the internet, the first phase of course was the role that universities played in both the creation of the internet itself and making it and making it accessible through browsers. During our current phase, we are building resources and a growing appreciation for how to use them. Soon we will enter the third phase, where teaching and learning will attempt to change the relationship between people and information. It is starting with the faculty; we are learning how to structure information for both research and teaching/learning uses. Next we will disseminate that to our students. Soon students will be able to join in this effort and be able to find, modify, share and create learning resources. These processes take time, especially if they are done deliberately and in keepin with our core value of scholarship.

7. References

Bourda, Y. & M. Helier. 2000. What Metadata and XML Can Do for Learning Objects. WebNet Journal. January-March 2000. pgs. 24-31.

Dietinger, T., C. Eller, C. Gutl, H. Maurer & M. Pivec. 1999. An Associative Repository for the Administration of Course Modules. Webnet 1999.

Negroponte. N. 1995. Being Digital.

Nickerson, R. S., D. N. Perkins & E.E. Smith. 1985. The Teaching of Thinking. Lawrence Erlbaum Associates.

Perkins, D. N. 1998. What is Understanding? In: S. Wiske, editor. Teaching for Understanding. Jossey-Bass. Chapter 2.