This paper, with active links and in color can be found at:
http://web.pdx.edu/~rueterj/techaward2000/index.htm
Please view online to see supporting documents.

2000-2001 Scholarship of Teaching with Technology Awards

Teaching with Structured Information:
The power and fury of database course design

John Rueter, Professor, ESR
Jo Meyertons, Title III Instructional Designer, Adjunct Faculty, GSE

Introduction

The information, content and skills in our courses have a structure. This structure may not be readily discernible by the instructor; it may be related to the structure of the curriculum, the discipline, or simply passed down from previous instances of that course. Embedded in the structure of a course are the assumptions about how students learn and what efforts instructors will make to help them learn. Increasingly, faculty are offering courses in alternative formats that include web-based resources. Many instructors are moving from one structure for information directly to another structure without ever reflecting on the details. Our work has tried to provide a framework to help faculty think about the structure of information and suggest a path for course revision.

Theoretical Background

As part of a workshop for faculty who are moving parts of courses to web-based delivery (sponsored by the Title III Project at PSU), we made a presentation on how to teach with structured information (1). The two main points in this presentation and accompanying paper were: First, the goal is to move toward a learning environment that changes the relationship between students and information. Second, as teachers, we should construct layers of support for learning between the discipline-oriented information domain and the level of the student.

In order to meet these goals, we designed a process that has six steps.

    1. Specify the learning objectives or other student outcomes
    2. Relate these outcomes in a concept map
    3. Describe the activities that students will perform to understand the material
    4. Describe the steps that students can grow through to learn this information
    5. Assess the students progress during the course
    6. Design tests for overall performance

This process includes having faculty develop concept maps and design paths through the related concepts to build the entire concept map, link by link. Teaching could start with a fully articulated example and move to exercises for the students that have fewer constraints or hints provided by the instructor. This "scaffolding" approach to teaching also is amenable to assessment and evaluation. Parallel or similar pathways through the material can be used as either progress assessments or test questions.

The resulting structure of the course may or may not end up being very different than what the faculty started out with, but the explicit relationships will help develop learning support and strategies for assessment. In addition, the structure allows the instructor to perform a type of causal analysis or back checking to examine the pattern of student performance.


Use of Databases

Electronic databases can be used as a tool to both deliver a course and track students. Portland State's defacto web course building environment, WebCT, has a sophisticated database engine that delivers web pages and tracks students (among many other features). Other database technology can also be used to track student progress and deliver dynamic content to students.

It is our assumption in this work that faculty need to understand how these powerful tools work in order to build congruency between the structure of the database and the structure of the course information. Instructors will have to understand how databases work much better than they do now.

Technology advances have made it possible for instructors to embrace the power of web database. These technologies enable instructors to design a course as a carefully planned framework within which all the course objects are linked to specific learning objectives. This process requires a great deal of effort and thought and, for most instructors, the acquisition of a new set of skills. The instructors need to learn how to use the database, how to create web pages that call scripts that access the database, how to write Cold Fusion scripts (that are very similar to html), and how to keep track of another layer of servers and applications. The technology we employed included: Access database software, ColdFusion server software (middleware), Excel spreadsheet software, email and web pages for student communications, and a face-to-face classroom.

Course Experiences

It's easy to have a theoretical framework but quite another to show how this works with actual courses (or "experience trumps theory"). We collaborated to design two different courses, following our own advice and steps. It was good medicine for us. The Biology 101 Fall term 2000 course was redesigned to keep track of every student answer to every assessment and quiz question. Each unit of the course was scaffolded to build from simple vocabulary and concepts to more cognitively challenging applications of this knowledge. Links were defined between individual questions in the assessments and quizzes such that we could check if students were progressing. These links represented different relationships between the two questions. For example some were the same and others, the information from the first quiz or assessment was assumed to be needed in order to build on for the later question. A database was essential to this enterprise. There were approximately 180 students at one time or another in the course and each of these students did ten assessments, three quizzes of five to nine questions each, and a final quiz with 17 questions.

In addition to using the database for tracking students, responses to their assessment and quiz questions were individualize using Access and Cold Fusion. For example, students would receive a comment back on their assessments that was individualized to their answer and gave them suggestions on how to improve. These comments were given to all students. Even students who submitted a good answer were advised on how they could improve that answer. For more details on how the database and web access to the database was used in this particular course please see reference 2.

In this course, we were able to track individual students with individual questions. This provided us with the power to address the progress of student learning within the course. For example, in the flow of the course, the same question was asked on an assessment as on a subsequent quiz. As can be seen in Table 1., 82 students missed it on the assessment and then continued to miss that same question on the quiz. I made sure that they were aware of what I thought of that effort. Other examples of this causal analysis are given in the that covers the entire course, was how well do students retain information from the first unit to the final and do they review effectively? These two questions can't be separated in my analysis, but they address the same issue.

 
Figure 1. Linkages between assessment and quiz questions.

 

Table 1. A set of queries was done on the database to look for the relationship between the performance on assessment 5 and subsequent performance on Quiz B question 2.There were 152 students who did both questions. The intervals are calculated to include the higher value.

   

A 0-1
Q 0-1

82

A 0-1
Q 1-2

14

A 0-1
Q 2-3

9
 

A 1-2
Q 0-1

15

A 1-2
Q 1-2

16

A 1-2
Q 2-3

5

  

A 2-3
Q 0-1

2

A 2-3
Q 1-2

3

A 2-3
Q 2-3

6

    
did worse on quiz did same
on quiz		 did better on quiz


ESR 202 - Winter 2001 - This course is currently being taught, but the structure of the course and the interaction between students and the course content are indicative of the type of approach that can be facilitated using a database. The course has over 100 specific learning objectives. For a list see this Cold Fusion script that will list all learning objectives in the database. Each student is supposed to demonstrate that they can meet each of these objectives in problem sets, written projects and the laboratory sessions. Each of these exercises is designed to allow them to meet particular objectives, for example see the course schedule with links to Cold Fusion scripts that show the specific learning objectives for each assignment. Each student can also track which of these learning objectives that they have demonstrated through similar Cold Fusion scripts.

The purpose of this course design is to fundamentally change the way students interact with course information. Students will understand how to use many different problem solving approaches. This "understanding" requires at a minimum that they can do the problem when prompted, but more importantly, that they can sense that there is a problem and act to solve that problem without being told "this is a problem about population growth, plug in an exponential function somewhere". By tracking student demonstration of the specific learning objectives, the instructor will know that they have the foundation skills that are required for understanding more difficult problems.

A student's grade in this course will be made up of both the demonstration and the performance of their understanding of many of selected specific learning objectives. The test for understanding will take place in a computer lab with all machines facing the wall. Ten students at a time will be give problems to solve. The instructor will watch the students solve the problems, give hints where necessary, and check off the specific learning objectives that the student was able to use. This testing system has previously been used in another course (Bi445). This action on the students' part is a crucial link between a student and information. Action is the true test of understanding.(5)

Implications for teaching and learning with technology

This approach to course redesign and teaching is very labor intensive. Data entry alone in Bi101 was a tedious task. But the level of work goes beyond that. Each unit had to be mapped, each lecture had to fit in as part of a scaffolding plan and each assessment and exercise had to be designed to fit. All of these were new bits of work that were added onto lectures and grading. In the end though, we built a course that had a database that matched the learning goals and information structure of that course. We came away from this effort with an understanding of how students' learning progressed through the term.

This level of effort may not be required for every course, but it seems that certainly more attention should be placed on understanding the structure of the information and constructing a database that matches those assumptions. It seems dangerous to launch hundreds of courses, from all different disciplines and colleges, using course construction tools that only represent one flavor of the many possible variations. If this requires more extensive faculty development and time for development, then that cost should be considered.

 

References and URLs

(1) Teaching with Structured Information. Title 3 Workshop presentation. http://web.pdx.edu/~rueterj/title3/teaching_structured_info/teaching_with_structured_information.htm

(2)Teaching with Structured Information: Part 2 example course. http://web.pdx.edu/~rueterj/title3/teaching_structured_info/part2_experiences.htm

(3)ESR202 Course home page http://web.pdx.edu/~rueterj/esr202

(4)Tutorials on using Access and Cold Fusion written and compiled by Jo Meyertons http://brain.clas.pdx.edu/cfindex.html

(5) Perkins, David. 1998. What is Understanding? In: Teaching for Understanding. Martha Stone Wiske, Editor. Jossey-Bass.

(6) Operational definition of understanding for ESR202 http://web.pdx.edu/~rueterj/esr202/orientation/understanding.htm

(7) Learning objectives for ESR202: http://brain.clas.pdx.edu/jrueter/get_full_list_script.cfm
http://web.pdx.edu/~rueterj/esr202/slo.htm

(8) Linked to each assignment: http://web.pdx.edu/~rueterj/esr202/course_schedule.htm
: