A. Introduction

Explicitly structuring the information in a course will help both teaching and learning. The students benefit by having consistent organization of the concepts and, especially in lower division courses, a limited domain of knowledge. The database helps faculty plan the course, outline individual lectures and make the assessments and graded assignments fit together.

The theoretical background and assumptions are presented in a previous paper (Teaching with Structured Information). In this paper I will present the details of my approach to using database to teach an introductory science course for non-majors. The details in the examples are instructive in the sense that they demonstrate some of the advantages and disadvantages of this type of structured teaching.

The course described here was General Biology (Bi101). The course is aimed at non-science majors who are trying to meet the University's science requirement. The university science requirement is new, and this might change the motivation from recent years. Students may be in the course more to meet this requirement than for their own interests. The course has 3 credit of lecture and 1 unit of lab. The course had two sections, a day section that met two times per week of about 110 students and an evening section that met only one time per week of about 60 students. The course three written assignments, an ungraded assessment every week, three quizzes and a final. All assessments, quizzes and the final were short answer questions. The score on every question was recorded to a database (Access) which was made available to individual student (using their student id number as a password) by using Cold Fusion on a university server. In addition to the scores, the database also would return individualized comments to students on their assessments.

The work I did in preparing and delivering this course is part of my overall effort to find ways to have the students be involved in meaningful transactions with the instructor. In this particular case, the feedback on the assessments, written assignments and quizzes were intended to help the student improve their performance. The value of this course to the student is in the process that helps them learn the material. They are paying tuition to get some of my attention. i.e. this is a case of "attention economics" (Dyson 1998). In a large course such as this, I have limited attention to devote to students and thus the value to students becomes how well I can direct the learning process for each student. I have observed that in many courses good students get less feedback (because the feedback is assumed to be corrective) than poor students. I am trying to develop a teaching style that gives feedback that is of the same quality and effort as the students' submission. This has two implications, 1) some students won't get very detailed feedback if they submit sloppy or hasty assignments (even though they are doing poorly) and 2) the value of the class is in the quality of the transactions (not in the grade or the credit) and that transaction is exactly equatable to my time and attention.

B. Reasons to teach with databases

Organizing your course content and structuring your teaching approach has the following potential advantages:

1. Defining the relationships between assignments and assessments helps the students understand, up front, what you are looking for.

2. Individualized feedback on the assessments can explain how each student would improve their performance on a similar question or a question of the same category.

3. Students can be given meaningful feedback during the term. This feedback can be based on improvement of individual student learning outcomes.

4. Grading of the quizzes and other assignments can be modified after the papers have been handed back. Either errors in grading or new interpretations of an answer can be applied to the entire class or just specific students.

5. Generation of the set of assessment feedback responses can identify students' difficulties and misconceptions. This aspect would be very valuable when teaching a new subject or teaching to a new group of students.

Details of the approach will be described in the "Process" section and specific examples given in the "Examples" section.

The all-important problems with this method will be left until last to emphasize the drawbacks.

C. Process

I used six steps to organize my course material for teaching in this mode.

1.Learning Framework. I chose a learning framework that fit the depth and processes that I thought the students would be going through in learning this material. The learning framework that I used was a combination of the first three levels of Bloom's taxonomy (Bloom et al ****), (vocabulary, concepts and application) and Perkin's (Perkins ****) description of understanding. I was expecting students to be able to demonstrate that they understand the concepts in the course by writing about them, using them in addressing current problems and tecalling these concepts and using them in an un-prompted manner. These learning goals required that I use short written responses from students rather than multiple choice questions.

2. Organize the course into units. I divided the ten week quarter into three units. Each unit has a set of themes, a study guide, assessments, written assignment and a quiz. In the past I have used five two week units successfully. The longer unit helps by providing more time for feedback before the quizzes. The two week unit version worked very well as an "internet" course, because it was easier to get timely feedback to the students.

3. Specific Learning Objectives. I created a list of vocabulary and concepts for each unit. There were approximately 40 vocabulary words and 15 concept questions for each unit (insert link). The students were expected to memorize and be able to use the vocabulary to describe the concepts. Application questions are novel uses of the concepts. These couldn't be listed on a study guide, but I did provide example application questions. In addition, I provided the algorithms that I used to derive application questions from the list of concept questions. These algorithms (insert link) were very simple, essentially I would combine two concept questions or ask the question in reverse. These algorithms for generating questions represent the problem solving strategies that I expect students to employ. These were discussed in class.

4. Concept map and themes. The concepts in each unit were mapped out with the major relationships between them. Concepts from previous units could be incorporated into this map. These maps were very useful to me in generating the major themes for the unit. A theme is a set of topics that are self-referential and thus reinforce student learning.

5. Scaffold the content. The book presents the material in a very hierarchical approach building by content (from a reductionist view). This can serve as a very important learning resource for the students but I felt the face-to-face lecture time should be used differently. I organized the lectures around examples, starting with a example that is fully explained and working toward bringing in examples that the student has to fill in from their own understanding at the end of the unit. My in class approach assumed that the students were studying and learning the vocabulary. I learning a little late in the course that many of the students weren't learning the vocabulary because there was no specific consequence.

6. Build causal relationships to previous content. In the learning framework that I used, I am assuming that students need to learn the vocabulary to describe the concepts, and they need to be flexible in their understanding of the concepts in order to be able to apply that to new questions. These processes of student learning can be supported by the scaffolding approach and the evidence that students are learning in this manner can be discovered by checking the causal linkages between individual questions in their work.

Examples of database use

These following examples are provided to explain how the database was used for teaching and to describe the details of using the technology.

D1. Providing individualized feedback on assessments

Every week the students were asked to do a written, in class assessment. The assessment question was usually a right out of the study guide and related to the day's lecture. The assessments were picked up in class and then discussed briefly. I usually asked several students to answer the question. This allowed for immediate feedback on the type of answer that I was looking for, the depth of the response. It also guaranteed that the students didn't leave the room thinking they got the question correct when they didn't.

Each assessment was worth one point toward the grade. I also gave the assessment a potential score, as if it were a quiz question. This potential score was meant to be a "heads up" to student who scored poorly.

I usually graded the assessments using Excel. I would read about 30 of the assessments and write out phrases that would correct the students error or address a misconception. For example on a question about biodiversity, I identified three areas that students should have addressed (flow of energy, functional roles and interactions between organisms), I identified several misconceptions that needed to be corrected and wrote out responses that I thought would help. In addition, I developed hints for how students could do better. If they did poorly, they got a hint that would get them just to the next step. If they did well, my hint would tell them how to generalize their answer or improve it in other ways. I tried to avoid giving the best answers a response limited to "very good". As I worked through the first 30 assessments, I was able to refine my answers to some of the errors and I was able to categorize the errors into several general types. This sample helped me understand how the class was doing in this concept area.

The rest of the answers to the assessment were generated in a andro-mechanical method. I named all the phrases in Excel using a two or three letter code for the entire string. Then I used the concatenation function (&) to link these phrases together in a text string. The outcome was an Excel sheet that had the student id, last name, first name, score on the assessment (1 point), potential score on the assessment (0 to 3 points) and a text comment. I also added a warning or additional comment field in which I made comments such as "poor spelling", "grammar problems" and "can't read your penmanship". These comments were separated out to indicate that they weren't used as a basis for grading but could interfere with my ability to judge their performance and understanding. (This last field was very useful in identifying several students whose command of English was not sufficient to perform in the class and who also had trouble being in the lab safely.) This worksheet was transferred into Access as a table and the entire Access database uploaded to a university server. The id number (as a text string) was used as the primary key for each table.

A list of the assessments and answers is listed at this link http://web.pdx.edu/~rueterj/bi101/grades/assessment_summary.htm

D2. Causal analysis of the learning process

The assumptions built into the framework of learning, ordering the specific learning objectives and scaffolding can be checked by causal analysis. We can analyze student learning by comparing an individual student's performance on an assignment that demonstrates a particular skill and comparing that to the same student's performance on a subsequent assignment that requires (or is assumed to require) that skill. This comparison is easy to do using a database and we can even group students by their relative performance on two assignments.

I developed a list of seven different relationships between material in the assignments. These different types of links are in three categories, those that are used to check student learning from one point to the next, those that represent the students' ability to build on one idea to form another, and two links that represent problem solving strategies based on previously learned concepts.

1. exactly or nearly the same question asked again
2. contains or deals with the same concept

3. restated concept
4. vocabulary necessary to build concept
5. concept necessary to use in application

6. problem solving using identified concepts
7. problem solving which requires concepts to be identified

Figure 1. Map of the assessment and quiz-questions in the course with different linkages. Contains means that the second question contains or deals with the same concept. Restate means to restate the same concept. Probsolv2 means a question in which the prerequisite content needs to be identified and associated to each other before the problem can be solved. Build-vocab means that the prerequisite vocabulary must be used to answer a concept type question. Build-concept means that prerequisite concepts must be used to solve an application question.

I used this approach midway through the term to emphasize to students how important it was to pay attention to the feedback on the assessments. I analyzed their relative performance on assessment 5 to Quiz B question 2.

The two questions were:

assessment 5: In the context of cellular respiration, why do we breathe in O2 and exhale CO2?

QuizB-2: Describe the path of oxygen from the air to oxidative phosphorylation. How is O2 used in cellular respiration?

The results are presented in Table 1.

Table 1. A set of queries was done on the database to look for the relationship between the performance on assessment 5 and subsequent performance on Quiz B question 2.There were 152 students who did both questions.The intervals are calculated to include the higher value.

The major point of this for the class was that 82 students didn't improve at all. Those students need to pay attention to the assessment. I also emphasized that I was pleased with the performance of the 28 students who did improve.

After the course was completed, I analyzed several other relationships to determine if students had been paying attention to this process.

Assessment 9: the question gave results from a cross that should have been put into a Punette square and the students should have determined that the mother and father had to be heterozygous and that the pattern of inheritance was "incomplete dominance".

Quiz C question 8: The students were given a Punnette square with the phenotype outcomes and asked for the genotypes of the individuals and the pattern of inheritance, which was "incomplete dominance".

The set of queries for this show that more students improved on set of assessment to quiz question than before (Table 2).This analysis shows that many students improved. Some of this can be attributed to the difference in scoring (since the assessment had a top end of 2 points).

Table 2. A set of queries was done on the database to look for the relationship between the performance on assessment 9 and subsequent performance on Quiz C question 8.There were 120 students who did both questions.Assessment 9 was only graded on a scale of 0 to 2. The intervals are calculated to include the higher value.

Other causal analyses were done to look at other questions. How well do students retain information from the first unit to the final and do they review effectively? These two questions can't be separated in my analysis, but they address the same issue.

Quiz A question 1: Using water as an example, explain the differences between covalent and hydrogen bonds.

Final question 11: Using DNA as an example, explain the differences between covalent and hydrogen bonds.

Table 3. A set of queries was done on the database to look for the relationship between the performance on quizA 1 and subsequent performance on final question 11.There were 168 students who did both questions.The intervals are calculated to include the higher value.

check these values

		A 0-1 Q 0-1 38	A 0-1 Q 1-2 22	A 0-1 Q 2-3 6
	A 1-2 Q 0-1 16	A 1-2 Q 1-2 14	A 1-2 Q 2-3 9
A 2-3 Q 0-1 17	A 2-3 Q 1-2 16	A 2-3 Q 2-3 18
29% did worse on quiz		42% did same on quiz	22% did better on quiz

This analysis shows that many students did worse (29%) on the final than they did on a similar question during unit A. In addition, a substantial number of students (all those with quiz of 0-1, 42%) weren't incorporating the concepts from unit A into the discussions of DNA structure and function in unit C. On the other hand, 22% did better on the final than on quiz 1 and 42% did the same on both quizzes. It's difficult to decide whether more effort should have been expended making the 29% of the students who did worse aware of this linkage to previous material.

As another follow up to whether students follow the process, almost the same question was asked on question 12 of the evening final. A similar concept question was asked on the day final.

F12 day: How is the structure of the mitochondrian related to the process of cellular respiration?
F12 eve: Describe the flow of oxygen from air to oxidative phosphorylation. How is O2 used in cellular respiration?

Following their progress in Table 4, 48 students got 1 or less on the assessment, the quiz and the final question. These 48 students showed no improvement. 26 students improved from quiz B to the final. This is evidence that they were starting to pay attention the process or it might mean that after seeing the question twice before it started to sink in. The rest of the values are spread out but can be lumped into three groups; improved from assessment to the final (49 students), had the same score on the assessment and the final (73 students) and, did worse on the final than on the assessment (9 students). The problem with this analysis is that so many people did poorly on the first assessment that there wasn't really anyway they could do worse on the final.

Table 4. A three way query between Assessment 5, Quiz B question 2 and Final question 12. This query should probably be redone to separate out day and evening.

Another type of causal analysis is to look for how students handle a rephrasing the question. One example of this is how assessment 8 should prepare a student to answer question 13 on the final. Both of these questions deal with analyzing a genetic relationship using a Punnette square. The data in Table 5 demonstrate the most of the students got the question correct on the assessment and were able to answer a question asked in the inverted manner. This shows that they didn't just memorize the answer, but were able to work with the concept.

Table 5: Causal analysis for a "restate a concept" type linkage between A8 and F13.

The most difficult questions for the students were the application questions that involved some type of problem solving. Question 15 on the final required the students to indentify the concepts that would be needed to answer the question and then manipulate those concepts. Causal analysis (Table 6)can be used to check that the student at some time demonstrated knowledge of the concepts that they will need. In this case, the students should have been able to recall and work with the concept of how enzymes work in metabolic pathways to describe the difference between burning and cellular respiration. Quiz B question 1 dealt with enzymes and pathways, and language from the answer to that question would have partially answered final question 15.

D3. Quiz regrading

The database is very useful for regrading quizzes that have already been handed back. In one case, the diagram in a question was misleading. Students who normally scored very well were missing that question, and other students, apparently unaware of some of the details provided a reasonable answer. With the database containing a record of every question for every student, it was easy to subtract that question from the scoring for that quiz. Thus the curve for that quiz was determined without that question included. Students who did answer the question were given "extra credit".

D4. Evaluating student preparedness

During the first weeks of the course, I was struck even more than usual that some students seemed to be unprepared to be in a college level course. Some students were unable to speak and understand English very well. This was of particular concern in the lab, where one of the TAs was actually worried about one student's safety. In addition some students had difficulty spelling and forming coherent statements on the quizzes. The admissions office has no control over English proficiency on students unless they are foreign students who are going to school full time. One student who couldn't even spell words correctly that were given in the question (i.e. she didn't even realize that she spelled words wrong). I asked her how she got through high school and her other college courses and she said that most of the tests were multiple choice and all of the other work could be done on computer and she could use the spell checker. It is difficult to have a dialog with students at a level appropriate for college courses if they can't speak English or write well enough to communicate simple ideas.

I analyzed the performance of students based on their status (year in school). Many freshman did not have a satisfactory outcome in this course (Table 7). My records show that there were 89 freshman that started in the class. 16 of these withdrew or faded out and a total of 29 (31% of all freshman) either withdrew or got a D or an F in the course. It seems that, in particular, Freshman are ill-prepared to take this course. Better information needs to be provided to Freshman about large-format courses.

Table 7. Course grades for students by enrollment status. The grade categories include the + and - subcategories.

Another particularly noticable deficiency in some students was their inability to write a coherent sentence on in-class work. The papers that students wrote out of class had much better grammar and spelling. Writing in-class however is a better indicator of their communication skills (except for students with test anxiety). On one final question that dealt with the concept of burning (as in a fire), I collected exceptionally poor answers both for content and phrasing and requested background information on these students (Appendix A). I was looking for evidence that the students who couldn't answer a question about fire were ill prepared for college. Instead, what I found was that these students had an average high school GPA of 3.2, 6 were Freshman, only one of the 8 got above a C in my class and 2 got Ds. I interpreted this to mean that this sample of poor writing was indicative of their overall performance in my course. In addition, I felt that their high school GPA wasn't a very good indicator of their preparedness for a college course. There may be a problem with the meaning of grades in college and high school.

Problems

There are two major problems with using databases in teaching a course like this. The first is data entry. Paperwork, organization and grade entry from written reports is time consuming process. Each assessment, assignment and quiz needs to be handled multiple times including alphabetizing. This course had enough students such that full student id numbers needed to be used to be unambiguous. Student data entry into cgi forms is also unreliable. There were many errors in student id numbers entered by the students themselves. Along with data entry is the problem of propagation of errors from the Excel sheet to the Access database. Several times, I overwrote updates that I had made on the Access database by transferring in a table from Excel. It is difficult to keep track of these issues when using multiple applications (Excel, Access and others).

The second major problem is that adequate tools weren't available for the tasks that I had to do. For example, cutting and pasting from Word documents into Groupwise email is awkward and mouse intensive. Mouse mileage is a good indicator of both tediousness and repetitive strain injury. (WebCT and other web-based interfaces generate high mouse mileage.) There are also no "worldware", i.e. common desktop software, that can be used for the visualization of the relationships between learning objectives and concept mapping. I will be trying out some particular tools, but unless these tools are readily available, it will be difficult to promote the use of databases for other faculty.

F. Summary and Conclusions

A database is a very valuable tool to improve the quality of communication with students. It allows the teacher to track individual students more easily and to perform queries that relate to how students build knowledge. Using this particular technology (Access database) allowed me to serve information easily to the students on a web-site. Access can be used in combination with Cold Fusion in support of web queries by the students. I was able to give students individualized responses to their work and thus prompt them to improve. This tool takes a great deal of time to set up, care for and harvest. This amount of effort might not have a net benefit in some courses, but I felt that it helped me in a class with about 180 students in two different sections.

Teaching strategies should be aimed at the learning processes of students. Structuring the information in a course by using lists of learning objectives, themes, concept maps and then scaffolding the lectures to address these learning processes takes a great deal of organization. One of the benefits of the database was to help visualize this organization and to be able to look at individual student progress through the process. I had to learn quite a bit about Access and queries in order to make my course communication work the way I wanted to, but I feel that I was better able to represent the underlying processes to the students. WebCT is based on a database and should have some of the same advantages that I list above. I have taught with WebCT but was totally unable to adjust the structure of the database to fit the course. My feeling is that heavy reliance on a tool (such as WebCT) that faculty don't understand is a risky proposition for both the faculty and the institution.

I feel that I made progress toward my goal of forcing students to be involved in a transaction in order to get my attention. There was high participation in the assessments and very few of the students wrote down hasty answers (even though they would have received full points for that). My responses to the assessments should have helped all students (including A level students) improve their answers. The quiz questions were predictable but difficult. Students who were moderately prepared should have been able to start answering the questions and get partial credit and feedback, and yet the questions should still have provided a challenge for even the best students.

References

Blooms

Dyson, Esther 1998. Release 2.1 A design for living in the digital age. Broadway.

Perkins