In the minutes of my school’s Board of Education from a meeting in the early 1980s, I remember seeing a discussion about the dangerous effects on students of listening to loud “rock music” in headphones. If only they knew how technology would advance! In my last year before retirement, technological advances had become such that many of my students walked around all day with a tiny device like a hearing aid in one ear. Long hair could hide them in classrooms where they were banned. Many young people spend the day serenaded by their favorite playlists and having their texts read to them. Like many of you, I wonder if this is good.
In 2015, cell phones were starting to become ubiquitous in my small, rural school. Students wanted to plug in and listen to music while they worked. I indulged this at first to see what would happen and I ended up banning the practice. But some of my high performing students were convincing in their pleas to listen to their tunes. I was curious enough to read up on the research.
Naturally, the answer is not simple. Sometimes it helps, sometimes it doesn’t. There are so many variables to consider that are missed if one only phrases the question “will listening to music help studying?”.
I settled on this policy for my classroom: I would grant a student a “license” to listen to music during working times in class provided that (1) they read and summarized adequately my own research paper on listening to music while doing school work and (2) that they maintained a class average of at least 85.
I wrote that paper on listening to music while studying while listening to music. I admit it. While I am working, I like to listen to music. If I am doing labor like painting a room or stacking wood, I like the 90s pop/rock playlists on Pandora. When I am doing academic work, I sometimes play something classical. Any of my former students reading this will no doubt roll their eyes because I probably did not allow their tunes during the lesson.
Typically, the weaker student with attention-focusing problems suffers from music during academic work. They engage with the lyrics of the song and the beat of the music in a way that adds cognitive load to their school assignment and impedes working memory. More than once I saw a kid air jamming guitar while summarizing a text on some world revolution. They interrupt their work to skip a song they don’t like, to choose a different playlist, or to search for the song they just realized would be great to hear right now. *Insert eye-roll here*
But some other students do seem to perform better on tasks and to focus better with the music on. In fact, so do I.
You can manage badges and permits for your class at Innovation Assessments.
We know now from mountains of research that human beings cannot truly multi-task. I would suggest that, in situations where music is not helpful for study, it is better sometimes a separate task that takes a student’s attention away from the one they probably don’t want to do anyway. In the paper, I suggested that it was a question of cognitive load.
For the student who benefits from this, the music becomes a part of the task just like a fiddler’s bowing is part of the overall task of fiddling with the left hand fingers on strings.
A problem I was never able to resolve was this: I wanted to create some test to see who would benefit from music and who would not. I actually had very few students ever apply for the license to listen to music in class. They were intimidated by reading that paper, I suppose.
In the end, I guess the question was not resolved satisfactorily. I still banned music. Some kids still sneaked it in. It didn’t matter much. Hopefully, one day someone clever will develop a test to see who benefits and does not.
In my grandparents’ day, we’re talking the 1930s, students who took world languages in high school were really aiming for a reading knowledge of the language. College-bound students of that era would be ready to read sources in their original language perhaps, and not so much ready for travel. In the hand-wringing self-criticism of the 1980s and 1990s, there came a growing concern that world language courses were not “communicative” enough. Students lamented that they took four years of such-and-such language and couldn’t actually speak it.
So in the spirit of reform, when I was being trained as a language teacher in the early 1980s, we were baptized in the holy water of communicative proficiency and realia.
I wish to demonstrate that teaching culture using the target language is an effective means of getting students to communicative proficiency.
But here, decades later, I have to seriously question this focus for students after the second year in a high school world language course. My reason is pretty simple: despite the great reform of those decades and the sincerest efforts to produce good communicators in the target language, world language high school alumni still rightly complain that they cannot really speak any of the language they learned. The criticism still stands.
We all seem to just accept this and the way it goes. World language teachers have lobbied for years, successfully, to promote our stock and trade. Among the benefits we can tout are an enhanced understanding of other peoples, possible career opportunities for those who develop proficiency, improved reading and writing competency in their own language, and so forth. What we cannot say is that our charges will retain the skills we taught them. Even my French four students came back years later to visit with their once-honed language skills dissipated completely. The fact is, one cannot develop linguistic proficiency in the traditional classroom.
The main reason to do this is that learning culture will stay with students longer and enhance their overall education more profoundly than learning a language they will most assuredly mostly forget.
I want to promote a way to teach world language in high school years three and four that makes learning another language highly satisfying while at the same time achieving whatever basic linguistic proficiency is possible in the traditional high school classroom.
Everyone includes culture studies in their world language classes, especially at the higher levels when the language skills are enough to make many authentic culture forms accessible. May I propose to make this culture study the centerpiece of year three and four studies and that it is indeed possible to do this at level three with the right scaffolding techniques. The main reason to do this is that learning culture will stay with students longer and enhance their overall education more profoundly than learning a language they will most assuredly mostly forget.
The correlation between the quarter marks’ average [in a culture course] and the French Regents Examination [a measurement of communicative proficiency] average score was 0.70.
In the early 1990s, I renamed my French III course “French Civilization”. I made culture studies, with thematic topics determined by students, the centerpiece of the lessons.
Can you really teach content and still get kids ready for the Regents?
Let’s look at the data. I taught French from 1991 to 2004, then a few courses from time to time, and finally again in 2022-2023. I saved my final grade sheets since 1993. I wish to demonstrate that teaching culture using the target language is an effective means of getting students to communicative proficiency. I think the data backs up my claim. I have analyzed my students’ grades in French Civilization (what I had entitled my third-year course) from the 1994-1995 school year through the 1999-2000 school year. During that six-year period, I taught 86 students third-year French as a civilization course.
The class average on the Regents during that six-year period was 80. The average of the quarter marking period grades for those students during that time was 82. The correlation between the quarter marks’ average and the French Regents Examination score was 0.70. The spreadsheet is available below. This strong correlation supports the idea that you can teach culture and build language proficiency. I submit that this method is better because of its long-term benefits to students in teaching them content much as they would learn in a social studies course. This is a life-enriching educational practice that still meets the communicative goals measured by world language examinations such as the Regents examinations in New York State.
Scaffolding lessons designed to make authentic culture materials accessible to students are in the wheelhouse of every language teacher. A complete training program is beyond the scope of this article. But a few worth mentioning are these: easier versions of classic texts, pre-teaching key vocabulary, training programs to help students construct meaning in a sea of unknown words, and surely coaching in the native language as students tackle the tasks. A culture course where the class chooses its topics of study will enjoy some motivation from interest alone. Good scaffolding lessons let teachers bring resources to accessibility and help students build confidence.
Here’s a lesson in reading skills for English that can easily be used in reading world language. Actually, I built this lesson from my world language teaching training and elementary ed. training.
Student interest becomes a driving force.
Most students who advance to this level of world language learning are interested and usually are academic-minded. The freedom afforded to choose their learning seems ideal in a way Rousseau would appreciate. They’ll need to be brave to tackle plays written in the target language or try to understand a classic film. But my experience has been that interest and choice are really strong motivators.
Students who have complained that they don’t really end up very fluent from high school world language courses have found little comfort in the rationale provided by adults around them. It’ll get you out of college courses. It’ll make your transcript look good. It’ll help you order a meal at that French restaurant in town. Your English gets better. You’ll learn about other cultures. You’ll learn grammar. And the list goes on… But emperor has no clothes, and the band keeps marching him through town.
For third and fourth-year students in high school language classes, the data support the idea that culture classes lead to the level of communicative proficiency measured by standardized tests like the New York State Regents examinations. The level of proficiency students can actually achieve is fairly limited and maybe we should be more up front with students about that; set more reasonable expectations. Culture and civilization classes enrich a person’s education, a person’s life, in ways that just teaching them to ask when is the next train to Madrid won’t do.
So let me go way back to my training to be a French teacher in the late 1980s. My sponsor teacher was Mr. Tom Ham at Potsdam High School. He was instrumental in teaching me techniques for developing conversation skills by “scaffolding” students up into more sophisticated expressions that had an element of improvisation within the proximal zone of development of the class.
French poses some difficulty for complex expressions that students of, say, Norwegian, don’t face. French has lots of silent letters, for example, and the sound that the letters represent is more alien to English speakers. “What’s this?” is “Qu’est-ce que c’est?” which, if you don’t speak French, sounds like “KESKUH-SAY”. Holy cow! Lots of extraneous letters. Languages like French are best learned when students have a lot of repetition in these difficult structures. The listen-record task seems to fit the bill.
Mr. Ham used a lesson that didn’t have a name so for lack of a better term, I call it the “listen-record”. Students are presented with a few model sentences with parts missing. The missing parts are improvised by using our current vocabulary list and prior knowledge. Each student goes, in turn, to ask a question, choose a classmate, and then the classmate creates a response. Everyone listens and records in the table the name of the respondent for each exchange.
At the end of the lesson, there is a quiz on the table during which I ask, in the target language, who did or said such-and-such. students look up the name in the table and record the name on the answer sheet.
I echo out everything everyone says repeatedly, so this enhances the experience and recall.
Listen-Record table with quiz on back fold.
A student’s completed table.
I really like this activity. Students have to be engaged all the time. They have to listen and practice speaking. They often improvise amusing permutations of the responses and this lends itself to long-term recall of the vocabulary. Best of all, this activity presents the same model phrases over and over. I echo out everything everyone says repeatedly, so this enhances the experience and recall.
These are all terms I have seen for a slightly different activity designed to build conversational competence with some improvisation. In the guided dialogue, the student participates in an extended conversation where every other exchange is in the target language. Back in the early 1990s, I found a British product for teaching French that was interesting. There were two books, an A and a B. Each chapter presented a different half of a conversation with necessary supportive materials like vocabulary and phrases. The activity was designed to have two students practice conversing.
A British communicative activities book I used in the early 1990s.
I tried on and off to use this during the time I taught French, from 1991 to 2004 (I taught social studies from ’04 to ’22). I could never get the students to commit to remaining in the target language throughout and that was the main reason I abandoned the activity. But I still wanted some kind of guided conversation that included scripted elements and improvised elements. I needed this to be topical so I could include it in a thematic lesson.
Reading and listening are important, but students who do too much of this and not enough speaking and writing without assistance end up notreally being able to use the language they toiled so hard to learn.
The Old French Regents Examination
Screenshot from the 1985 French Regents part 2.
Those of you not in New York State may not know about our state testing system called Regents exams. Back in the 1980s, the old exams had a part that I liked but which was eliminated in the new exams in the 1990s. In this part, the teacher read a setting in French twice, then something in the target language to which student were to respond. There was a prompt in English telling students in a general way how to respond.
I like this exercise because it calls upon students to produce language. Many commercial world language programs heavily emphasize receptive language because it’s easier to grade. Reading and listening are important, but students who do too much of this and not enough speaking and writing without assistance end up not really being able to use the language they toiled so hard to learn.
Okay, so I’m not good at naming things… But the series of interconnected conversation lessons that I developed using all three of these experiences I will term the “Scaffold Dialogue Package”.
Step 1: Learn the Vocabulary – Students should use whatever methods they usually do to learn the words in the thematic list. I use the online flashcards and word bank quizzes here at InnovationAssessments.com.
Step 2: Complete the two listen-record tasks: For a class of twelve, this taskes about one 40-minute period. I have taught this using paper and digitally using Google docs. We all like paper better for this. I harvested grades for this by calculating percent correct on an open-notes quiz on the table.
Step 3: Complete the Scaffold Dialogue- The scaffold dialogue is based on the two listen-record tasks. It’s important to echo what each student says both for improving pronunciation and for letting people hear enough to record the elements in the table. To avoide confusing when I echo and when I am giving the teacher response, I say “s/he said…” before echoing what the student said. I have harvested scores for this in two ways. If I have time, I give an open-notes quiz. Otherwise, I collect the sheets and select one column to score. This takes a whole forty-minute class period for a class of about twelve. I leave a lot of scoring leeway for these if I grade a column. There can be no English and it has to have some significant elements of what the student said, but for younger learners that suffices. For French 3, I require a little more spelling accuracy.
Step 4: The Test – Administer the test – it takes about twenty minutes or so. Repeat each item twice. When scoring, remember that only errors that affect auditory comprehension matter. Allow 2 free errors for French 1 and early French 2 and 1 free error for French 2 later in the course and French 3. I usually gathered a total score by calculating percent of checks out of fifteen.
For a class of twelve, the whole package takes about four class periods. Once students get in the habit of these, it goes rapidly. Each listen-record has a slightly different improvised structure, so there’s enough varation month to month for interest. Listen-record 1 usually is more basic and causes review of basic structures (like the Education one has students recall names of classes and school supplies). Listen-record 2 is more sophisticated, calling on students to offer opinions or to explain.
I would like to play the role of a crusader in this post, if you don’t mind. There’s something I would like you to consider about your teaching practice, or maybe reconsider as the case may be. I’d like you to abandon assigning your students comprehension questions on most kinds of texts you assign in your social studies class. Instead, I’d like you to consider assigning students to summarize instead.
When students do the search and copy instead of reading the text fully, they deny the author the chance to build with the reader an information schema of their own that lends itself to long-term recall.
Okay, so who do I think I am anyway to question a practice that’s at least a century old? Well, I cannot claim to be any super authority. I taught social studies for eighteen of my thirty-two years teaching. I worked in small, rural mountain schools and my roster averaged about one hundred students a year, which is likely smaller than that of my readers. I think these are good starters to convince you to at least hear me, but I have always felt that the argument from authority to be one of the most pernicious fallacies. So in honor of my thirty-two years, grant me the indulgence of reading further. As for accepting my suggestion, let the arguments sway you themselves.
When you assign your students to read a text selection and answer questions on it, you must know what the majority of them do. They skim the text to locate the answer, often they copy it wholesale, and then they’re done. Teachers who accept this kind of work are missing an important opportunity to foster greater reading comprehension skills and greater long-term recall. Writers, even writers of textbooks, build meaning in a schema of content to deliver to the reader, who constructs meaning related to their experience. Ideas are hierarchically arranged and organized to support a set of main ideas. When students do the search and copy instead of reading the text fully, they deny the author the chance to build with the reader an information schema of their own that lends itself to long-term recall. At the risk of putting too fine a point on it, questions on text are largely a useless exercise even when the questions are of high quality.
Once students leave sixth grade in my region, the formal lessons in reading stop. But a great number of sixth graders do not read at a sixth-grade level. They cannot advance their reading without texts at their level. And they have to actually read these. It is possible and desirable in middle and junior high school to promote the development of reading in youngsters by structuring what they do to process text. Teaching students to write competent summaries is one of the best ways to let them develop their reading skills. (This should be paired with offering texts at their independent reading level whenever possible).
What does the research say?
Summarizing and note-taking are two of the most useful academic skills students can have. (Marzano, Pickering, & Pollock, 2005)
Summarizing is in the top two most powerful writing tasks that support the development of student reading. (Graham & Hebert, 2010)
I was greatly influenced by a paper entitled Writing to Read: Evidence for how Writing can Improve Reading in 2010 published by the Carnegie Corporation of New York. A meta-analysis clearly demonstrates that composing summaries is superior to answering questions on text. The reader may argue that this research was conducted to address methods for using writing to enhance reading. But I would argue that (a) this is rightly a major goal of assigning most secondary students a textbook reading task and (b) the same cognitive processes that go into reading comprehension go into long-term recall of information. Students answering questions on text are less likely to be encoding from working memory into long-term memory than those composing summaries in their own words.
Around 2006 when I was making major investments in curriculum and methodology development, I sat down with a noted professor of literacy who happened to be working in our district, Trudy Walp. I explained my qualms about assigning students questions on text and I asked her, what is the very best thing I can have students do with text from a reading specialist’s perspective. She did not hesitate to say: have them retell what they read.
When I assigned my students summaries on their textbook articles early on, my top students wrote terribly long and detailed compositions that were not brief enough to be a summary. I needed to shorten my students’ summaries and to develop a rubric for evaluating the quality of a summary. The resulting task I used for some fifteen years: the Five and Three.
The assignment for the five and three is to summarize the assigned text (usually four pages in a typical high school textbook) in five sentences exactly, no more and no less. This is the “five”. The “three” is the personal reaction to the text. Students were assigned to connect this text to their own life somehow. What does it remind them of? What do they think about what they read? Why? These had to be exactly three sentences long. This was the second element that the reading specialist recommended to me. Students need to make the text meaningful to them in some personal way that makes new information integrate into their preexisting schema.
The five-and-three became the foundation of my students’ textbook reading work. They also had the option of composing Cornell notes for an assigned reading. I would invite the reader to return to the blog for a post dedicated to extolling the virtues of Cornell note-taking. The five-and-three faded in importance in certain classes where video lessons became an effective information delivery tool when reading instruction was no longer a priority.
The mechanical summary evolved in 2020 when I had a group of learners who struggled and who just would not produce summaries even given time to do so in class. I offered them the opportunity to write a summary by copying the first sentence of each paragraph word-for-word, then to connect and arrange these into complex sentences such that there were only five sentences. They had to compose the three-sentence personal reaction to the text just like normally done. For this, I offered a maximum score of 76 because, I argued, it had less value not being processed in their own words. I reason that not processing in their own words likely limited the ability of encoding to happen from working memory to long-term memory.
With the Assistance of AI
At this point you are likely wondering how you could possibly grade all these summaries. I will grant you, the workload takes some management, but I would argue that it is only slightly more time consuming than grading answers to a set of questions and the benefit to your students makes it all very worth it.
The AI grading assistant was accurate enough to save me tons of time scoring summaries. The summarizer app actually let me create models on which to train the AI that were highly accurate when compared to human-generated summaries… eerily so!
The ability to skillfully summarize is a lost art worth recovering for our students. It enhances the development of their reading ability and it promotes greater recall of the content we teach. It’s my hope my experience developing this system can make this suggestion seem do-able and worthwhile.
SOURCES
Graham, S., & Hebert, M. (2010). Writing to Read: Evidence for How Writing Can Improve Reading. New York: Carnegie Corporation.
Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2005). Classroom Instruction that Works: Research-Based Strategies for Increasing Student Achievement. Upper Saddle River, NJ: Pearson Education, Inc.
When the new Frameworks for social studies in New York State came out about six years ago, the first item that caught my eye was the constructed-response question sets. If you are not familiar with this task, in brief, it calls upon the student to examine historical sources (I use exclusively primary sources) by providing historical or geographic context, identifying the point of view, intended audience, or purpose of a document and then using the two documents in either compare-contrast, cause-effect analysis, or turning point identification.
Originally, the task was supposed to call upon the student to evaluate the reliability of the source. This was scrapped, probably after field testing questions. I regret their decision to scrap this (though it does appear as part of the analogous task, the short essay, in US History grade 11). Students can be taught to evaluate the reliability of sources at an early age (I incorporated it into my sixth- through eighth-grade social studies work). Students can be trained to evaluate based on features like the point of view (bias), intended audience, purpose, time and place, and authorship. Teaching this does require patience and practice. What I think happened is this: few teachers address the reliability of sources and so when they field tested the new exam format, everyone did so poorly that they scrapped the question. I wish I could find out how my kids did. I had been teaching the reliability of sources since they were in middle school.
I advocate a strategy of assigning one CRQ in each unit of study in grades nine and ten without access to notes. I used this as one of the tests at the end of a unit of study. I would tell students about what kinds of documents would appear and what historical context they should be able to recall in advance. It is a challenging task for them.
The first challenge for novices is to understand what it means to provide context. Faced with the question “What is the historical context of this document?”, beginners will retell what the document says. The reason for this mistake is that, since they were little kids, teachers have asked them to relate what a text means to prove they understood it. This task asks students to bring to bear what they know about the history behind the document, which requires a level of recall I don’t think we’re asking students to do enough.
Another challenge for students writing the CRQ is the third question where they must use two documents in analysis. Cause-effect relationships are the easiest for students, it seems, so I teach those first. When students are asked to compare and contrast the documents, some students have to be cautioned to select significant elements to compare. The fact that one document is a map and one is a diary entry, for example, is not a significant fact. The turning point question is the hardest, mostly because it calls on students to recall history and to understand historical trends both before and after the event. I find it useful to break this up into small tasks: first say what the turning point is, then say what is so special about it.
After a difficult task, especially for writing, I like to do a “debriefing”. In the debriefing after the CRQ, I like to share student answers anonymously in a slide show and discuss them. It is a very effective strategy for stamping out common errors early and permanently. Sample debriefing PowerPoints from an eighth-grade and ninth-grade CRQ are posted below.
Certain constant reminders were recurring for all my classes:
— Things are not reliable because of what they say or show.
— Things are not reliable because they are in quotation marks.
— Things are not reliable because they happen to match what you think is already true.
— Unreliable statements can be true.
— When you’re asked for historical context, do not describe what the document says.
And what about middle school? Can the constructed-response task be modified for that level?
Students with an IEP often are very challenged by these tasks because of the reading level of the texts. Since I use primary sources exclusively in my CRQ’s, this is doubly difficult for students with reading limitations. The answer to this is to consider obtaining CRQ’s designed with a lower reading level.
If I may be so bold as to give advice, I would suggest that students do these throughout Global nine and ten, perhaps once a month. I suggest that they be as tests instead of open-book or take-home assignments because students need to prepare for recall situations. The debriefing afterward is vital to class improvement and it makes for a long-lasting correction. I would advocate resisting the temptation to leave this off for the Global ten teacher. It really does take a long time to learn to do this well.
There was not a lot of math in my teacher preparation in the 1980s. Actually, I managed to take my BA without a math course (I took extra science instead). I sure wish I had statistics from the start!
Sometimes no matter our experience or preliminary testing, we are surprised at how poorly a class does on a test. Most teachers resort, quite rightly, to some kind of “curve” to alter the scores. Even a valid, reliable test can be too difficult for students.
In effect, the standardized scoring was a useful set of training wheels that naturally disappeared once the class met the normal performance level!
A second issue of interest is in scaffolding difficult tasks for our students. Some things require time and practice to learn to do. Students may face low scores at first on such tasks. We don’t want their grades or their confidence to suffer. “Curving” the grades on a task while students are still in training for it is a good practice.
Enter the z-score standardization procedure. I don’t want this to be a post about mathematics (mostly because I am not confident to do so), but I would like to promote this as one of the best ways to alter a set of test scores in situations where (1) the group’s scores are lower than expected (say, more than 5 points below the class average overall in the course) or (2) the class is still practicing a difficult skill.
My interest in z-score began around 2010 when I was working to establish that the different capstone unit tasks I let kids choose were, in fact, of equal difficulty. I value differentiated instruction, but I also strongly value fairness. Z-score standardization let me establish how the rubrics for tasks in my class compared to a state test.
Standardizing the scores requires data that basically establishes a norm. How “should” the class have performed based on how a large set of previous students have performed? One of the things that makes grade standardizing hard is that one does not always have access to this data. How should my kids have done compared to how all my previous kids have done? Well, I saved my data.
I taught French for the first thirteen years of my career (plus two years later on) and then the other eighteen years I taught social studies. Now, mind you, I’m not a person to save a lot of stuff. My classroom was always pretty bare and I threw out stuff I wasn’t using. But data, that’s something I like to save. I have hard copies of my final grade sheets for all my students from 1994 to 2013. I also have all my Regents results (for those of you not in new York State, “Regents” are standardized state tests in different subjects). Permit me to share my data with the reader. This data will give you the mean and standard deviation on population sizes of around 100 (between 92 and 100) for French grades 8-10 and for social studies grades 8-11. See below.
I learned to do this with the help of my colleague in the math department, to whom I am grateful for answering a lot of my questions over the years and helping me learn basic statistics. I am a computer programmer and I wrote an app to do the calculations. It’s available for free and I invite you to use it. Just enter your class’ test scores, then the mean and standard deviation of the standard test you’re standardizing the scores to. The app generates a table of standardized scores and some statistical information.
A good example of using this is when I was teaching Global Studies 10 and US History 11. The new New York State Regents exams in these subjects have stimulus-based multiple-choice questions. These are hard for students at first. I had them do one each unit as a test. Standardized scoring let me modify their scores so that their grades were not harmed and their confidence preserved. So here’s the beauty of the standardized scoring: the method sets the mean of the current task to that of the standard, then adjusts everyone’s score using standard deviations. As the class improved on this task month by month, the class average approached the standard mean, so the grades were affected less and less. In effect, the standardized scoring was a useful set of training wheels that naturally disappeared once the class met the normal performance level!
A number of my education courses back in the ’80s were kind of useless. Hopefully, teacher training is better today. (I only had a one credit course in behavior management theory! Sheesh!) Statistics would have been a good course for me because I used it so extensively in my career. Readers who are interested might subscribe to InnovationAssessments.com to see the other statistical apps that you might find useful.
In the early 2000’s, there was a lot of talk about differentiated instruction picking up steam. Whether or not it was just another bandwagon remains to be seen, but it strikes me that differentiating middle and high school social studies classes is not only de rigeur right now but it is the right thing to do. Like anything else, how this is done is worth considering.
As a beginning teacher in the early ’90s, my unconscious goal was to teach the course so every kid would aim to get an A. The unspoken, unchallenged notion seemed to have been that every kid who works hard enough can, and therefore should, get an A. Disabusing myself of this notion improved my teaching practice and, therefore, my students’ progress.
I offer the analogy of a paper grocery bag, when overfilled, breaks at the bottom and all the groceries are lost. It is probably a question of cognitive load, or overload, as the case may be, that inhibits encoding of information into long-term memory.
The concept begins with the idea that the average student across the world earns probably a C or C+ in most courses. That is, after all, the definition of average. This is really the goal of instruction for the majority of students. The mandate is to pass. I was greatly influenced by a paper from 1993 (Dempster, F. N. (1993). Exposing Our Students to Less Should Help Them Learn More. Phi Delta Kappan) in which the author argued that limiting the amount of content delivered will increase how much students retain. Much instruction seems to be guided by the notion that if we inundate students with knowledge they will scoop up more of it, as much as they can. The opposite is the case. I offer the analogy of a paper grocery bag, when overfilled, breaks at the bottom and all the groceries are lost. It is probably a question of cognitive load, or overload, as the case may be, that inhibits encoding of information into long-term memory. The “Basic Proficiency” curriculum is a parallel set of modified tasks for the regular classroom that may be accessed by anybody any time whether special education or not.
The beneficiary of this program is the student who struggles to get that C+ in the course. Their performance and satisfaction is enhanced by offering less; more manageable chunks of data to process, integrate, and retain.
So that’s the theory. In practice, it calls for a lot of preparation. I selected key assignments for modification: the reading task, the multiple-choice quiz, a writing task, and capstone options. The system worked thus (scroll to the bottom for sample parallel curricula): students electing basic proficiency at the start of a unit needed to declare their intention at the start. Then as the unit progressed, they accessed the modified versions of each assignment. When I moved my course materials completely online to go paperless in 2019, students changed from pulling modified assignments from a different folder to accessing a different set of links at Innovation Assessments.
For this kind of differentiated instruction to realize its full benefit, it must be paired with some remediation time outside class.
Fairness
I began developing this strategy about 2007. There are a number of key things I learned along the way. The first was to address the issue of fairness. A reasonable critique of differentiated instruction, and an element of this practice that was often overlooked by its proponents in the beginning, was the importance of ensuring that assignments that were different were actually of equivalent value. If student A is doing less than student B, how can student A reasonably expect to earn the same marks as student B? Furthermore, there were cases where strong students chose basic proficiency because they wanted an easy grade. Both of these problems were resolved by setting limits on the maximum score a student could earn on a modified task.
One of my friends in the special education department once made the case that a student with disabilities who was doing their best within their ability should be entitled to an A. I took a more conservative position on this, maintaining that the value of a work product was little influenced by the effort of the producer. There were valid and reliable ways of measuring the quality of student essays and the criteria were unaffected by ability.
Differentiated Reading
I would suggest that one of the key features of this plan, one which I embraced with some reservations at first, was to offer different levels of textbook reading for students on the basic plan. I was able to find history textbooks at a fifth grade reading level for all my classes (I was teaching US history in grades seven and eight and global history in grades nine and ten). In advance of the school year, I selected page numbers of articles that mirrored what was to be assigned from the standard textbook. Students process text in my classes using one of two methods, the “five and Three summary” (blog post coming soon on that) or Cornell Note Taking.
It should be noted that these texts contained about half as many words in a larger font and less than half the information of the standard text. Students choosing the easier reading were also choosing to learn less content. While they may learn some of the missing content from other activities such as my lecture, they were still having less served up to them.
Many assignments were the same for everybody. Each unit progressed through the same type of activities, key elements of which were offered on a modified basis. Management considerations dictated that students had to choose one package or another, Standard Inquiry or Basic Proficiency, on a unit-by-unit basis. the could not choose on a task-by-task basis. It was not possible to manage it.
Don’t Worry: There was plenty of enrichment for advanced students.
In the spirit of differentiation, I maintained a collection of college level books my ambitious learners could choose for their reading assignments. I had developed rubrics for more sophisticated versions of our essay work so students could attempt the next grade level of work. Differentiating for these students was done on a task-by-task basis and less formally, but being self-directed scholars they managed this mostly themselves. I only needed to provide the materials and encouragement to challenge them.
So How Did It Go?
I carried out the plan for a little over ten years in all my classes, grades seven through ten social studies. I found some interesting things. Firstly, I discovered that weaker students who chose basic proficiency in grades seven and eight mostly moved to standard inquiry by grades nine and ten. They tired of the maximum score limitations and they developed the skills to approach academics more effectively by having materials at their ability level to work on. Secondly, I discovered that some students who would be candidates for basic proficiency would sometimes choose standard inquiry if they liked the topic. The American Civil War was often one that had most people doing standard work. The effect of this was to give the weaker students confidence.
In 2012, I did a study of student progress on the plan to see whether I wished to continue it. Results were strong enough to continue the practice.
In my last year teaching social studies before retiring, I had to discontinue the program. The reasons were practical. My course assignments and rosters increased during this time period. A big disappointment for me was the loss of this extra remediation time in my schedule. I lost my remediation periods in the schedule in favor of teaching more courses (we taught six different subjects / grade levels where I worked). This showed me that modifying the work was sometimes not enough. Some students needed more time with their teacher outside of class and being denied this was a serious blow to my program. For this kind of differentiated instruction to realize its full benefit, it must be paired with some remediation time outside class even if only thirty minutes a week.
When I did a study of the work submission rates of my students during the remote learning of the pandemic, I discovered that there was a huge drop in work submission and homework completion overall starting after 2017 when my remedial class periods were cancelled. I invite the reader to return in the future and read my blog post on my experiences teaching in the pandemic. Pertinent to this discussion, the detrimental effect of eliminating extra help for my students was demonstrable over time.
I present this basic proficiency idea to the reader as a possibility they might consider for their classrooms. I found it to be a recipe for success, especially when paired with appropriate remediation. One needs to bear in mind that even when the assignments are all prepared in advance, there is a significant investment in time for management of the plan and for scoring a wider variety of assignments. Technology can help a lot with this (Like the Innovation Assessments learning platform!) but schools are advised to provide teachers who do this enough planning time and student contact time to make it happen. It is rewarding for the learner and well worth the investment.
When I switched from teaching French to social studies in 2004, one of my first projects was to develop lesson plans for formal debate and mock trials for my classes. In time, these became a centerpiece of my units, second only to primary source work.
I value the critical discourse of debate in my middle and high school classes very highly, in the first place because it causes participating students to learn a greater quantity of history.
Maybe my interest in debate comes from my own youth. I was sent to parochial schools run by Franciscans who valued debate and the clash of ideas. While I was aware that I caused no end of irritation to my teachers by my willingness to play devil’s advocate in just about any discussion, my patient teachers helped me refine rhetorical practices in writing and speech. One of my religion teachers lent me a book on Aristotle to help me get my reasoning act together! Reasoned discourse was valued in our lessons and appealed to my innate rebel and, while I left high school without the belief system they sought to embed, I did leave well educated.
I value the critical discourse of debate in my middle and high school classes very highly, in the first place because it causes participating students to learn a greater quantity of history. In order to argue effectively, one needs facts and to understand the relationships between events such as cause-effect. Having to improvise arguments, or even plan and compose them for that matter, causes the student to develop schema of information that is long-lasting. In the second place, I hold rhetoric in high esteem because it develops the kind of critical thinking skills so necessary in a democracy. Citizens who are too easily swayed by propaganda or who consume social media without a critical eye are less citizens than they are pawns of powers seeking to use them. I think I left teaching French back in ’04 because I did not feel what I was doing was important enough somehow… but that’s for another post…
The most basic type of debate is the “discussion style”. Teams sit across from each other at a table with a moderator at the head. They give timed, prepared speeches in turn and then engage in improvised cross-examination. The rules are fairly simple and many students came to really look forward to the debate. I required everyone to do this at first, but I soon learned it was best left as an elective unit capstone task for the willing and to offer other things for students who do not like public speaking.
Students who were fearful of public speaking needed rhetorical training as well. The next development was the online discussion. I coded an app at InnovationAssessments.com that worked especially well for moderated class discussions and streamlined the grading and scoring process for me. In every unit, there was an online discussion topic students had to address. Their assignment was similar to one I was given in some online college classes I took. They were to post their response to the prompt, giving two grounds for their position (sometimes I assigned them a position even if they held the opposing view). In step two, they were to reply in the opposing view to the student above them in the feed on the app. Finally, they were to go back later and reply in defense to the student in the class who offered them an opposing view. Once trained, students completed this assignment as a matter of course in each unit. During working periods in class, discussions would often erupt as students wrote and this was marvelous.
I gave formal lessons in rhetoric and identifying logical fallacies and I built a logical fallacy tracking function into the forum app so students could flag posts that contained logical fallacies.
I hold rhetoric in high esteem because it develops the kind of critical thinking skills so necessary in a democracy.
An Elective Course in Rhetorical Strategies
A few years ago, I had the opportunity to teach “Debating Current Events”, an elective course in different rhetorical styles and skills. The course was assigned to me in response to a discussion at the Board of Education level about how the school can try to foster greater understanding and act as a block to rising polarization in society. The course syllabus opened with: “Debating Current Events gives students the chance to participate in the great clash of ideas of our democracy. But it’s not really just about clashing ideas: it’s about forming understanding of the opposing view that leads to appreciation of diverse opinions. Students will examine the historical and social context of emerging current events. They will become skillful consumers of information, learning to carefully examine sources and to read critically. They will understand persuasion not only as a strategy in their own discussions but as a tool to understand and evaluate media communication.” I had six brilliant and engaged students whose opinions fell across the political spectrum. It was a great way to start each day.
My course materials for Debating Current Events are available for sale at my store. Units of study are also sold separately. Unit 1: The political spectrum, tolerance and toleration, Robert’s Rules of Order (I give the class a great deal of control over our work using Robert’s Rules. Unit 2: Aristotelian rhetoric (does not include training in syllogisms) and Logical Fallacies. Unit 3: Toulmin’s Rhetorical Method. Unit 4: Rogerian Rhetoric
The Mock Trial
By far the most popular activity in my classes was the mock trial. In this simulation, students are placed in a historical context as actors in a murder mystery trial they write themselves. I do not know what first gave me the idea to do this, but it was one of the first units I designed when I started teaching social studies. The unit begins with the election of judges and attorneys. The rest of the participants are randomly assigned to teams to role play as witnesses for the defense and the prosecution. I would seed the development of the historical fiction story we were going to write as a class by giving a basic scenario. The murder was placed at some time in history we were just learning about. Over three days, a story emerged, as attorneys and witnesses imagined their side of the story and the means, motive and opportunity were fleshed out. Judges completed an online mini-course in courtroom procedure during these days. The trial commenced on day four, with opening statements and the prosecution putting on its case. It took six to seven days to do this, so we only did one a year (although one year I had the opportunity to teach a half year elective class just in historic mock trials). Embedded in history, the stories we composed really stuck with us. Students would come back a decade later and comment on one of the trials we did and maybe some striking event on the witness stand. I invite the reader to return to the blog in the near future for a detailed account of how to teach an unscripted mock trial unit. To the point of this post, the four attorneys and three judges needed to be models for the class in the kind of evidentiary reasoning that I think we would all agree every citizen would benefit to possess.
In the 2019-2020 academic year, I developed a Model House of Representatives unit for my US History classes. Like the mock trial, this unit plan was “modular”: able to be set in any historical time period. Students were trained in a basic version of one of the debate formats used by the United States House of Representatives after being assigned to political parties based on a survey of their personal political leanings. Parties elected their leaders and the majority was set in alignment with which party was in majority in the year in which the session was taking place. Members drafted bills, spoke on the house floor in debating a bill, etc. Crafting bills turned out to be an extremely useful activity in and of itself. Students were assigned a problem of the time period and to craft a law that would address it. I invite the reader now to return to this blog in the near future for a more detailed account of how one to use a Model House of Representatives activity.
I have often wondered whether an alternative career choice for me would have been as an attorney. I have an interest in law and justice that no doubt influenced me. But beyond that, these activities bring two important elements to the course in social studies: a deeper knowledge of historical context and the ability to reason well. A positive side benefit is in the ability to spot propaganda in social media, a lesson to be addressed in a future blog, so do stay tuned!
The first time I sat in on the regional scoring* for the new New York State Global History and Geography Regents, the other scorers and I had conversations about the difficulties teaching the enduring issue concept to students. Some waited until the end of the school year to practice these, since the essay prompt draws on documents across cultures and eras and calls on the student to observe some patterns. They argued that this could not be done during the year because it required many time periods. I think there’s a better method.
Learning to write the enduring issue essay calls upon the student to read sources, identify some common issue (as opposed to a theme), and then select three documents to combine with their own recollections of historical context to relate their conclusions. Frankly, I think it’s an outstanding task. I love that it calls upon students to draw conclusions from evidence — to synthesize for themselves. The class discussions we had while practicing these were interesting as students came to observe patterns I would not have thought of and to defend them admirably. Waiting until the end of the year during review sessions is a bad time to teach students how to do this. Most all students whose papers I scored from other districts scored 2 out of 5 on these essays.
Bringing most of my students’ scores to 3-4 out of 5 came from committing ourselves to write one of these essays every ten weeks in Global 9 and 10. For those of you not familiar, in New York State students take Global History and Geography I in grade nine and the second half, part II, in grade ten. The state Regents exam only now covers the tenth grade course. I mention this because there is a temptation for Global 9 teachers to skip the enduring issue essay and wait to grade ten just before the Regents. Permit me to suggest that this is a mistake and a missed opportunity.
The enduring issue essay was 45% of a student’s score on my ten week interim exams. The first few in grade nine were heavily supported, as I coached students to bring up historical context to connect with the documents they chose.
The first challenge for students was to learn to distinguish a “theme” from an “issue”. “Movement of People” is a theme. The violent conflicts caused by movement of people is an issue. Students who have a more narrowly defined issue that societies have to address now have a clear direction for their writing. When students merely notice a theme, such as “power”, they fall into the trap of just proving that this was something that was a “thing” because it’s in the three documents they chose. It may seem like splitting hairs here, but it is a very important distinction and it matters in the quality of their essay (and therefore their score). The essays we wrote at weeks ten and twenty both included a lot of coaching on my part on formulating an enduring issue that was focused enough to lay the groundwork for an excellent essay.
The problem mentioned by my colleagues from other districts, namely that they felt they could not teach this until they had covered a lot of history and therefore not until the end of the year, is resolved by composing essay prompts only on the topics we have already studied. So on the week ten essay in Global 9 an 10, the documents I selected were only from the civilizations / time periods we studied at the time. The reader is invited to browse my online store for essay prompts designed for different points in the course.
The next challenge to overcome for novice writers was to avoid the temptation to merely summarize what the document says. This pretty much goes against all their reading experience to date, where teachers demanded they say what they read or answer questions on it in order to prove they understood. Writing this essay well demands that students draw upon their recalled background knowledge. The tendency for students to just summarize the documents was a very difficult habit to break. I came to advise them to spend no more than a few sentences in a paragraph to summarize the document and to spend the rest of the words in that paragraph to bring in background history to the document and to state explicitly how it supports their issue.
My goal throughout the training is to get students to write a level 3 paper. I realize I am tempted to try to teach everyone to shoot for a 5, but that is not reasonable. A score of 5 represents above grade level. A score of 4 is reserved pretty much for those who recall a lot of history. Despite all our best efforts, our students do not really on average recall a lot of history. So in my training I shoot for polishing a level of writing that is still above what I was seeing in the compositions of neighboring schools and that was do-able given the typical memory of your average student. Those students capable of the 4’s and 5’s suffered no disadvantage from this because, once they perfected the method of identifying issues (as opposed to mere themes), selecting and interpreting documents, and then bringing in historical context then all they needed to do to impress the raters was to dump a ton of historical knowledge in there.
This essay assignment is a strong feature of the new New York State Global Regents examination. It gives evidence of critical thinking and it promotes a rich classroom experience. Training in this should not wait for the end of the year and is best done throughout both grades nine and ten. The effort pays off and students often come to like this task, the latter being a surprising result of this training program.
* For the reader not familiar with scoring high school Regents exams in New York State, about a decade ago the state instituted regional scoring. They felt there was some kind of funny business going on when teachers were scoring their own Regents exams, so they mandated that we could no longer grade our own students’ work. So since I worked in a tiny district, I had to schlepp off to another school where I would grade their exams and they would grade mine. The security was absurd: I could not even handle my students’ papers, lest I be accused of foul play. For the essays, each student’s essay was scored by two teachers trained on a rubric and sample papers from prior field testing. If the raters disagreed in score by more than 1 point, a third rater was called in. The system has some advantages but in hindsight it seems a little unnecessary.
When I started teaching a long time ago, I never imagined that I would see a little laptop on each of my students’ desks and something like ChatGPT. For a little less than half my career, I taught French. Google translate never occurred to me as anything more than science fiction. Yet here we are.
Could I write an app that would grade my students’ summaries? The answer turned out to be a pretty decent “yes!”
Like all technological advances, we’re going to do it even though it wouldn’t be so bad if we didn’t. Human beings just can’t help themselves. Luddites smashed the textile machines that took their jobs, but innovation in manufacturing went on anyway. Self-restraint is not a human virtue we can ever maintain for long. Our inner drives, born in Paleolithic desperation, eventually will have their way. We cannot ban AI development because somebody, somewhere is going to do it and then those that didn’t will be at a disadvantage. All things considered, I think educators can look forward to the AI developments soon to be upon us with a sense of optimism instead of dread.
AI cannot replace teachers. One big reason is that AI cannot develop the kind of personal rapport with students that has always been the foundation youngsters need in order to learn.
My interest in artificial intelligence grew from my interest in computer programming. When I started out learning BASIC in the early 1990s, I built a program that would block vulgar words in my students’ data entry fields. We were using pre-Windows DOS machines, 286’s donated from a closing Air Force base nearby. The project occurred to me to try to make what I now know to be called a “chatbot”. I tried devising software that would converse with me in simple sentences and such that, if it didn’t know how to respond, it would ask me and then store that as an option for future response. The reader will not be surprised to find that this project did not work. In hindsight, I now know I was way, way out of my league in attempting something like that. Besides that, the computing power necessary for machine learning, let alone the troves of digital data needed to train an AI on, did not exist in 1994. But I am contented to know that I had the basic gist of the idea of machine learning that real engineers would eventually put to use.
When I submitted the algorithm-generated summary into the AI grading assistant, which evaluates it based on comparison to human-composed ones, it scored 100%. Every. Time.
About five years ago, I started exploring the idea of automating some of my grading. I was teaching social studies and I would assign summarizing as the way students were to process textbook articles. I am convinced this is a far better method that having students answer questions on text they are reading. The problem was that I had about a hundred students across six different grade levels. The beginning of a unit would generate several hundred summaries a week to grade. Could I write an app that would grade my students’ summaries? The answer turned out to be a pretty decent “yes!”
AI-Scored Summaries
The AI grading assistant here at Innovation Assessments was trained on 500 human-scored summaries. The algorithm looks at eleven features of the text and compares it to the same text features of up to seven other models of summaries scoring 100%. These features include things like a Flesch-Kincaid readability measure, word count, common proper nouns and verb phrases, and statistical comparisons like cosine and Jaccard similarity. Before analysis, the app removes stop words, reduces words to their root form (lemma) and reduces many words to a common synonym (so the app can understand ideas written in slightly different wording). The method I used to establish the scoring algorithm was to chart these comparisons in a spreadsheet and adjust them until the AI scored the work about as I would have most of the time.
I was very pleased with the results on this. The scoring of student work became a lot faster. The app brings up the AI score estimate and I can check it to confirm. This is why I call it an “AI grading Assistant”: it still needs a human supervisor. As time went on, though, I came to trust the app more and more. When I set up the assignment, I would enter my own summary of the target text from the start. Once students completed the task, I went first to score the work of students who usually get 100%. I could add up to six of these to the “corpus”, which is the body of model text the software uses to judge. The next step was to run the AI grading assistant on the work submissions of the rest of the class.
The scoring of summaries in this way required one or more human-composed models. Next, I wondered whether I could write an algorithm that would summarize a text. I am not able to write code that can write “in its own words”. Instead, my little bot mainly extracts the first sentence of each paragraph and then some selected other sentences verbatim if they meet certain criteria (such as the presence of key words identified by frequency in the text). I had my doubts about how effective this would be. Surely, it would lose some important meaning sometimes since it was a formula and not really “reading” like a human would. Well, get this …
… When I submitted the algorithm-generated summary into the AI grading assistant, which evaluates it based on comparison to human-composed ones, it scored 100%. Every. Time.
AI-Scored Short Answer Tests
Another challenge of teaching social studies with a lot of reading and writing was the large volume of grading student work in the form of short answer tests, particularly document-based analyses. Could some similar software assist in scoring short answer tests?
The app development method was about the same: I had hundreds of student work samples to analyze. Using some similar methods as for grading summaries, the new app allowed the teacher to add up to five versions of full-credit answers into the corpus for comparison. One feature that was not examined in the summary AI grading assistant was the degree to which a student’s writing was analytical (as opposed to merely descriptive). This project went fairly well – well enough for an amateur programmer and accurate enough such that the short answer scoring was a huge help to me. Click here to read more about development of an algorithm to measure the degree of analysis in a student writing sample.
The AI-assisted scoring of short answer tests was most successful at evaluating responses that had a limited range of credit-worthy answers. The AI performed well for questions like “What caused the fall of the Roman Empire?” The AI did not perform well on questions such as evaluating the reliability of a primary source, since the range of possible correct answers would have required a lot more models to train on than the five the software allows. Nonetheless, the short answer AI grading assistant saved me tons of time. It allowed me to maintain a teaching method that was very time consuming by lightening the workload so I could spend my time in curriculum development.
Opportunities for AI to Coach Students
I came to have so much confidence in the AI grading assistant that I built in access for my students. Students composing their summaries at InnovationAssessments can access the coach, which gives them a pretty accurate score estimate while they write. This take a little mystery out of “how am I doing?” and helps develop strong summarizing skills. That’s reading comprehension and basic composition.
The AI grading assistant is also an effective coach in short answer exercises. Enabling the coach for a practice run at short answer tasks permits students to have instant estimates of the quality of their work submissions and the AI offers little hints and suggestions drawn from the corpus of model answers on which it was trained.
We’re Not Being Replaced Yet
AI used in the way described here did not replace me. It still required supervision. I would assert that it enhanced my work, allowing me to use a better teaching methodology that was not very practical otherwise. The way I wanted to teach was really a recipe for burnout in the context of my particular teaching job. Assigning three summary tasks to a hundred students over a two week period, well, the reader can do the math. AI assisted scoring let me do the best job I could without burning myself out. That is a great reason to continue AI development and research, even for amateur programmers like myself.
There is a very solid reason why AI will not replace us. AI cannot replace teachers. One big reason is that AI cannot develop the kind of personal rapport with students that has always been the foundation youngsters need in order to learn. AI cannot form emotional bonds with people. If the day ever comes that it can do this, then we have something more than intelligence that is artificial, we will have a consciousness.
