Training the Innovation AI to Help you Grade Video Summaries

Teachers use video lessons a lot these days, some they create themselves, and other videos that they find on Youtube. Not everyone has time to build a set of cued comprehension questions for each video lesson. Some teachers assign their students to summarize what they saw in the video. This is a fantastic way to keep kids engaged in the video, but it’s a lot of work scoring them.

The Innovation AI grading assistant is the perfect solution. In this post, I would like to show you how to generate a video lesson in Innovation and then quickly train the AI to help you score the summaries very rapidly.

Quick Links to Short Video Tutorials

A Word about the Innovation AI

There are two broad categories of AI: those that work using a complex algorithm and those that work using large language model learning. The Innovation AI is of the former type. The latter, the machine-learning AI, is exemplified by ChatGPT and the Bing AI under development right now. They “learn” by analyzing vast amounts of data across the internet. The Innovation AI is trained on five to seven models that the teacher provides for comparison.

I developed the Innovation AI to help me grade summaries and short answer tests. When I taught social studies, I often assigned summaries of texts instead of comprehension questions. It works by comparing the student text to a number of models and scoring the comparison on a dozen features. These features include measures of similarity like cosine and Jaccard, as well as readability, number of words, level of text complexity, and so forth. The scoring rubric was designed using 500 of my students’ work submissions that I had scored manually so that the AI essentially grades as I would.

The Innovation AI is highly effective for helping you score summaries and short answer responses where the range of possible answers is fairly limited. The AI does effectively recognize different ways to say the same thing using natural language processing algorithms.

Training the AI

When you train the AI, you give it model answers to use in the comparison algorithm. For short answer tasks, the limit is presently five models. For writing samples such as summaries and compositions, you can store up to seven model answers.

The process, in summary, is this: (1) Compose your own summary of the task or let the Innovation AI generate a summary from the source text for you; (2) Manually score the work submissions of your students who usually get full credit. When you find a submission to which you would award full credit, ask the AI to score it. If the AI cannot recognize it as a full credit answer, you “add it to the corpus” of model answers. The next time you ask the AI to score a student submission, it will compare it to each of the models in the corpus and award the highest score earned by the student in those comparisons.

Since many of us reuse our assignments from year to year, you really only have to do this once. I trained the AI on most of my Global 9 and Global 10 assignments in 2018 and just continued to score with those for several years.

Creating an Inbox for a Task with Embedded Video

  1. Select the Inbox button from the new course playlist element dashboard.

2. Enter the title and some optional attributes. Paste in the embed code from the video you want students to watch and summarize.

Embed code from youtube.
Paste in the embed code.

3. Once created, you can click and drag the element to its right position in your class playlist.

Students Save their Summaries

When students access the task from the course playlist, they will see the video you embedded and the space underneath to compose their summary.

AI can coach students to write better!

The AI grading assistant can be engaged to coach students along the way. As they compose their summaries, they can periodically click the “Coach” button to get an estimate of their grade so far. In my experience, this promoted student prolonged engagement for a better work product.

Teaching Stimulus-Based Multiple-Choice for Document Analysis

The stimulus-based multiple-choice test item was introduced into the New York State social studies Regents examinations starting in 2019 for Global History and Geography II and for United States history. The task poses challenges for students such that it merits some regular, focused training throughout the year.

In a stimulus-based task, the student is directed to respond to a document, map, or image using their ability to analyze and their knowledge of historical context. In the case of the New York State exams, there are eighteen “task models” used when designing questions. For example, a student may be asked to evaluate and classify (identify) best use of a source or to respond based on knowledge of historical context. Principles of reliability assessment are applied here, such as when students are asked to identify point of view, purpose, context, bias, format of source, location of source in time and/or place, and/or intended audience of sources using background knowledge.

Click here to shop stimulus-based tasks at my store for grades seven through eleven social studies.

The first important habit of thought to train students to engage is to think beyond the document. Habit since their first reading lessons has asked them to find the answer in the text somehow. It takes a lot of practice and reinforcement to get students to activate their schema on the topic; to think of the story of which the document is but a fragment. The question cannot be answered without background knowledge.

  • part I of the Global and US History Regents
  • 25-30 questions
  • primary or secondary source documents
  • M-C questions are always paired with stimulus
  • primary or secondary source
  • maps
  • charts
  • cartoons
  • may have more than one stimulus tied to it
  • no more stand-alone questions
  • estimate 30-45 minutes for this part of the exam

I used almost exclusively primary source documents for my stimulus-based tasks. This can be challenging for weaker readers, but with practice in skills for addressing difficult texts, this obstacle can be addressed.

My custom was to assign a stimulus-based multiple-choice at the end of every unit starting in October. At first, students find these very difficult. I use a z-score standardization procedure to adjust the scores so as not to bomb out their GPA while they are just learning. Click here to read up on standardized scoring. It is a great way to score students in tasks they are not yet proficient at.

  • Practice! Students are generally not good at these at first. 
  • Read the question first. be certain you know what it is asking.
  • Remind yourself that the answer is rarely found in the document itself.
  • Identify the historical time period the documents go with.
  • Consciously call to mind the historical context of the document before you read. Try to recite to yourself who, what, when, where, why of the time period.
  • Use process of elimination to narrow down the options.

Some of my students always would wonder why ask questions this way. If the test writers want to know whether a student knows something, why not just ask? I don’t have a good answer for this. I strongly support instruction in social studies that calls upon students to think critically and make meaningful connections with knowledge. I also think that students should actually possess knowledge. This assessment method was no doubt inspired by AP history exams. It remains a question in my mind as to whether this level of complexity is necessary for an instrument for secondary school evaluation. If we want to know whether the student knows what caused the French Revolution, for example, maybe we should just ask them that?

Teaching the US History Regents Short Essay

The updated New York State Regents examination in United States History and Government, part II, is a short essay task designed to measure students’ ability to work with historic documents. It is a mature version of the “CRQ” found on the tenth grade Global Regents. Students are called upon to understand text, engage it with historical context, and assess a text’s reliability.

In document set 1, students describe the historical context surrounding two documents and identify and explain the relationship between the events and/or ideas found in those documents (Cause/Effect or Similarity/Difference or Turning Point).

Turning point is always the most challenging for students, mainly because it demands a strong knowledge of historical context which only the higher performing students usually possess. In stating similarities and differences, it is important to stress to students that this should be a substantial feature of the two texts, not trivial. For example, some students may respond something like this: “Document one is a cartoon and document two is a newspaper report”. This is trivial and should be discouraged. For cause and effect analyses, remind students that some events may lie outside the documents at hand, so they may need to rely on their historical knowledge.

Click here to short for short essay prompts at my store.

Document set 2 asks students to describe the historical context surrounding two documents and (for one identified document) analyze and explain how audience, or purpose, or bias, or point of view affects the document’s use as a reliable source of evidence.

A good strategy for ensuring students possess the skills to address this task is to be certain to assign one every month or so throughout the year, followed by a debriefing where the class can study their classmates’ work (anonymously) and develop strategies for improvement.

Weaker readers are particularly disadvantaged in this task, although since the test items are field tested before administration it is likely the field testing will mitigate some issues with the difficulty of reading some primary source texts. Students can be taught compensatory reading strategies to help deal with difficult texts.

As always, the challenge is to ensure that students have learned a strong body of historical context. That is, the best marks are reserved for those who actually recall the history and who can analyze it (cause-effect, turning point, etc.) This is best achieved by regularly administering quizzes on historical knowledge. I like to give students time in class to study for these. The apps here at Innovation Assessments are especially suited to that end. A lot of social studies assignments can tend to be just look-it-up and transfer kinds of exercises without real demands on students to remember. This is an easy instruction error to remedy.

Teaching factors affecting the reliability of sources is another matter. This takes a great deal of time and practice and, I would argue, is of upmost importance for a person’s education in this day and age. Students, for the most part, do not intuitively analyze and explain how audience, or purpose, or bias, or point of view affects a document’s use as a reliable source of evidence. They tend to take what we give them on face value. It is important to teach students to think critically and approach all historic documents with a healthy skepticism.

Click here to shop my store for primary source essay prompts for grades seven through ten.

I think teaching reliability should begin young, down in middle school. Engaging students with documents that have very vague reliability weights is a good practice. In a debriefing after the task, it is useful to anonymously display some student reliability evaluations for all to see and to discuss. It is important to do this regularly, starting off right at the start of the school year. There are really no stock phrases that students can learn by rote for this, given the variety of context and source material. I had the benefit of working in a small school where I had the same students grades seven through ten or eleven, so I could implement reliability assessments early in my program. In larger school districts, it would be good to consider a commitment to reliability factor training from an early age. I assigned one longer primary source to analyze each topic (so about once a month). This short essay included an extended analysis of reliability in a conclusion paragraph. The training paid off and when my eleventh graders were preparing for the Regents in US History and Government, they had little difficulty with reliability factors.

A good piece of advice on this is to assign students to do this every month in grade eleven. I suggest assigning it as a test each time. Coach students on the historical context they have to memorize in advance. Lots of teachers assign these for homework, but this entirely misses the point of such training. Student independent work practices are highly efficient in applying the minimal effort to a task, including copying their colleagues’ work or copy-pasting from a source. If they are not doing these without notes, they’re not really practicing.

If you are afraid to assign your students this as a test because they are not likely to do well at first and don’t want to bother their GPA, I recommend using standardized scoring. You can use the z-score calculator here at Innovation Assessments. Use 78 as your standard mean and 14.8 as your standardized standard deviation. Read more about standardized scoring here and where I got those figures. The beauty of this system is you can apply this to their grades every month and as the class improves, as the class average approaches the standardized mean (78 in this case), then the algorithm affects their scores less and less.

Once you have scored their papers, select out some problem responses for class discussion in a debriefing. Keeping the responses anonymous, review how to improve the answers next time.

The short essay is 14% of the Regents score, so for passing the text it’s a good practice. But I would suggest that this kind of work as a regular lesson is extremely valuable as an educational tool. Studying primary sources rightly should take center stage in our social studies lessons. It is not just teaching to the test to do this. It is developing a critical thinking skill set and insisting on recall of historical context that are the values here. Remember, that the highest valued performance is that which is based on a substantial recall of historical context!

The Primary Source Essay for Secondary Social Studies

Permit me to introduce you to one of my favorite assignments … and one which my students actually liked as well!

I had made a commitment to using primary sources as a central feature of my social studies units when I started teaching social studies in 2004 after switching from 13 years of teaching French. Maybe I was influenced by my training in teaching world languages, which promoted a value for “relia” and authentic documents in teaching language.

This task became a truly high-value asset in my lesson plans. Students became good at them and many would request one to do as a capstone task in a unit. … The critical thinking skills fostered by this kind of writing task were important building blocks for the kind of critique that they would in more advanced courses learn to do.

My first primary source analysis assignment, devised around 2005, looked a lot like a tax form. Students were guided through a series of mental tasks to analyze the source they were given. I managed to find a copy in my archives and it is linked below.

Looks like I created an “advanced” version as well:

Grading these really felt like I was some kind of IRS agent doing an audit. The task evolved to become an essay task. This made more sense in a lot of ways. For one thing, students were going to be graded on standardized tests based on their performance on essays. Regular essay writing was gradually becoming an important centerpiece of my courses through 2006 and 2007.

By 2016, the rubric for this task and its procedure had become finalized and fully developed. The rubric is linked below. In each unit of study, students had an extended primary source essay to examine. I assigned this even in grade six, although a shorter composition was expected for younger students. An example of a ninth grade version of this task is linked below, entitled “Journey of Faixan to India”.

  • A source citation is given first in a modified format based on a style used by genealogists.
  • A brief historical context is given to guide students in what they should consider about the time period. This is explicitly not to be used in the composition itself.
  • An essay organizer is given next, stating explicitly what goes in each paragraph. The essay begins with a description of the source and its audience, followed by some relevant historical context and then a summary of the source. The final paragraph is an assessment of two to three factors affecting the reliability of the source. This was the longest paragraph and of great importance.

Students did well on these, possibly because they did one every month through all the years they had me as a teacher (which for some could be four years in a row). These were never homework — in fact, it was not allowed to do them outside class at all. During class working times, impromptu discussions developed around reliability issues and especially in global studies around translation issues. The absolute importance of understanding historical context became evident to them and I believe my students became adept at assessing the reliability of sources.

This task became a truly high-value asset in my lesson plans. Students became good at them and many would request one to do as a capstone task in a unit. The impromptu discussions that came about in class working periods were key to developing understanding. Applying the historical context to a lengthier primary source text fixed important chronologies, relationships, and turning points in students’ minds. The critical thinking skills fostered by this kind of writing task were important building blocks for the kind of critique that they would in more advanced courses learn to do.

So here was my experience doing remote teaching in the pandemic…

I have been procrastinating writing this post. Probably because, like many of the readers who taught online during the pandemic, this is not exactly a fond memory. However, I have two reasons to move this task to the top of my to-do list. Firstly, I am aware that it has already been three years and I don’t want to forget anything important. Secondly, I will be teaching a course online next year and I want that to be a highly successful enterprise. I think there must be lessons to learn on reflection from teaching secondary level students online.

Pre-pandemic Experiments in Remote Teaching

My first foray into teaching online was a pilot online class I taught in 2016-2017. It was called Virtual United Nations. My school was a small, rural Adirondack district with limited offerings outside the core subjects and remediation. I was active on the technology committee and we found distance learning, as it was envisioned in the 1990s, was not going to work for us.

I had four students in the pilot course. It was an asynchronous course, the final project of which was participation in a regional high school United Nations simulation. For me personally, this course had the dual purpose of satisfying a graduate course requirement in digital teaching and learning through Empire State College. My paper is attached below.

Pandemic Remote Teaching for Me

At the time the pandemic hit, I was teaching social studies for grades 6-10 and a French class for grade 6. Schools in New York State closed 13 March 2020, supposedly for two weeks. I did not set foot in the school building again until September.

Like most schools in the region, our students all had Chromebooks. We scrambled to send middle and high school students home with these and chargers. Our plan evolved somewhat from its start, but eventually we settled on mostly synchronous remote classes that followed the same daily schedule as our regular day. At the end of each week, we were required to submit a summary of the week in our remote classes, including what we covered, student and teacher morale, and participation rates. I was able to search my email sent box to save copies of my reports from 13 March to 12 June 2020.

Lessons Learned

Here are some highlights of what I learned about online teaching from my weekly summaries in 2020:

  • 24 Apr I prefer delivering prerecorded video lessons to the live video. I think the live video sessions may turn out to be more PR than substance for a lot of reasons. I like them, but we’ll see. I embed my prerecorded video sessions with questions. 
  • 01 May It makes me feel happy to interact on video with [students]. I am still just learning how to conduct video classes. This week, I make them show their faces and I call on them to respond.
  • 8 May Sadly, my little Zoom “working together” idea did not work – no one showed. The streaming lessons are the highlight of my week because I get to see the kids, but I only get 6-8 now whereas I was getting the majority the first week. 
  • 08 May Education for people under ~17 needs to be a social, in-person experience and the governor’s “re-imagined” education might fail to see that. We are still primates, after all, and education requires the physical presence of teachers until upper high school. 
  • 15 May Participation in live streaming classes is very poor even with a reward system. I only get 3-6 kids out of classes of 20. These are among the highlights of my week, though. I put together nice presentations and I like seeing them.  
  • 15 May I got good participation in a survey asking successful online learners to share their secrets.  
  • 15 May I probably did not need to shorten 8th and 9th grade curriculum as much as I did. 
  • 15 May I have found breaking the tasks down to smaller steps so there is some small thing to do each day a highly effective practice for remote learning. 
  • 12 June  Breaking tasks down into daily 25-30 minute bits works great and I’ll do that in-building now too 
  • 12 June   “Remote learning” is NOT “homeschooling” and I worry that we burden parents too much. 
  • 12 June I think about 40% of my students need to be in school with a person teaching them. 

The paper “Successful Students Share Their Secret for Online Learning” is linked below. This was an extended survey of 21 (out of 87 total students on my roster that year) of my most successful students in remote learning during the pandemic. You can see for yourself the good advice they shared and some interesting survey results that may not be very surprising.


Overall, participation actually remained fairly stable, especially in grades nine and ten. I did a study in May of task completion rates of my freshmen and sophomores since 2017 and found only little changed in the pandemic. This was very surprising, since subjectively, it felt like it was a real struggle to get work in. I gave little assignments and took a grade a day. That’s how I teach in-person too, actually. The paper linked below, “Task Completion Study”, also includes a fairly comprehensive description of how I taught online. Many of those practices I retained when we returned to in-person and we had Chromebooks in class.

Remote Learning Conditions

The curriculum and assignment length were modified under remote learning. Assignments were decreased in size and duration. I avoided assigning multi-day tasks because many people just wait until the last minute to do it all, to poor effect. Students were also getting graded on “lectures” now, which consisted of 12-15 minute video tutorials embedded with questions. Some of the few multi-session tasks I normally assign in class, I now broke up into smaller assignments. The unit reading task, normally four pages to read and process, was reduced to three . In consequence of the setup of remote learning assignments, I did not complete the curriculum in grade nine to the point in history that I usually do and I moved some topics to the second part of the course to come in grade ten. I actually had more teaching time for grade ten under remote learning because, with Regents exams cancelled, I now had twelve extra lessons since I did not have to stop teaching and do review. There were on average 45 assignments across grades nine and ten in these courses over the date range. The 15% jump in the number of assignments under remote learning is because the tasks were broken down into more numerous, smaller steps. This cannot be interpreted as an increase in workload.

Resources in my online courses were curated through my own website, Assignments were organized and posted at Students were expected to complete one social studies assignment each school day. Courses were mostly asynchronous, save for a 30-minute live streaming lesson once a week which was optional and non-graded (there was a reward system built in for attendees). Video tutorials with multiple-choice questions ran 12-20 minutes, averaging around 14 minutes. Students were invited to attempt these twice for a higher grade. Textbook reading tasks consisted of either three- or four- page reading selection to process or multiple-choice comprehension questions. Each unit has an essay on a primary source where one session was reading and outlining the important points and one session was to compose the essay. There were a number of multiple-choice and matching quizzes which usually take about five minutes exclusive of pre-study time (students can rehearse the questions and answers in advance of some of these, so it’s very easy). Other assignment types follow similar patterns. In my opinion, a good estimate of the time commitment for the average student would be about 25 minutes a day on social studies in grades nine and ten under remote learning for the date range. I charge a late fee of ten points per day late just as in regular school and if students contact me with good reason or if students work to catch up a lot of work in a short time span, I always waive these late points.

Morale: Theirs and Mine

The short story on student morale as near as I could detect it was that it was always good. None of my weekly summaries reported adverse student morale. However, as the year progressed, some students dropped out of participation so I can only report on the students who were still involved.

My own morale story is more complex. In my memory of the events, it is all negative. But reading the contemporaneous notes of 2020, I see that it started out just fine. On 20 March, I actually wrote “My week went great. I absolutely love working from home. If I could, I would switch to just doing this.” Ugh. By my last entry on 12 June, I wrote “I endured the most serious aversion to teaching I ever experienced. This is not because it was online. I can teach online just fine. The reasons were complex… no room here… but I am getting better and look to the summer and next school year with a sense of optimism and hope.

The problems were not really connected to online teaching…

When on 12 June 2020 I wrote “The reasons were complex… no room here… ” I was alluding to a collection of things that were aggravating in the extreme. This only got worse in the coming school year. The things that made working in my district an aggravating and unpleasant experience are largely beyond the scope of this article, but these things, they were born of the pandemic. My school district imposed actions, directives, and policies which ranged from the insufferable to the outrageous. This trend gradually broke my working relationship with my district and inspired me to hasten my retirement and to request part-time work in my last year. Though I was not one of the 16% of the faculty who resigned or retired early that year in my district, I was aiming to get out as soon as feasible.


The pandemic fostered a large number of innovations for my teaching in a digital environment. One issue of note, which I would like to address in a separate post, is the issue of online testing. Is it possible to offer valid, reliable assessments in a remote teaching situation? I am convinced that this will improve in the coming decade, but there is a lot that can be done now.

A few of the innovations that were inspired by the pandemic remote learning situation involved devising ways to give good online tests. My district subscribed to a service called “GoGuardian”, which allows me to view and control student screens. I cannot emphasize enough how important this was. But at my own site, I developed a number of useful apps. One was a parent proctoring system. Parents needed to enter a code to let students take a test. Another was a sort of AI proctor I developed in JavaScript that tracked student activity online while they worked.

I wrote a short paper on this experience. It is linked below. With the help of my colleague in the math department, we established that an examination of student scores on online tests showed no real change from performance in the classroom, thus supporting with this small study the idea that secure online testing is possible.

I am reminded now of a student who was taking one of my “credit by examination” tests. While he was writing his essay, I was able to watch his screen and my own app would prevent him from leaving the test web page in order to look something up (to cheat). I watched him type sentences I knew were not likely his — I discovered he was using his phone. Easy catch. I typed in his sentences into Google search and quickly located the web page he was copying from on his phone. It was easy to award no credit for the task and provide the very websites in rationale where he cheated.

Teaching in the pandemic for me was an adverse experience, though not for the reasons one might think. I believe it would be a good effort toward justifying the adversity if some lessons can be gleaned from my patient endurance.

What do you think about listening to music while studying?

In the minutes of my school’s Board of Education from a meeting in the early 1980s, I remember seeing a discussion about the dangerous effects on students of listening to loud “rock music” in headphones. If only they knew how technology would advance! In my last year before retirement, technological advances had become such that many of my students walked around all day with a tiny device like a hearing aid in one ear. Long hair could hide them in classrooms where they were banned. Many young people spend the day serenaded by their favorite playlists and having their texts read to them. Like many of you, I wonder if this is good.

In 2015, cell phones were starting to become ubiquitous in my small, rural school. Students wanted to plug in and listen to music while they worked. I indulged this at first to see what would happen and I ended up banning the practice. But some of my high performing students were convincing in their pleas to listen to their tunes. I was curious enough to read up on the research.

Naturally, the answer is not simple. Sometimes it helps, sometimes it doesn’t. There are so many variables to consider that are missed if one only phrases the question “will listening to music help studying?”.

I settled on this policy for my classroom: I would grant a student a “license” to listen to music during working times in class provided that (1) they read and summarized adequately my own research paper on listening to music while doing school work and (2) that they maintained a class average of at least 85.

I wrote that paper on listening to music while studying while listening to music. I admit it. While I am working, I like to listen to music. If I am doing labor like painting a room or stacking wood, I like the 90s pop/rock playlists on Pandora. When I am doing academic work, I sometimes play something classical. Any of my former students reading this will no doubt roll their eyes because I probably did not allow their tunes during the lesson.

Typically, the weaker student with attention-focusing problems suffers from music during academic work. They engage with the lyrics of the song and the beat of the music in a way that adds cognitive load to their school assignment and impedes working memory. More than once I saw a kid air jamming guitar while summarizing a text on some world revolution. They interrupt their work to skip a song they don’t like, to choose a different playlist, or to search for the song they just realized would be great to hear right now. *Insert eye-roll here*

But some other students do seem to perform better on tasks and to focus better with the music on. In fact, so do I.

You can manage badges and permits for your class at Innovation Assessments.

We know now from mountains of research that human beings cannot truly multi-task. I would suggest that, in situations where music is not helpful for study, it is better sometimes a separate task that takes a student’s attention away from the one they probably don’t want to do anyway. In the paper, I suggested that it was a question of cognitive load.

For the student who benefits from this, the music becomes a part of the task just like a fiddler’s bowing is part of the overall task of fiddling with the left hand fingers on strings.

A problem I was never able to resolve was this: I wanted to create some test to see who would benefit from music and who would not. I actually had very few students ever apply for the license to listen to music in class. They were intimidated by reading that paper, I suppose.

In the end, I guess the question was not resolved satisfactorily. I still banned music. Some kids still sneaked it in. It didn’t matter much. Hopefully, one day someone clever will develop a test to see who benefits and does not.

You may forget your French someday, but you won’t forget what you used French to learn about.

In my grandparents’ day, we’re talking the 1930s, students who took world languages in high school were really aiming for a reading knowledge of the language. College-bound students of that era would be ready to read sources in their original language perhaps, and not so much ready for travel. In the hand-wringing self-criticism of the 1980s and 1990s, there came a growing concern that world language courses were not “communicative” enough. Students lamented that they took four years of such-and-such language and couldn’t actually speak it.

So in the spirit of reform, when I was being trained as a language teacher in the early 1980s, we were baptized in the holy water of communicative proficiency and realia.

I wish to demonstrate that teaching culture using the target language is an effective means of getting students to communicative proficiency.

But here, decades later, I have to seriously question this focus for students after the second year in a high school world language course. My reason is pretty simple: despite the great reform of those decades and the sincerest efforts to produce good communicators in the target language, world language high school alumni still rightly complain that they cannot really speak any of the language they learned. The criticism still stands.

We all seem to just accept this and the way it goes. World language teachers have lobbied for years, successfully, to promote our stock and trade. Among the benefits we can tout are an enhanced understanding of other peoples, possible career opportunities for those who develop proficiency, improved reading and writing competency in their own language, and so forth. What we cannot say is that our charges will retain the skills we taught them. Even my French four students came back years later to visit with their once-honed language skills dissipated completely. The fact is, one cannot develop linguistic proficiency in the traditional classroom.

The main reason to do this is that learning culture will stay with students longer and enhance their overall education more profoundly than learning a language they will most assuredly mostly forget.

I want to promote a way to teach world language in high school years three and four that makes learning another language highly satisfying while at the same time achieving whatever basic linguistic proficiency is possible in the traditional high school classroom.

Click here to shop for French civilization units in my store.

Everyone includes culture studies in their world language classes, especially at the higher levels when the language skills are enough to make many authentic culture forms accessible. May I propose to make this culture study the centerpiece of year three and four studies and that it is indeed possible to do this at level three with the right scaffolding techniques. The main reason to do this is that learning culture will stay with students longer and enhance their overall education more profoundly than learning a language they will most assuredly mostly forget.

The correlation between the quarter marks’ average [in a culture course] and the French Regents Examination [a measurement of communicative proficiency] average score was 0.70.

In the early 1990s, I renamed my French III course “French Civilization”. I made culture studies, with thematic topics determined by students, the centerpiece of the lessons.

Can you really teach content and still get kids ready for the Regents?

Let’s look at the data. I taught French from 1991 to 2004, then a few courses from time to time, and finally again in 2022-2023. I saved my final grade sheets since 1993. I wish to demonstrate that teaching culture using the target language is an effective means of getting students to communicative proficiency. I think the data backs up my claim. I have analyzed my students’ grades in French Civilization (what I had entitled my third-year course) from the 1994-1995 school year through the 1999-2000 school year. During that six-year period, I taught 86 students third-year French as a civilization course.

The data support this method.

The class average on the Regents during that six-year period was 80. The average of the quarter marking period grades for those students during that time was 82. The correlation between the quarter marks’ average and the French Regents Examination score was 0.70. The spreadsheet is available below. This strong correlation supports the idea that you can teach culture and build language proficiency. I submit that this method is better because of its long-term benefits to students in teaching them content much as they would learn in a social studies course. This is a life-enriching educational practice that still meets the communicative goals measured by world language examinations such as the Regents examinations in New York State.

How do you teach language and content?

Scaffolding lessons designed to make authentic culture materials accessible to students are in the wheelhouse of every language teacher. A complete training program is beyond the scope of this article. But a few worth mentioning are these: easier versions of classic texts, pre-teaching key vocabulary, training programs to help students construct meaning in a sea of unknown words, and surely coaching in the native language as students tackle the tasks. A culture course where the class chooses its topics of study will enjoy some motivation from interest alone. Good scaffolding lessons let teachers bring resources to accessibility and help students build confidence.

Here’s a lesson in reading skills for English that can easily be used in reading world language. Actually, I built this lesson from my world language teaching training and elementary ed. training.

Student interest becomes a driving force.

Most students who advance to this level of world language learning are interested and usually are academic-minded. The freedom afforded to choose their learning seems ideal in a way Rousseau would appreciate. They’ll need to be brave to tackle plays written in the target language or try to understand a classic film. But my experience has been that interest and choice are really strong motivators.

Students who have complained that they don’t really end up very fluent from high school world language courses have found little comfort in the rationale provided by adults around them. It’ll get you out of college courses. It’ll make your transcript look good. It’ll help you order a meal at that French restaurant in town. Your English gets better. You’ll learn about other cultures. You’ll learn grammar. And the list goes on… But emperor has no clothes, and the band keeps marching him through town.

For third and fourth-year students in high school language classes, the data support the idea that culture classes lead to the level of communicative proficiency measured by standardized tests like the New York State Regents examinations. The level of proficiency students can actually achieve is fairly limited and maybe we should be more up front with students about that; set more reasonable expectations. Culture and civilization classes enrich a person’s education, a person’s life, in ways that just teaching them to ask when is the next train to Madrid won’t do.

Building World Language Speaking Skill with the Scaffold Dialogue

So let me go way back to my training to be a French teacher in the late 1980s. My sponsor teacher was Mr. Tom Ham at Potsdam High School. He was instrumental in teaching me techniques for developing conversation skills by “scaffolding” students up into more sophisticated expressions that had an element of improvisation within the proximal zone of development of the class.

Jump to The Scaffold Dialogue Package

Shop for Scaffold Dialogue products at my store.

French poses some difficulty for complex expressions that students of, say, Norwegian, don’t face. French has lots of silent letters, for example, and the sound that the letters represent is more alien to English speakers. “What’s this?” is “Qu’est-ce que c’est?” which, if you don’t speak French, sounds like “KESKUH-SAY”. Holy cow! Lots of extraneous letters. Languages like French are best learned when students have a lot of repetition in these difficult structures. The listen-record task seems to fit the bill.

The Listen-Record Task

Mr. Ham used a lesson that didn’t have a name so for lack of a better term, I call it the “listen-record”. Students are presented with a few model sentences with parts missing. The missing parts are improvised by using our current vocabulary list and prior knowledge. Each student goes, in turn, to ask a question, choose a classmate, and then the classmate creates a response. Everyone listens and records in the table the name of the respondent for each exchange.

At the end of the lesson, there is a quiz on the table during which I ask, in the target language, who did or said such-and-such. students look up the name in the table and record the name on the answer sheet.

I echo out everything everyone says repeatedly, so this enhances the experience and recall.

Listen-Record table with quiz on back fold.
A student’s completed table.

I really like this activity. Students have to be engaged all the time. They have to listen and practice speaking. They often improvise amusing permutations of the responses and this lends itself to long-term recall of the vocabulary. Best of all, this activity presents the same model phrases over and over. I echo out everything everyone says repeatedly, so this enhances the experience and recall.

“Guided Dialogue”, “Guided Conversation”, “Scaffold Dialogue”

These are all terms I have seen for a slightly different activity designed to build conversational competence with some improvisation. In the guided dialogue, the student participates in an extended conversation where every other exchange is in the target language. Back in the early 1990s, I found a British product for teaching French that was interesting. There were two books, an A and a B. Each chapter presented a different half of a conversation with necessary supportive materials like vocabulary and phrases. The activity was designed to have two students practice conversing.

A British communicative activities book I used in the early 1990s.

Shop for Scaffold Dialogue products at my store.

I tried on and off to use this during the time I taught French, from 1991 to 2004 (I taught social studies from ’04 to ’22). I could never get the students to commit to remaining in the target language throughout and that was the main reason I abandoned the activity. But I still wanted some kind of guided conversation that included scripted elements and improvised elements. I needed this to be topical so I could include it in a thematic lesson.

Reading and listening are important, but students who do too much of this and not enough speaking and writing without assistance end up notreally being able to use the language they toiled so hard to learn.

The Old French Regents Examination

Screenshot from the 1985 French Regents part 2.

Those of you not in New York State may not know about our state testing system called Regents exams. Back in the 1980s, the old exams had a part that I liked but which was eliminated in the new exams in the 1990s. In this part, the teacher read a setting in French twice, then something in the target language to which student were to respond. There was a prompt in English telling students in a general way how to respond.

I like this exercise because it calls upon students to produce language. Many commercial world language programs heavily emphasize receptive language because it’s easier to grade. Reading and listening are important, but students who do too much of this and not enough speaking and writing without assistance end up not really being able to use the language they toiled so hard to learn.

The Scaffold Dialogue Package

Shop for Scaffold Dialogue products at my store.

Okay, so I’m not good at naming things… But the series of interconnected conversation lessons that I developed using all three of these experiences I will term the “Scaffold Dialogue Package”.

Step 1: Learn the Vocabulary – Students should use whatever methods they usually do to learn the words in the thematic list. I use the online flashcards and word bank quizzes here at

Step 2: Complete the two listen-record tasks: For a class of twelve, this taskes about one 40-minute period. I have taught this using paper and digitally using Google docs. We all like paper better for this. I harvested grades for this by calculating percent correct on an open-notes quiz on the table.

Step 3: Complete the Scaffold Dialogue- The scaffold dialogue is based on the two listen-record tasks. It’s important to echo what each student says both for improving pronunciation and for letting people hear enough to record the elements in the table. To avoide confusing when I echo and when I am giving the teacher response, I say “s/he said…” before echoing what the student said. I have harvested scores for this in two ways. If I have time, I give an open-notes quiz. Otherwise, I collect the sheets and select one column to score. This takes a whole forty-minute class period for a class of about twelve. I leave a lot of scoring leeway for these if I grade a column. There can be no English and it has to have some significant elements of what the student said, but for younger learners that suffices. For French 3, I require a little more spelling accuracy.

Step 4: The Test – Administer the test – it takes about twenty minutes or so. Repeat each item twice. When scoring, remember that only errors that affect auditory comprehension matter. Allow 2 free errors for French 1 and early French 2 and 1 free error for French 2 later in the course and French 3. I usually gathered a total score by calculating percent of checks out of fifteen.

For a class of twelve, the whole package takes about four class periods. Once students get in the habit of these, it goes rapidly. Each listen-record has a slightly different improvised structure, so there’s enough varation month to month for interest. Listen-record 1 usually is more basic and causes review of basic structures (like the Education one has students recall names of classes and school supplies). Listen-record 2 is more sophisticated, calling on students to offer opinions or to explain.

The Virtue of Assigning Summaries

Dear fellow social studies teachers,

I would like to play the role of a crusader in this post, if you don’t mind. There’s something I would like you to consider about your teaching practice, or maybe reconsider as the case may be. I’d like you to abandon assigning your students comprehension questions on most kinds of texts you assign in your social studies class. Instead, I’d like you to consider assigning students to summarize instead.

When students do the search and copy instead of reading the text fully, they deny the author the chance to build with the reader an information schema of their own that lends itself to long-term recall.

Okay, so who do I think I am anyway to question a practice that’s at least a century old? Well, I cannot claim to be any super authority. I taught social studies for eighteen of my thirty-two years teaching. I worked in small, rural mountain schools and my roster averaged about one hundred students a year, which is likely smaller than that of my readers. I think these are good starters to convince you to at least hear me, but I have always felt that the argument from authority to be one of the most pernicious fallacies. So in honor of my thirty-two years, grant me the indulgence of reading further. As for accepting my suggestion, let the arguments sway you themselves.

When you assign your students to read a text selection and answer questions on it, you must know what the majority of them do. They skim the text to locate the answer, often they copy it wholesale, and then they’re done. Teachers who accept this kind of work are missing an important opportunity to foster greater reading comprehension skills and greater long-term recall. Writers, even writers of textbooks, build meaning in a schema of content to deliver to the reader, who constructs meaning related to their experience. Ideas are hierarchically arranged and organized to support a set of main ideas. When students do the search and copy instead of reading the text fully, they deny the author the chance to build with the reader an information schema of their own that lends itself to long-term recall. At the risk of putting too fine a point on it, questions on text are largely a useless exercise even when the questions are of high quality.

Once students leave sixth grade in my region, the formal lessons in reading stop. But a great number of sixth graders do not read at a sixth-grade level. They cannot advance their reading without texts at their level. And they have to actually read these. It is possible and desirable in middle and junior high school to promote the development of reading in youngsters by structuring what they do to process text. Teaching students to write competent summaries is one of the best ways to let them develop their reading skills. (This should be paired with offering texts at their independent reading level whenever possible).

What does the research say?

Summarizing and note-taking are two of the most useful academic skills students can have. (Marzano, Pickering, & Pollock, 2005)

Summarizing is in the top two most powerful writing tasks that support the development of student reading. (Graham & Hebert, 2010)

Click here to shop my TpT store for passcodes to give your students access to training videos with embedded auto-corrected questions on composing summaries.

Writing to Read

I was greatly influenced by a paper entitled Writing to Read: Evidence for how Writing can Improve Reading in 2010 published by the Carnegie Corporation of New York. A meta-analysis clearly demonstrates that composing summaries is superior to answering questions on text. The reader may argue that this research was conducted to address methods for using writing to enhance reading. But I would argue that (a) this is rightly a major goal of assigning most secondary students a textbook reading task and (b) the same cognitive processes that go into reading comprehension go into long-term recall of information. Students answering questions on text are less likely to be encoding from working memory into long-term memory than those composing summaries in their own words.

The Five and Three Summary Style

Around 2006 when I was making major investments in curriculum and methodology development, I sat down with a noted professor of literacy who happened to be working in our district, Trudy Walp. I explained my qualms about assigning students questions on text and I asked her, what is the very best thing I can have students do with text from a reading specialist’s perspective. She did not hesitate to say: have them retell what they read.

When I assigned my students summaries on their textbook articles early on, my top students wrote terribly long and detailed compositions that were not brief enough to be a summary. I needed to shorten my students’ summaries and to develop a rubric for evaluating the quality of a summary. The resulting task I used for some fifteen years: the Five and Three.

The assignment for the five and three is to summarize the assigned text (usually four pages in a typical high school textbook) in five sentences exactly, no more and no less. This is the “five”. The “three” is the personal reaction to the text. Students were assigned to connect this text to their own life somehow. What does it remind them of? What do they think about what they read? Why? These had to be exactly three sentences long. This was the second element that the reading specialist recommended to me. Students need to make the text meaningful to them in some personal way that makes new information integrate into their preexisting schema.

The five-and-three became the foundation of my students’ textbook reading work. They also had the option of composing Cornell notes for an assigned reading. I would invite the reader to return to the blog for a post dedicated to extolling the virtues of Cornell note-taking. The five-and-three faded in importance in certain classes where video lessons became an effective information delivery tool when reading instruction was no longer a priority.

The Mechanical Summary

The mechanical summary evolved in 2020 when I had a group of learners who struggled and who just would not produce summaries even given time to do so in class. I offered them the opportunity to write a summary by copying the first sentence of each paragraph word-for-word, then to connect and arrange these into complex sentences such that there were only five sentences. They had to compose the three-sentence personal reaction to the text just like normally done. For this, I offered a maximum score of 76 because, I argued, it had less value not being processed in their own words. I reason that not processing in their own words likely limited the ability of encoding to happen from working memory to long-term memory.

With the Assistance of AI

At this point you are likely wondering how you could possibly grade all these summaries. I will grant you, the workload takes some management, but I would argue that it is only slightly more time consuming than grading answers to a set of questions and the benefit to your students makes it all very worth it.

I’m an amateur programmer and I developed an AI grading assistant that is very good at scoring summaries. This app is available to subscribers to Innovation Assessments. Say, why not sign up for a free 60-day trial? The price is surely right if you decide to subscribe?

The AI grading assistant was accurate enough to save me tons of time scoring summaries. The summarizer app actually let me create models on which to train the AI that were highly accurate when compared to human-generated summaries… eerily so!

This blog post describes the AI grading assistant.

The ability to skillfully summarize is a lost art worth recovering for our students. It enhances the development of their reading ability and it promotes greater recall of the content we teach. It’s my hope my experience developing this system can make this suggestion seem do-able and worthwhile.


Graham, S., & Hebert, M. (2010). Writing to Read: Evidence for How Writing Can Improve Reading. New York: Carnegie Corporation.

Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2005). Classroom Instruction that Works: Research-Based Strategies for Increasing Student Achievement. Upper Saddle River, NJ: Pearson Education, Inc.

Teaching the NYS Global Regents CRQ

When the new Frameworks for social studies in New York State came out about six years ago, the first item that caught my eye was the constructed-response question sets. If you are not familiar with this task, in brief, it calls upon the student to examine historical sources (I use exclusively primary sources) by providing historical or geographic context, identifying the point of view, intended audience, or purpose of a document and then using the two documents in either compare-contrast, cause-effect analysis, or turning point identification.

Global 9 CRQ Products in my store, organized by topic.

Global 10 CRQ products in my store, organized by topic.

Originally, the task was supposed to call upon the student to evaluate the reliability of the source. This was scrapped, probably after field testing questions. I regret their decision to scrap this (though it does appear as part of the analogous task, the short essay, in US History grade 11). Students can be taught to evaluate the reliability of sources at an early age (I incorporated it into my sixth- through eighth-grade social studies work). Students can be trained to evaluate based on features like the point of view (bias), intended audience, purpose, time and place, and authorship. Teaching this does require patience and practice. What I think happened is this: few teachers address the reliability of sources and so when they field tested the new exam format, everyone did so poorly that they scrapped the question. I wish I could find out how my kids did. I had been teaching the reliability of sources since they were in middle school.

I advocate a strategy of assigning one CRQ in each unit of study in grades nine and ten without access to notes. I used this as one of the tests at the end of a unit of study. I would tell students about what kinds of documents would appear and what historical context they should be able to recall in advance. It is a challenging task for them.

Click here to purchase a set of PowerPoint slide shows for teaching reliability factors from my store.

The first challenge for novices is to understand what it means to provide context. Faced with the question “What is the historical context of this document?”, beginners will retell what the document says. The reason for this mistake is that, since they were little kids, teachers have asked them to relate what a text means to prove they understood it. This task asks students to bring to bear what they know about the history behind the document, which requires a level of recall I don’t think we’re asking students to do enough.

Another challenge for students writing the CRQ is the third question where they must use two documents in analysis. Cause-effect relationships are the easiest for students, it seems, so I teach those first. When students are asked to compare and contrast the documents, some students have to be cautioned to select significant elements to compare. The fact that one document is a map and one is a diary entry, for example, is not a significant fact. The turning point question is the hardest, mostly because it calls on students to recall history and to understand historical trends both before and after the event. I find it useful to break this up into small tasks: first say what the turning point is, then say what is so special about it.

After a difficult task, especially for writing, I like to do a “debriefing”. In the debriefing after the CRQ, I like to share student answers anonymously in a slide show and discuss them. It is a very effective strategy for stamping out common errors early and permanently. Sample debriefing PowerPoints from an eighth-grade and ninth-grade CRQ are posted below.

Certain constant reminders were recurring for all my classes:

— Things are not reliable because of what they say or show.

— Things are not reliable because they are in quotation marks.

— Things are not reliable because they happen to match what you think is already true.

— Unreliable statements can be true.

— When you’re asked for historical context, do not describe what the document says.

And what about middle school? Can the constructed-response task be modified for that level?

Students with an IEP often are very challenged by these tasks because of the reading level of the texts. Since I use primary sources exclusively in my CRQ’s, this is doubly difficult for students with reading limitations. The answer to this is to consider obtaining CRQ’s designed with a lower reading level.

If I may be so bold as to give advice, I would suggest that students do these throughout Global nine and ten, perhaps once a month. I suggest that they be as tests instead of open-book or take-home assignments because students need to prepare for recall situations. The debriefing afterward is vital to class improvement and it makes for a long-lasting correction. I would advocate resisting the temptation to leave this off for the Global ten teacher. It really does take a long time to learn to do this well.