Creating quiz questions with GenAI and importing into Blackboard Ultra

Screenshot of some questions in Blackboard.

This post is about a short experiment with preparing a formative set of multiple choice questions using the tools Copilot, ChatGPT and Blackboard Ultra.

The topic selected for this experiment was a set of questions about 3rd Normal Form is database design. This is a well described topic on the internet and the GenAI tools have undoubtedly been trained on this material.

The experiment was whether the tools could generating questions in a format that can be imported into Blackboard Ultra.

Generating questions

Each of Copilot, ChatGPT and Blackboard Ultra could generate reasonable questions for a prompt such as "Create two multiple choice questions on the topic of 3rd normal form for database design."

Copilot and ChatGPT comments

Copilot and ChatGPT seemed to favour the same option as the correct option for each generated question. Copilot preferred the first option, and ChatGPT preferred the third option. That could just be random for such a small number of questions. Blackboard Ultra generated four questions and had a spread of correct options.

Two questions generated by Copilot can be found below. The Copilot prefix was added manually by me.

Screenshot of two questions generated by Copilot. The first asks "Which of the following best describes Third Normal Form (3NF)? and the second asks: "What is a requirement for a table to be in 3NF?"
Example questions generated by Copilot

Two questions generated by ChatGPT can be found below. The ChatGPT prefix was added manually by me.

Screenshot of two questions from ChatGPT. The first asks "Which of the following best describes the requirement for a table to be in Third Normal Form (3NF)?" and the second asks "Given the following relation: Employee(EmpID, EmpName, DeptId, DeptName), and assume that EmpId is the primary key, each employee belongs to one department and DeptId determines DeptName. What prevents this relation from being 3NF?"
Example questions generated by ChatGPT

Note, these examples are after they were imported to Bb Ultra, as described below.

Bb Ultra comments

Unlike ChatGPT and Copilot, the Blackboard Ultra interface already creates questions in a format that can be included in a test. No further processing is required.

You get to choose which, if any, of the questions you want to use.

It is noted that when you go to the screen to generate possible questions, the Bb Ultra interface seems to start generating questions that are based on the module connote. You can type a specific topic into a box or point it at some course content to be more specific. I tried the option to type specific content. I didn't try linking questions to a content item in the Bb module.

Example questions generated by Bb are shown below. The Bb prefix for each question was added manually by me.

Example of two questions generated by Blackboard Ultra.

Blackboard Ultra needed no further processing to have the tests available. The questions are generated and left for review. You can select which of the generated questions to include by selecting each one.

Going from Copilot and ChatGPT to Blackboard Ultra

For Copilot and ChatGPT questions, you need to take extra steps to get the questions into Blackboard Ultra. For this experiment, I focused on using Multiple Choice type questions. The Blackboard format for quiz questions is described in the page Upload Questions on the Blackboard help pages. There, we see the format for Multiple Choice questions, as shown below.

MC TAB question text TAB answer text TAB correct|incorrect TAB answer two text TAB correct|incorrect

Copilot

In my first attempt with Copilot, I gave the following prompt.

Given the following format, can you prepare the question text in the format. Note that the TAB represents a tab character.

MC TAB question text TAB answer text TAB correct|incorrect TAB answer two text TAB correct|incorrect

Copilot then generated a set of output items.

MC  Which of the following best describes Third Normal Form (3NF)?  It eliminates transitive dependencies.  correct It ensures all attributes are atomic.   incorrect
MC  What is a requirement for a table to be in 3NF? It must be in 2NF and have no transitive dependencies.  correct It must have a composite primary key.   incorrect

Interestingly, it generated outputs for two of the question answers. It had dutifully followed the format, and generated sufficient answers for the format. As the format from Blackboard only showed two answers for the question, that is what Copilot did.

With the following prompt, I was able to get all of the answers.

For the format, include all of the generated answers. Where it says 'answer two', you can do that for answers 'three' and 'four'.

The Bb format does require tab characters. Copilot did not generate those. I could ask it to use a different delimiter, e.g. a semicolon. It was able to do that.

I later asked it to produce a file download. It said that it could not do that. Again, it offered me the text with 'actual tab characters'. I could copy that text, but no tab characters were evident in the text. Maybe that is a way that the text is shown in the browser, but it did mean the text could not be used without some further modification.

Screenshot of the response to the question 'Can you convert this to a downloadable file?' The response states that it can't generate a downloadable file, but offers a version that it says has the tab characters that can be copied.
Screenshot of the response to the question about creating a downloadable file, asked in Copilot.

Using the semi-colon, I took the text, put it in Excel, used TextSplit() and had some text that I could import into Bb Ultra. That worked okay.

ChatGPT

For ChatGPT, I used the following prompt after generating the questions.

Given the following format, convert the text to this format. The TAB represents a tab character. The answer text and correct|incorrect pairs can be repeated for as many answers are in the question.

MC TAB question text TAB answer text TAB correct|incorrect TAB answer two text TAB correct|incorrect

It produced the following.

MC  Which of the following best describes Third Normal Form (3NF)?  It eliminates transitive dependencies.  correct It ensures all attributes are atomic.   incorrect   It allows partial dependencies. incorrect   It requires a composite key.    incorrect
MC  What is a requirement for a table to be in 3NF? It must be in 2NF and have no transitive dependencies.  correct It must have no repeating groups.   incorrect   It must allow multivalued attributes.   incorrect   It must be denormalized for performance.    incorrect

This output did include \t characters at the correct points. That could be saved to a file and imported into Bb Ultra.

ChatGPT also offered a file to match this format. That could be downloaded and successfully imported into Blackboard.

Conclusion

This is a limited test. It was only looking at the ease of generating and moving questions into Bb Ultra. It wasn't making any assessment of the quality of the questions.

It is trivial to create questions in Blackboard Ultra. It was a little annoying that its default is to start the process to generating questions on the whole module rather than wait for you to prompt the topic. That seems to be pointlessly using AI resources (and generating carbon).

It was quick to generate questions in Copilot. More work is needed to take the output and get it into Blackboard Ultra. Copilot didn't seem to understand the use of \t characters, or it was lost in the way that it generated the responses for display in my browser.

It was quick to generate questions in ChatGPT. Also, it followed the format when asked and it offered a downloadable file on the correct format.

This has only considered Multiple Choice questions. Blackboard can work with other types. With suitable prompting, it may be easy to create the other types of questions in Copilot and ChatGPT too.