These resources are intended to assist you with using PeerAssessment.Com, improving the student teamwork in your classes generally, and/or conducting Science of Teaching and Learning research.
I invite you to send me resources you use in your classes to post here and share with others. Where you borrow items from others, please include the source so that credit is appropriately expressed, and others can follow up for more information.
Using PeerAssessment.Com Resources
PeerAssessment Brochure (PDF, 9/23/2021) The brochure summarizes the key features and benefits of PeerAssessment.Com. For more information, please see the whitepapers on particular areas of the application and articles about its use on this page.
ONLINE ASSISTANCE: Please log in to PeerAssessment.Com and select HELP in the upper right corner.
DIRECT ASSISTANCE: You are welcome to contact me at any time for assistance with PeerAssessment.Com or its integration into your course pedagogy. Email: [email protected] Phone: (208) 869-2801
Teaching With PeerAssessment.Com (PDF 1/23/2022) This article suggests some ways to think through how to incorporate peer assessments in your teaching using the PeerAssessment.Com tool. Consider these ideas for effective practice:
- TIMING: How many assessments should you use?
- PREPARATION: How should you prepare students for giving effective peer feedback?
- REPORT DISTRIBUTION: When should you release results to students?
- GRADING: When should you grade assessments?
PeerAssessment.Com Reports (PDF 1/3/2022) This illustrates sample pages from the five main reports: Student, Instructor, Grade Report, Feedback Assessment Report, and Other Report. In addition the Assessment Data Download spreadsheet is described. This is made available to instructors to apply further analysis or visualization to their class assessment data.
Calculating Individualized Grades From Peer Assessment Scores (PDF 1/13/2022) This article describes three approaches to calculate individualized grades based on the peer-assessed scores. Pros and cons are considered, and there is a fully worked example for each approach.
Instructor and Student Views of the Questionnaire (PDF 9/22/2018) The purpose of this document is to help you, the instructor, map what you see during set-up to what your student’s will see filling out the assessment.
PeerAssessment.Com-Price-List (PDF 8/15/2021) You may purchase classes one at a time, or receive quantity discounts for buying 4 or more classes at a time. Site Licenses including SSO are also available.
Class Purchasing Instructions for PeerAssessment.Com (PDF 9/1/2021) Simply create an account and log in to the application, then press “Purchase Plan”. These instructions will take you through the new options for types of plans to purchase. The instructor may make their own purchase, or another person may do so on their behalf. In either case the purchaser will need to create an account in the system. After completing the purchase, please forward the program information to instructors who will need it to set up classes.
Giving Peer Feedback Resources
from Rob Anson
Feedback is the core of a peer assessment–in the workplace as well as for educational teams. Constructive feedback, delivered with respect, can generate engagement and turn around an individual’s work behaviors. Poorly delivered feedback can just as effectively do the opposite. It boils down to this: the more we can build our student’s ability to give constructive, respectful feedback to one another, the more they can add to our education community and to their own careers.
To that end, this section includes various resources to learn about high quality feedback. Also, check out the Feedback Assessment Report, in PeerAssessment.Com, to assist you in evaluating the peer feedback your students give to one another.
- Tips for Giving Feedback to Your Peers–Brief whitepaper on giving quality feedback written–for your students.
- How to Give Feedback the Right Way–5 minute read from Impraise about how to write constructive feedback–for your students.
- The Power of Feedback–A major research article summarizing research on feedback in education used to aid student learning.
- Feedback Assessment Report–New report added to PeerAssessment.Com for instructors to evaluate feedback given by each student.
- How Atlassian Built an Award Winning Culture Using Feedback–Quick peek at a corporate culture based on feedback.
Anson, R (2020, PDF or DOCX). Tips for Giving Feedback to Your Peers. Whitepaper.
Description: This is a brief whitepaper written for students involved in educational team work who are asked to give feedback to their peers. It draws together best practices from various authors to help students recognize and author effective feedback for others.
You are welcome to distribute this whitepaper to your students. Or, feel free to download the .DOCX version and adapt it to your needs under your own name.
Access Link: How to Give Feedback the Right Way
Link to Summary Poster: How to construct meaningful feedback: C.O.I.N.
Impraise. (NA) How to give feedback the right way. Accessed on 1/24/20 from https://www.impraise.com/blog/how-to-give-feedback-the-right-way.
Description: This 5 minute read from Impraise is an excellent source for your students to learn a model for giving feedback! “Many people find giving constructive feedback daunting. If you remember to use the C.O.I.N formula, you can ensure you’re giving actionable, constructive feedback that will help your peers, manager or direct report improve their performance. You can use this Context – Observation – Impact – Next (C.O.I.N) formula to make both positive and constructive feedback more impactful, but since people find it more challenging to give constructive feedback, this is what we’re focusing on.”
PeerAssessment.Com Feedback Assessment Report (Includes actual student comments entered into PeerAssessment.Com, with names changed.)
The Feedback Assessment Report in PeerAssessment.Com captures responses to the open-ended questions–one Peer Assessment and three Team Assessment. All the responses that one student authored to their team and team members are displayed on one page. Within the PDF file, reports are sorted by author name.
This PDF report makes it easy to read and evaluate the feedback authored by each student. It can be used to assess and guide students to provide higher quality feedback to their team members.
A couple ideas for providing grades and/or feedback to students:
- Print off the report. You can handwrite suggestions and a grade on the paper to hand back.
- Open the report and your electronic gradebook/rubric grader in windows next to one another.
Download: The Power of Feedback
Abstract from The Power of Feedback:
“Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative. Its power is frequently mentioned in articles about learning and teaching, but surprisingly few recent studies have systematically investigated its meaning. This article provides a conceptual analysis of feedback and reviews the evidence related to its impact on learning and achievement. This evidence shows that although feedback is among t he major influences, the type of feedback and the way it is given can be differentially effective. A model of feedback is then proposed that identifies the particular properties and circumstances that make it effective, and some typically thorny issues are discussed, including the timing of feedback and the effects of positive and negative feedback. Finally, this analysis is used to suggest ways in which feedback can be used to enhance its effectiveness in classrooms.”
Citation: Impraise. (NA) How Atlassian built an award winning culture using feedback. Accessed on 1/24/2020 from https://www.impraise.com/blog/how-atlassian-built-an-award-winning-culture-using-feedback
Description: Atlassian has built a thriving, award-winning, corporate culture using feedback as a cornerstone that we can learn from in our classroom. This 10 minute read talks about the foundations for a corporate culture of feedback. Atlassian believes that “peer reviews are an integral part of getting the whole picture when it comes to employee performance. […] When you involve your colleagues in your performance evaluation you cut through the competitiveness and elevate collaboration. This approach focuses on development, and strengthens the team.”
Virtual Student Teams Resources
from Rob Anson
Teamwork is tough enough in the best of circumstances. It is even harder when adding challenges students face in teams–simultaneously learning content, figuring out your instructor and team members, plus (often) no team member has authority and experience to sort it all out. Then imagine trying to pull your team together remotely. You can’t look them in the eye, shake their hand, or buy them a beer to work it out. And don’t get me started about factoring in Covid-19!
Fortunately, a few weeks back an email from Trello dropped in my inbox. Trello markets a fantastic, flexible tool for teams to manage their project work–teams in my senior projects class have used it for years. Both Trello, and their parent Atlassian, have extensive experience with remote teams–creating tools to support them, and using them internally. They leveraged this experience, together with other companies known for innovative remote teams, to create a free online guide called “How to Embrace Remote Work: The Complete Guide to Setting Up Your Team for Remote Work Success“. The actionable advice, while directed at teams in the business world, applies as well to student teams in an educational setting.
My goal here is to make this information even more relevant, and digestible, for you and your student teams to access. I cooked Trello’s Remote Team Work Guide down even further, pulling excerpts and adding some further material. Then I collated and edited the excerpts to focus it, in an abbreviated fashion, on topics student teams need to master in order to work remotely.
Download Remote Student Teams section in single document — Feel free to use this distilled guide in whole or in part. But please make sure to give credit to Trello for making the original guide available. *
The topics include:
- Critical Collaboration Tools for Remote Student Teams
- Choosing the Right Communication Tool
- Video Meeting Etiquette
- Holding Productive Video Meetings
- Creating a Strong Remote Team Culture
- Fast Team Building Activities for Remote Video Meetings
- Additional Resources for Remote Teams
* I am not affiliated with Trello or Atlassian in any way. I am only a grateful customer.
Critical Collaboration Tools for Remote Student Teams
(Contains excerpts from Garber, 2016 and Moon, 2020. Comments from Rob Anson are in italics.)
Your team needs to pick a few basic tools for team collaboration, and set up some explicit guidelines for using them. Everyone on the team needs to use the same tool set, or there is a real chance they could self destruct. (Note: I’ve resorted the list below in order of priority for student teams.)
- Document Storage: Whether it’s Google Drive , Dropbox , or a team space on your LMS, teams need to pick one document storage tool and stick with it. Using different group storage, or individual storage, is a recipe for losing or overwriting documents if someone is out sick or leaves the team. Also, as important as the storage space, is a simple to follow plan to organize it with standards for naming files, using folders, and versioning.
- Writing and Reviewing Documents: The most important tools for student teams are those to collaboratively create and evolve documents and presentations. Google Apps has pretty much captured that market, with Google Docs, Sheets, and Slides. With very similar interfaces, teams can use any tool create, develop, and review reports both in real-time or asynchronously. See How to Collaborate in Google Docs for a 3 minute guide to the collaboration features. Students especially need to know how to use Comments, Version History and Share to recognize Google Docs potential for teams.
- Video Meeting Space: Video conversations are second only to in-person gatherings as a means to understand someone’s tone, meaning, and reactions. They are also invaluable to building more cohesive team relationships. Remote teams can be as effective as in-person teams by adopting a tool like Zoom, Skype, or join.me that supports video cameras and screen sharing. Video meetings take some work to become productive. Some practice applying video meeting etiquette and productive video meeting techniques is a priority. Plus help students use screen sharing strategically to focus the team and get work done.
- Shared Plan: Teams need to know what each member is up to and where projects are in different phases. Trello is a visual project management tool that is hands on for the team. It is used to both manage tasks progress and organize work products for the task itself. In my experience, students love Trello once they have a clear plan for how to use it. I start them with a template and some norms for using it, then let teams evolve from there. Like any project management technique, Trello requires some discipline.
- Collaborative Discussions: Across a project, teams need to be able to engage in a variety of different conversations to generate, refine, link, research, and debate ideas. Today’s leader in organized team chat rooms is Slack. I would suggest Slack for longer, larger and more intensive student projects, or where it is leveraged across a program of classes. In my opinion, it is much more than the standard 3 credit classes I normally teach requires.
- Common Calendar: Using the same calendar tool makes scheduling meetings immensely easier within a team. Whether it’s Google Calendar or Fantastical, giving team members equal access to a shared calendar will put a stop to endless email threads discussing schedules and time zones. I do not push Google Calendar with my students, as many have other work preferences, our LMS manages assignment duedates, and Trello is usually sufficient for team projects. However, students have a devil of a time figuring out when they can meet up. A super simple app called Doodle is great for finding when we can all get together.
Sources/For more information
Team Collaboration Using Google Docs (or Sheets or Slides)
(Written by Rob Anson, with input from Sources)
Google Docs has some great collaboration features for group writing. But what we need is a plan for our team to use those features harmoniously. This brief reviews the features, and lays out a plan for your group to use them productively.
I’ll assume you know how to use Google Docs. But if you’d like to brush up, under Sources below, there are links to good 10-15 minute videos on Google Docs, Sheets and Slides. All three tools include exactly the same collaboration features; I’ll refer to Docs in this article, but the collaboration features and plan apply to all three.
The collaboration features we’ll discuss include: Shared Drive, Comments, Recommend Edits, and Version History. Summary of Collaboration Features in Google Docs And there are links to articles on each under Sources.
To make this easier to relate to, lets say your team was assigned a two week research project report working in person. Actually though you could easily tweak this plan for any group project even if it was 16 weeks in length, a presentation or a design document, or done remotely in a virtual team using Zoom (or some other video conference tool) with a pandemic raging around.
The group writing plan includes 6 steps. Longer projects may iterate steps 3-5 a few times, and projects with more non-writing work may need a more detailed task plan, but here is the basic idea.
1. Set Up: Share Documents & Files [individual team member]
2. Plan Content: Outline Report & Assign Tasks [team meeting]
3. Write Draft [individual members]
4. Review Draft [individual members]
5. Proofread and Check Document [team meeting]
6. Finalize Document [1-2 team members]
1. Set Up: Share Documents & Files [individual team member]
You could share individual documents, but that is a lot of extra work and less effective. Instead, create a Shared Drive for all your team’s files, and give your team members access. In the Shared Drive, create all your Google files, drag & drop other files (eg. PDFs, Word, JPG, etc.), and add folders as needed. The point is that all files are automatically shared with the group.
2. Plan Content: Outline Report & Assign Tasks [team meeting]
A little time planning will save a ton of time working. So meet together, and here is your agenda. I strongly recommend your team asks one member to facilitate the agenda and discussion, enabling everyone to have input. And one person will be the recorder, to capture the plan. Here is what you will do:
- Start with an Outline — Everyone needs to read the assignment carefully first. Then start to identify section headings to outline the report. Be descriptive to people remember, and use all CAPS so the headings stand out, you can reword and clean it up later. The recorder captures it where everyone can see and participate.
- Brainstorm Ideas — With an outline in place, give it a test run. In other words, brainstorm some starter “seed” ideas under important sections. Things like “Talk about this…”, “Find data on that…”, “Address factors in the chapter 4 checklist…”, etc. If the team knows the direction already, then ask everyone to open the document in the Shared Drive, and–at the same time–insert ideas in appropriate headings for 15-20 minutes. In other situations, where you still need to work out a direction, research question, or topic, make it a team discussion supported by the facilitator and recorder.
- Assign Tasks — Next is the task plan–assign who does what. You can put the tasks & names under the section headings, or insert a table at the top of the document to enter tasks * names. There are probably both writing and operational work tasks to capture. Focus on tasks to have done before the team meets next.
- Set the Due Date — Before you split up to work, set the next meeting date f possible. This is also a “hard” (required) date for the assigned tasks to be done.
3. Write Draft [individual members]
Most of the draft writing (and other work) will be done working separately. Just a few guidelines will keep this smooth and productive:
- Section Authors — A simple rule for this phase helps keep the peace: Only section authors enter content into the section. However, when you have suggestions for other sections, then Great!–attach a Comment to that section with your ideas to start the conversation. Then let that author respond or incorporate it. It will be appreciated.
- Draft vs. Notes— Remember, this draft will evolve into the final submission. Thus, only enter information into this document that is intended (at least for now) to go into the final report. Put all those other notes in a separate document–working ideas, reminders, connections, quotes from articles you might use, calculations, etc.
- The Due Date — Remember the due date for completing your sections! It is a good idea for someone to take responsibility to send a reminder email out.
- Communicate — Don’t go silent on your team. Keep up some communications so everyone stays engaged. Also, when you add a Comment to someone’s section, that will also generate an email to members.
4. Review Draft [individual members]
Reviewing is the most critical step, especially writing in groups. Writing separately can get a lot done quickly, but it also introduces a lot of rework issues for content and form. The purpose of reviewing is to create a final report with quality content (accurately communicate your ideas/research, logically develop arguments, address the assignment requirements) and quality form (correct grammar, spelling and punctuation, good flow and word choice).
Review needs to go on throughout the writing process, but the type of review changes. At this stage, early-mid process, you should focus 90% on reviewing content; don’t waste time perfecting the form until ideas are well setup. That said, writing styles and skills often vary widely such that some (10% ?) attention to form will inevitable.
This review requires giving team members time to concentrate. Each will individually read and mark-up the document, placing attention on content suggestions. Two Google Docs features are used for mark-up, Comments and Recommended Edits.
- Comments –> Content — Use Comments to address content. Highlight all or part of a passage, then insert the Comment. By typing your thoughts into the comment bubble, the draft remains readable so others can consider and assess the suggestions. And because you can Reply to Comments, conversations (via email) can efficiently take place to resolve the passage.
- Recommend Edits –> Form — Suggesting works like Track Changes in MS Word. To start, click the pencil in the top right, and choose Suggesting. In this mode, you can edit the text as you normally would, but it captures your edits as suggestions. That is, it marks the text as changed AND your edit shows up as a Comment. Another person can simply accept or drop each change.
5. Decide Revisions [team meeting]
When there are a a number of content suggestions–Comments–it is time for a team meeting. Discussion is needed to decide on the content suggestions, and how to address them in the text. To aid the discussion, everyone can open the document, but one person should share their screen and record revisions from the conversation. It is a good idea to ask one member to facilitate the discussion, helping all members express their ideas while encouraging the team to make decisions they can live with, then move on.
6. Proofread and Check Document [2 team members]
Proofreading and checking the final copy should be done by two strong writers. Finalizing the document includes a few things done by one or two strong writers.
- Carefully proofread and edit the writing–spelling, grammar, punctuation, transitions.
- Set up/check report elements, including titles, team information, page headings/footings, page breaks, etc.
- Ensure the references include all sources found in the report, and that source citations are entered correctly.
- If there are appendices, ensure these are organized and presented appropriately.
When the document form is clean, you are ready to submit. Pay attention to the instructor’s submission instructions. You can use FILE > Download to create a final file in the correct format for the instructor (eg. PDF, Word, Powerpoint, etc.)
* Oops… Fixing Issues: Version History
What should you do if a team member accidentally deletes or changes important passages of the document? Don’t panic! Use Version History.
Open the FILE menu and select Version History > See Version History. This will list all changes and who made them, by time of the change. Scroll down to select the version just before the deletion/change, then do one of three things.
- Click Restore This Version to replace the entire document with that earlier version.
- Click Make a Copy to make a separate copy of the entire earlier version. You can then go back through it to see what you need.
- Click on the earlier version then highlight the section you want to retrieve. Use EDIT > Copy to place it in your clipboard. Return to click on the latest version, and Paste that in at the appropriate place.
Sources/For More Information:
- Group Writing, by The Writing Center at University of North Carolina, 2020
- Collaborative Features Instructions
- Get Started with Shared Drives, by Google, 2020
- How to Collaborate in Google Docs, by Justin Pot, March 24, 2019
- What Is Version History and How to Use It in Google Docs?, By Justin Pot, March 21, 2019
- Beginner Tool Tutorials (if you need to review the basics)
Choosing the Right Team Communication Tool
(Contains excerpts from Gamblin, 2017. Comments from Rob Anson are in italics.)
There are so many ways to communicate at work that it’s overwhelming. Every day, I find myself asking the same question: should I email, instant message, phone, video, or text this information? It’s a mundane question, but it’s important. Because if you don’t choose the right mode of communication, you’ll be ignored, misunderstood, put on the back burner, or dismissed with a vague, insufficient response.
The tools below are listed from most to least communication bandwidth. Video, second only to face to face, has the largest bandwidth to simultaneously convey our words, tone of voice, gestures, expressions and full array of non-verbal information. That bandwidth allows us to express our meaning in multiple ways to ensure we are correctly understood. Consequently, when using tools with less bandwidth–phone, text, and email–there is a greater chance we will be misunderstood.
- Pro: Video is the next best thing to in-person communication. The real-time communication of video makes sharing information fast and reliable, which makes it easier to convey complex topics and resolve sensitive issues more quickly and with less misunderstanding. Also, video is just naturally more engaging, personal and sociable. So it is indispensable for building trust and forming relationships.
- Con: Effective video communication requires appropriate video technology. Bad video communication can make for extremely infuriating and ineffective communication.
- Pro: When video or in-person communication is not feasible, use the phone to communicate information that’s sensitive, complicated, or immediate. A phone call is more intrusive and formal than a text/chat, but is less so than video. Talking things through is how we fix miscommunication, so it’s one of the clearest communication methods we have.
- Con: It has generational issues–young people are less inclined to make a telephone call or answer a call. Also, there is no electronic “paper trail”to check our memories so we could walk away with different memories about what was said or agreed to.
- Pro: Texting allows for more relaxed collaboration and discussion than email. And because it’s a more familiar way to communicate, you can build closer relationships with colleagues as you include memes, gifs, and emoticons. Rapid give and take responses is perhaps the best thing about instant messaging.
- Con: If your topic is too complicated or includes too many people, texting can become messy and burdensome. Also, many of us formed texting habits of quick give and take with our close friends who intuitively understand each other. However, if we apply those texting habits to work team communications we can quickly end up misspeaking or misinterpreting, with consequences.
- Pro: If it’s not timely information, email is super convenient. You can send general information to a whole host of people on your own schedule, and receive their replies on their schedule. It is especially important when a record of the communications is needed. Its longevity and formality means you need to pay more attention to what you say and how you say it.
- Con: Email is less effective in establishing a human connection; it’s formal, faceless, voiceless, and often tone deaf. Also, without this human connection, it also raises the probability of miscommunication.
Sources/For More Information:
Video Meeting Etiquette
(Contains excerpts from Hart, 2020. Comments from Rob Anson are in italics.)
If you aren’t that familiar with the additional expectations placed upon you in a video meeting, that’s okay. We’re here to help you learn the dos and don’ts of video meetings. Take the following information into consideration so you don’t find yourself committing a major faux pas at your next team meeting. Or, share this with a coworker to help them out the next time they’re making one of these mistakes.
Make sure your technology works correctly–before the meeting starts!
- DO: You need to do a few test runs with a friend before your first big meeting. Open up the video tool your team will be using and try it out. Especially important is the mic and camera. Make sure they work. And see how it looks and sounds from another computer, before you know you are ready.
- WHY: You don’t want to delay a meeting with an important client–or even your colleagues or friends–because your video system isn’t working properly. Just make sure everything will run smoothly before you start the real thing!
Be on time
- DO: This is standard for any meeting, but worth a reminder. It is especially important in a video meeting, as it can be a bit harder to catch up.
- WHY: When you walk in late, you’ll be making noise and distract anyone who is speaking in the room. Additionally, when you’re on time for a meeting, it’ll make getting set up with technology easier and less painless so the meeting can start on time.
Wear work-appropriate clothing
- DO: While it might be tempting to work in your favorite sweatshirt all day, wear at least what would be appropriate if the meeting were face-to-face. Doesn’t have to be extra fancy and formal, just make it appropriate.
READY TO ROLL
Frame the camera correctly
- DO: When you’re on video, make sure you frame your camera in a way that feels natural and allows you to look at the camera. Try to sit at eye level to the camera lens and position yourself so that it shows midsection up.
- WHY: Placing the camera too high leaves other participants staring down at you like a bad tv show. If the camera is too low it will create unflattering and awkward angles.
Get the light right
- DO: Make sure that there is enough light on your face and in the room. Your entire face should be lighted and easy to read. Don’t create shadows that make you look like a shady character from a 1930s horror film.
- WHY: Poor lighting conditions have an enormous effect on your video quality. Make sure your non-verbal expressions are easily see.
Look into the camera
- DO: It takes some practice, but you need to look at the camera lens, not at the pictures of people you are speaking to. Look into the camera.
- WHY: Looking into the camera lens is the equivalent of looking into the person’s eyes. If your camera is not on your monitor, this takes some practice to get comfortable. Not looking into the camera will make you come across as more aloof and not paying attention.
- DO: Do NOT check emails or work on your PowerPoint presentation during video meetings! Likewise, prove to your colleagues that you are paying attention by keeping your camera on, and on you. Turning off your camera during a meeting invites others to question your engagement.
- WHY: Not only does research suggest only 3% of people can multitask effectively, but you also look rude to your participants.
Mute yourself when not speaking
- DO: Make it a practice (out of common courtesy to your colleagues) to mute yourself when you’re not talking. For most video conferencing software, it’s as simple as the click of a button.
- WHY: Even when you think you’re being quiet, most microphones will pick up minor background noises, like coughs, sneezes, or typing. These sounds can easily distract other video meeting participants and potentially even cause annoyance.
Sources/For More Information:
- Do’s and Don’ts of Video Calls, (Video, 1:48 min) Great video of Video Call Etiquette created by the students and staff at UC Riverside. Demonstrates best practices of framing, lighting, etiquette, sound, etc.
- Video Conferencing Etiquette: 10 Tips for a Successful Video Conference, by Meredith Hart, March 25, 2020
Holding Productive Video Meetings
(Contains excerpts from Duff, 2020 and Bernazzani, 2019. Comments from Rob Anson are in italics.)
How often are your meetings productive? Be honest. Between meeting attendees arriving late and your coworker loudly typing an email at the other end of the conference table, it can be hard to stay focused. If you’re holding a video meeting, it can be even more challenging to manage your time effectively.
People are more likely to multitask and miss a point if they don’t have someone next to them to hold them accountable. It’s harder to communicate deadlines and deliverables, and technical problems can stop those attending remotely from hearing or seeing everything in the meeting room. According to the online collaboration company Wrike, 46% of those surveyed answered “rarely”, “never”, or “some of the time” when asked if they leave meetings knowing what the next action item is.
Don’t arrange a call if something simpler would suffice
- Do: Use video conferencing tools when a text or email doesn’t suffice. For example, when you find yourself writing multiple paragraphs or a topic would be easier to discuss “face-to-face”, then a video meeting is a good idea. Likewise if you need to make sure they got the message.
- Why: It is all about making the best use of your time. If the issue isn’t pressing or doesn’t require more than a couple of sentences for clarification, you’re better off writing out the message.
Use an agenda
- Do: Create an agenda that outlines the topics to cover, who is responsible for what, and how long the meeting will take. Then, include 5-10 minutes at the end to share final thoughts, discuss next steps, etc.
- Why: An agenda is critical for a productive conversation. Distributing an agenda before the meeting makes everyone aware of subject material and saves time from unnecessary and irrelevant brainstorms. And finally, meeting agendas help everyone keep track of the meeting–what was covered, what still needs to be discussed, and what action items, assignments, and next steps did we decide on for the project to move forward.
Appoint a meeting facilitator
- Do: A meeting facilitator is responsible for running meetings smoothly while making sure every point fits into the proper amount of time. A good meeting facilitator keeps track of the discussion topics and keeps attendees to their time limits by guiding the conversation. The team should designate a meeting facilitator to keep track of the agenda and conversation during the meeting. Either rotate the job or assign someone permanently; but someone needs to be responsible for the meeting. The most important items the facilitator must guide include:
- Review the agenda. Start by reviewing the agenda. Then come back to it to move the group from one item to the next.
- Guide the conversation. Keep track of discussion topics and help attendees stick to their time limits. Your time and your colleagues’ time is extremely valuable; it’s inconsiderate and unreasonable for team meetings to run over the allotted time.
- Referee the conversation. The point of a meeting is to give information to and get feedback from the group. That can’t happen if one person dominates. The facilitator respectfully asks questions like “What do you all think of this?” to create an opening and encourage those who have not had the chance to speak.
- Wrap up. Five to 10 minutes before the official cutoff time, bring the conversation back to a plan of action to complete the next steps. It is good to reiterate details like deadlines, timelines, and task assignments. But most important is to make sure everyone leaves the room united around the mission of why the team gathered in the first place.
- Why: A 2019 study of Remote Work found that interruptions and being talked over are two of the biggest meeting challenges. These, in addition to managing the organization and flow of the meeting, are the facilitator’s primary responsibilities. Doing these effectively can be the biggest single determinant of the group’s ultimate success.
Designate a note-taker
- Do: The note-taker is responsible for noting the topics discussed under each agenda item, and recording agreed upon action items, decisions, task assignments, and deadlines for deliverables. After the meeting, the note-taker should share documentation with the entire group for their review and post it to the group’s online storage.
- Why: If the meeting is important enough to hold, it is important enough to record its decisions and task assignments. It helps those who missed the meeting catch up. And it also prevents rehashing the same decisions made at the last meeting, or people not knowing what tasks they are assigned.
Sources/For more information:
(Summary by Rob Anson)
An asynchronous meeting is one where the participants are not all meeting at the same time. Instead of, say, 4 hours meeting face to face or remotely via video, we can meet over a one week period where people come and go. This overcomes important scheduling obstacles. But in turn it opens up more challenging coordination issues.
As with any meeting, you still need time frames, goals, agendas, and rules. But for an asynchronous meeting, you need to take them a bit further:
- Start and stop times also needs dates and clear time zone accommodations
- The goal(s) may need more explanation to keep everyone on the same page.
- The agenda will slowly unfold over hours or probably days. For example, two days for brainstorming solutions, another two days for rating and discussing them, and then three more days to generate tasks to implement the top idea.
- Rules are much more elaborate, especially if new collaboration tools are used. Participants will need more detailed instructions on what to post, where to post, and how to post it.
The following articles discuss some of the art and science of facilitating asynchronous meetings. Because the tools play a much greater role in this arena, tool capabilities tend to become integrated with the facilitation.
- 10 things about remote facilitation we (AJ&Smart) wish we’d known sooner (Ushakova, 2020) — This is from a collaboration tool maker called Miro. What makes this article so helpful is that it contains tips from a pair of very experienced remote, different-time, meeting facilitators. They confidently point out, “Trust us when we say – remote facilitation has a lot to offer if you know how to do it correctly. We might even dare say… Remote facilitation can make your workshops better.”
- Not In Real Time: How To Run An Asynchronous Meeting (Amin, 2020) — While asynchronous meetings can be very productive, it takes that much more preparation and more culture development to make these work. This article will help you understand the challenges and gives some practical advice for harnessing these.
- Some collaborative tools supporting Ascynchronous (and other types of) meetings:
- Miro — “The online collaborative whiteboard platform to bring teams together, anytime, anywhere.”
- MURAL — MURAL enables innovative teams to think and collaborate visually to solve important problems. People benefit from MURAL’s speed and ease of use in creating diagrams, which are popular in design thinking and agile methodologies, as well as tools to facilitate more impactful meetings and workshops.
- Stormboard — Use Stormboard’s shared workspace to generate more ideas, and then prioritize, organize, and refine those ideas to make your meetings, brainstorms, and projects more productive and effective.
- Cardsmith — Inspired by sticky notes on a whiteboard, Cardsmith is a versatile, online tool for visual collaboration.
- The Definitive Guide to Facilitating Remote Workshops (Tippin, Kalbach & Chin, 2018, sponsored by MURAL) — This is aptly subtitled, “Insights, tools, and case studies from digital-first companies and expert facilitators”. And is the most practical and complete guidebook I have found for facilitating remote interactive workshops for teams engaged in work or learning. Part 1 hits the basic 3 T’s–Teams, Tools, Techniques–with tips and key questions about each. Part 2 focuses in on 6 Essential Factors, such as Time Zones and Transitioning to Online. Part 3 includes 3 case studies. Part 4, Digital First Practice, includes a detailed remote workshop agenda annotated to explain key considerations. Part 5, Remote Friendly Methods, takes 6 generic activities of a workshop (eg, Warmup, Diverge, Converge, etc.). It breaks each method down into how to run it and provides a clear example. It concludes with Part 6, Checklists and Guides, that includes a helpful 1 pager of its Digital-First Principles.
Creating a Strong Remote Team Culture
(Contains ideas and excerpts from Trello, 2020 redirected toward student teams, with additions from Rob Anson.)
Some people are wary of remote teams because they fear a lack of team camaraderie. It’s true that there are no silly interactions in the kitchen or casual hallway “stop and chats” on remote teams. However, with effective planning, these social moments can be baked into a remote dynamic. Talking with people in dozens of remote teams confirms that you really can create a strong remove team culture. There are two things you need to do.
First, establish a clear set of norms or “rules to live by” with 100% buy-in across the group. Here are a few good ones to consider.
- Always assume positive intent. Tone and nuance can get lost over texting or email, so assuming your colleague is coming from a positive place helps with any potential misunderstandings.
- Keep important information accessible for everyone. Make sure to log side texting decisions, record video meetings, and always post notes in public spaces.
- Embrace communication across distributed time zone work schedules. This takes planning ahead: No decisions are made last minute. It may seem like extra work, but it’s actually more organized.
- Expect Structure. Establish a process, structure, and agenda around meetings and updates so everyone can follow along no matter their location. Assign a meeting facilitator and note taker to ensure key decisions are captured in writing
- Different Yet Equal. Accept this fundamental reality: All remote team members are equal, but their experiences differ. It’s OK for co-located teammates to get together in person. But be considerate. If it is a company-sponsored event, provide an alternate perk for remote folks.
Second, keep people communicating through a healthy system of meetings, events, and habits. Here are 3 types of social interactions that can help overcome some of the distance created by virtual environments. These are for peer to peer, team to instructor, and instructor to class.
- Oh Hey There, Mr. Rogers —When groups are assembled, you can’t guarantee everyone knows everyone else. Enter “Mr. Rogers,” a 15-minute weekly pairing of team members who connect on a video chat to, well, just chat. Randomly pair up two team members each week until all members have met. For 15 minutes, they are to hop on a video call and get to know one another. Afterwards, they post a screen capture from the meetup and log some highlights…. Why does Mr. Rogers work? It reveals common interests and sparks conversations. And, it provides a break from work talk to build personal relationships at a reliable cadence.
- Team Check-In –Team check-ins are one of the most effective ways you can motivate virtual student teams with the feedback to stay focused and productive. You can do this with the whole class, but it is much more effective one team at a time to prioritize each team’s specific concerns. The concept is simple: start with a regularly scheduled time (15 minutes or so with a cushion to run long if necessary) for the instructor to meet with each team at key points through the project. Approach it as the team’s time with you, to ask questions, discuss work in progress, or practice a (short) presentation for some feedback. It works because it is expected (regularly scheduled is important!), personal (you see faces over video and interact directly), and the time belongs to the team.
- Class Town Hall — This is a simple idea that can be used with your entire online class. Again, this works best when scheduled in advance. Ask students to electronically send in, before the event, their questions on any topic relevant to the class. (You can decide how open and informal you want to go!) Select 2-3 topics among the questions, and a question to lead each with. Depending on the topic, you may be doing most of the talking or facilitating a conversation among the students. This Town Hall idea blends some structure with some informal, spontaneous interaction to break down barriers.
Sources/For more information:
Fast Team Building Activities for Remote Video Meetings
(Contains excerpts from Humphreys, 2017. Comments from Rob Anson are in italics.)
Even video conferencing, which allows you to pick up on facial expressions and some body language (which constitutes 55% of communication), has its limitations. When the person you’re talking to is a two-inch-tall disembodied head that magically transforms into a buffering wheel every 15 seconds, deciphering facial expressions and body language can be a challenge.
But remote meetings don’t have to be that awkward. Building rapport and understanding helps fill in the communication gaps during those technologically-challenging moments. Here are five quick icebreakers to help your team kick off your next remote meeting on the same page.
- Point Your Camera Out The Window — One of the great aspects of videoconferencing is that it allows you to catch a glimpse of an unfamiliar city in real time, without ever leaving the comfort of your office. If you live in Minnesota and someone is joining your meeting from New Zealand, you’re going to want to peek out their window. The opportunity to play tourist for five minutes will be fun for everyone involved and will help the team get common perspective of their surroundings. If people in the meeting don’t know where everyone joining is located beforehand, you can turn it into a fun guessing game.
- Desk Show And Tell — Explaining the story behind a piece of artwork or an accessory on your desk helps other people get to know you better. Alternatively, you can use the objects on your desk as the basis for a game: Which item within reach would you take with you if you were stranded on a tropical island? I already know my answer: My laptop charger. In lieu of a proper rope, it would be the ideal instrument for rappelling up trees to pick coconuts.
- Two Truths And A Lie — This is a popular segment on Jimmy Fallon, and there’s no reason to believe it won’t be popular during your remote meetings. The rules are simple: Write down three statements, two of which are true and one is false. Read all three statements out loud and give everyone else a minute to guess which of the statements is false.
- Team Coffee Or Team Lunch — It’s scientific fact that eating together promotes bonding. Most companies that pride themselves on having a strong company culture have picked up on this, and just because your team can’t physically meet at the same restaurant doesn’t mean that this tried and true team-building ritual is off limits. It’s fairly straightforward to hold team lunches remotely via videoconferencing: Have food, eat in front of the camera. It may feel weird, but it works. Breaking down virtual barriers by breaking bread is simple to organize and something everyone will have in common.
- Summarize Your CV In 60 Seconds — Everything becomes more exciting when a time limit is involved. Your colleagues are going to want to know a bit about your background, but they don’t want to know about it in Proustian detail. By forcing everyone to summarize their careers in 60 seconds, you’re not only sharing information that will help cement bonds between team members, but you’re doing it in a way that would nab solid ratings if your meeting were broadcast on TV.
- Bonus Activity! Your First Paid Job — This is Rob’s favorite for new teams just starting out. Give each person 1 minute to identify their very first paid job and one thing they learned from it (relevant to life, or to the team’s task at hand?) It works well with any group, especially very diverse ones, taking everyone back to a time we were on common footing.
Sources/For More Information:
- 5 Fast Team Building Activities For Remote Video Meetings, By David Humphreys on April 07, 2017
There are many other resources available related to remote teams. Below are a few practical effective articles and websites that I would like to point out in particular:
- Team Mindset Lab Resources Page (Catalyst Research Team, 2020) — The Team Mindset Lab’s mission is to “conduct high-quality research in the field of team development and performance to bridge the gap between research and practice.” They have developed a set of short, practical papers called “Leading Effective Teams from Home” that provide research based advice on important topics of team leadership.
- Preparing Student Teams for the Workplace (Galbraith and Webb, 2013) — This article does a great job providing groundwork for things to consider with student teams and real teamwork assignments. It concludes with a section on best practices for instructors preparing students to work in teams. “This paper will provide best practices for creating productive teams in the classroom in preparation for the workforce.”
- Enhancing Student Collaboration in Global Virtual Teams–Kohut 2012 Journal of Effective Teaching (Kohut, 2012) — Kohut makes the point that global virtual teams are widely used in industry, but few students have experience working in this environment. The paper provides ideas for creating a Global Virtual Team experience for your students, addressing topics like group composition, student responsibilities, team cohesion and communications, problem solving, team deliverable requirements, and evaluating tasks.
- Facilitating Team Processes in Virtual Team Projects Through Web-Based Technologies and Instructional Scaffolds (Pazos and Magpili, 2015) — This study also addresses remote student teams. It talks about associating activities and tools to provide a “scaffolding”, involving feedback, directions, guided instructional materials, etc. to help student teams become more effective. The authors embedded these scaffolds in an online tool that was tested to see how it positively impacted student self-management and interpersonal skills critical to effective teamwork. These capabilities are available in various online tools.
- What students think about groupwork in business education-benefits challenges student suggested solutions (Lee, Smith and Sergueeva, 2016) — This excellent study draws qualitative input from students regarding their perceived benefits, challenges and suggested solutions from working in teams. It effectively organizes these perceptions using Wheelan’s group development model to show that students focused more on the earlier stages to suggest instructor changes. (Stage 1 is Dependency and Inclusion; Stage 2 is Counterdependency and Fight.) The authors consolidate these suggestions into a set of recommended best practices for instructors to improve student teamwork experiences. Examples include “… focus more on the earlier stages of group development by assigning groups based on students’ strengths and weaknesses, offering a better introduction to groups, and assigning more group-related time or meetings during class.”
- A Guide for Working from Home (Shaw, 2020) — When Covid hit, many students suddenly had to attend classes from home, facing similar issues as did those in the workforce. This is a friendly guidebook with tips for this new lifestyle. “Working from home doesn’t need to be something scary – in fact, there are a lot of benefits to working from home. If you’re working from home, possibly for the first time in your career, we’ve put together this handy guide to help you to get to grips with working from home and optimizing your productivity.” The article highlights are discussions of how to set up your home office, and establishing productive work habits.
Peer Assessment Research Resources
Citation: Anson, R. & Goodman, J. (2014) A Peer Assessment System to Improve Student Team Experiences, Journal of Education for Business, 89:1, 27-34, DOI: 10.1080/08832323.2012.754735
Groups are frequently used in courses, but there is substantial evidence that insufficient attention is paid to creating conditions for successful teamwork. One key condition is high-quality, individual, and team-level feedback. An online peer assessment system and team improvement process was developed for this test case based on three design criteria: efficient administration of the assessment, promotion of quality feedback, and fostering effective team processes. Sample data from 13 teams were collected to propose a means of testing PeerAssessment.Com against these criteria.
Citation: Anson, R., Poole, S., & Fairbanks, A. (2020, February) Developing Nursing Student Communication and Teamwork Skills: Automating Peer Assessment. RN Idaho, 42(4), 15. Retrieved from https://s3.amazonaws.com/nursing-network/production/attachments/242818/original/RN_Idaho_FEB_2020.pdf?2020
Description: This article examines the key role played by communication and teamwork in healthcare quality, and how peer assessments–using PeerAssessment.Com–can improve the teamwork skills of nursing students.
Citation: Hughs, R. L. & Jones, S. K., (2011, Spring). Developing and Assessing College Student Teamwork Skills. Chapter 5 in New Directions for Institutional Research, no. 149, Wiley Periodicals, Inc. DOI: 10.1002/ir.380
Description: This chapter describes research on team member contributions to overall team effectiveness, and various applications of this research
to developing and assessing teamwork by students on team and
group projects and assignments.
Notes: I wish to point out the excellent section called “Implications for Educators”, where the authors discuss key changes to how we typically apply teamwork in our classes:
- Committing to the Development of Teamwork
- Making Assignments That Elicit Teamwork
- Focusing on the Process
- Providing Meaningful Feedback
Citation: Brutus, S., & Donia, M. (2010). Improving the effectiveness of students in groups with a centralized peer evaluation system. Academy of Management Learning & Education, 9(4), 652-662.
Abstract: We describe the impact of a centralized electronic peer evaluation system on the group effectiveness of undergraduate business students over a pair of semesters. Using a quasi-experimental design, 389 undergraduate students evaluated, and were evaluated by, their peers using a web-based system that captures peer evaluations in quantitative and qualitative formats and allows for the reception of anonymous feedback. Results show that the effectiveness of students, as perceived by their peers, increased over semesters. This effect could be directly linked to the use of the system. The results of this study underscore the benefit of centralizing peer evaluations for the assessment of important skills and their development in higher education. The implication of these results and possible avenues of research are detailed.
Citation: Britton, E., Simper, N., Leger, A., &Stephenson, J. (2017). Assessing teamwork in undergraduate education: a measurement tool to evaluate individual teamwork skills. Assessment & Evaluation in Higher Education, Vol. 42, No. 3, 378–397
Abstract: Effective teamwork skills are essential for success in an increasingly team-based workplace. However, research suggests that there is often confusion concerning how teamwork is measured and assessed, making it difficult to develop these skills in undergraduate curricula. The goal of the present study was to develop a sustainable tool for assessing individual teamwork skills, with the intention of refining and measuring these skills over time. The TeamUp rubric was selected as the preliminary standardised measure of teamwork and tested in a second year undergraduate course (Phase One). Although the tool displayed acceptable psychometric properties, there was concern that it was too lengthy, compromising student completion. This prompted refinement and modification leading to the development of the Team-Q, which was again tested in the same undergraduate course (Phase Two). The new tool had high internal consistency, as well as conceptual similarity to other measures of teamwork. Estimates of inter-rater reliability were within a satisfactory range, although it was determined that logistical issues limited the feasibility of external evaluations. Preliminary evidence suggests that teamwork skills improve over time when taught and assessed, providing support for the continued application of the Team-Q as a tool
for developing teamwork skills in undergraduate education.
Citation: Galaleldin, M., Boudreau, J., & Anis, H. (2019). Team formation in Engineering Design Courses, in Proceedings 2019 Canadian Engineering Education Association (CEEA-ACEG19) Conference, University of Ottawa; June 9–12, 2019
Abstract: Engineering design courses often include a team-based project. Project-based learning offers a great opportunity for engineering students to learn about teamwork and collaboration. It also gives students a chance to learn about themselves and improve their conflict management skills. Choosing the right team members for a specific project is not trivial, as the choice of the team often affects the project outcome and the students’ experience in the course. Moreover, there is a debate among engineering educators as to whether it is better to force team composition or not. In this paper, we investigate the impact of team composition and formation on project outcomes and student satisfaction in a second-year engineering design course at the University of Ottawa. The course is open to all engineering students and has an accessibility theme. Students work in teams with a client that has a specific accessibility need. Students meet the client three times during the semester and deliver a physical prototype by the end of the semester. For this study, students in the design course were divided into two groups. Students in the first group were allowed to pick their teams, while the instructor created the teams in the second group based on multidisciplinary composition and year of study. Both groups had the same instructor and the same course material, labs, project choices, etc. Semi-structured interviews were conducted with a few teams in each group.
Citation: Galbraith, D. & Webb, F. (2019). Teams That Work: Preparing Student Teams For The Workplace, American Journal Of Business Education, March/April 2013, 6:2
Abstract: Organizations today often require collaboration in the form of work teams. Many tasks completed within organizations, whether in the workplace or in academia, however, can be beyond the capabilities of individuals alone. Productive teamwork and cooperative activities in business are expected and can begin very early in a person’s career. The pedagogy for teamwork instruction in the classroom may not simulate real workplace events or parallel organizational behavior in order to attain a successful outcome. In universities, teamwork often breeds frustration and dysfunction, since the teams often do not perform at a high level or reach their full potential. This paper will provide best practices for creating productive teams in the classroom in preparation for the workforce. This insight will include ideas that will bond team members through collective values and goals, resulting in effective teams and a productive environment.
Citation: Oakley, B., Felder, R., Brent, R. & Elhajj, I. (2004). Turning Student Groups into Effective Teams, Journal of Student Centered Learning, 2:1, 9-34.
Abstract: This paper is a guide to the effective design and management of team assignments in a college classroom where little class time is available for instruction on teaming skills. Topics discussed include forming teams, helping them become effective, and using peer ratings to adjust team grades for individual performance. A Frequently Asked Questions section offers suggestions for dealing with several problems that commonly arise with student teams, and forms and handouts are provided to assist in team formation and management.
Note: This is an outstanding paper for the instructor, with a number of useful appendices:
- Getting to Know You-1 (survey of information for students to complete about themselves)
- Getting to Know You-2 (a time schedule for students to mark their availability)
- Team Policies (useful team policies, roles and responsibilities that is useful for considering items for a Team Charter)
- Team Expectations Agreement (a short assignment for the team to develops a simplified agreement on its rules)
- Evaluation of Progress Toward Effective Team Functioning (a team-level assessment to identify areas of concern. This is reproduced in PeerAssessment.Com as question set T-04)
- Team Member Evaluation Form (a peer-level assessment to provide a formative and a summative assessment of the individual team member. This is reproduced in PeerAssessment.Com as question sets P-01, P-02)
- Autorating System (a step by step method to adjust team grades by a normalized peer assessment scores)
- Coping with Hitchhikers and Couch Potatoes on Teams (a short case of a rather dysfunctional student team, with some very helpful advice on “absorbing” and “mirroring” dysfunctional behavior)
Citation: Yang A, Brown A, Gilmore R, Persky AM. A Practical Review for Implementing Peer Assessments Within Teams. Am J Pharm Educ. 2021 Oct 25:8795. doi: 10.5688/ajpe8795. Epub ahead of print. PMID: 34697020.
Objective. The objective of this review is to maximize the utility of administering peer assessments in teamwork settings in professional pharmacy curricula.
Findings. There is a lack of best practices for using peer assessments in the academic setting. The studies reviewed used peer assessments of team for formative and summative assessment, assessing teamwork at limited number of time points to multiple time points; attaching student names to the assessment or being anonymous, and with explanation of why the tool was being used to no explanation.
Conclusion. To get the best use of peer assessments, instructors must define the purpose for their use, explain the purpose of teamwork, orient students to the tool being used, assess teamwork over time and provide feedback, minimize grades associated with the assessment, and use partial anonymity when collecting feedback.
Citation: Sridharan, B, Tai, J, and Boud, D. (2019) Does the Use of Summative Peer Assessment in Collaborative Group Work Inhibit Good Judgement? The International Journal of Higher Education Research, v77 n5, p853-870, May 2019.
The accuracy and consistency of peer marking, particularly when students have the power to reward (or penalise) during formative and summative assessment regimes, is largely unknown. The objective of this study is to evaluate students’ ability and behaviour in marking their peers’ teamwork performance in a collaborative group assessment context both when the mark is counted and not counted towards their final grade. Formative and summative assessment data were obtained from 98 participants in anonymous self and peer assessment of team members’ contributions to a group assessment in business courses. The findings indicate that students are capable of accurately and consistently judging their peers’ performance to a large extent, especially in the formative evaluation of the process component of group work. However, the findings suggest significant peer grading bias when peer marks contribute to final grades. Overall, findings suggest that students are reluctant to honestly assess their peers when they realise that their actions can penalise non-contributing students. This raises questions about the appropriateness of using peer marks for summative assessment purposes. To overcome the problems identified, this paper proposes a number of measures to guide educators in effectively embedding summative peer assessment in a group assessment context.
Download: Individualising teamwork scores-2020
Citation: Shishavan, H.B., and Jalili, M. (2020) Responding to student feedback: Individualising teamwork scores based on
peer assessment, The International Journal of Educational Research Open, (100019) 2020.
Teamwork assessments often result in a single final product, for which all team members receive a single mark
regardless of their contribution to the team project. In order to respond to feedback from students in terms
of fair marking of the teamwork assessment, we implemented peer assessment as a recommended pedagogical
intervention to individualise team marks and prevent team members from taking advantage of free riding. Team
marks were individualised proportional to the average peer assessment mark each student received from their
peers in the team. We analysed course evaluation data before (n = 151) and after (n = 129) the implementation
of the peer assessment process from students participating in Engineering Design courses. Analysing data in light
of Social Interdependence Theory (Johnson & Johnson, 2009) showed that the peer assessment process improved
the cooperation of the team members which enhanced the students’ teamwork experience and their engagement
with the course. While only 24% of the students appreciated teamwork aspects of the course before the peer
assessment, it increased to 34% post peer assessment. Furthermore, the implementation of the peer assessment
process decreased complains about free riding from 26% to 7%.
Citation: Shishavan, H.B., and Jalili, M. (2020) Let’s be frank: Individual and team-level predictors of improvement in student teamwork effectiveness following peer-evaluation feedback, The International Journal of Management Education, (100538), 2021.
This paper examines the individual and team-level predictors of improvement in student teamwork effectiveness following peer-evaluation (PE) feedback. The goal of this study is two-fold: first, to understand the differences in students’ initial reactions to PE feedback, as well as their subsequent decisions and actions to improve; and second, to identify the team-level processes that contribute to improvements in students’ teamwork effectiveness. The mixed-methods study design combines the benefits of qualitative and quantitative data collection and analysis. The sample consists of 266 undergraduate students in 51 teams, working together for the duration of one academic semester. Data were collected in multiple waves, using open-ended surveys and interviews, as well as a standardized online PE system. Both the qualitative and quantitative analyses revealed that PE feedback is the most salient factor influencing students’ improvement in their teamwork effectiveness. Moreover, students’ grade aspirations and prior experience using the PE system are positively related to the level of improvement in teamwork effectiveness. The team-level factors have more complex effects, with different team processes influencing improvement along different dimensions of teamwork effectiveness. These findings have important pedagogical implications for improving students’ teamwork effectiveness.
The University of Queensland has a formalized tool and procedure for conducting group peer assessment and analyzing the assessment data. Their website is an excellent source of information about one approach to group peer assessment. Below are some of the pages to note.
Overview of Group Peer Assessment — Provides a brief review of the PETS procedure for assigning group work, and use of Likert versus PAF (Peer Assessment Factor) questions.
PAF Formula and Moderation Overview — Explains the calculation and interpretation of the PAF, SAPA (Self Assessment over Peer Assessment), and Ratio measures.
PAF – Moderation Case Studies — A great deal of emphasis is placed on the identification and moderation of scores applied by students that are not in agreement with other scores. Moderation may result in removing one specific score that a student assigned, or all of their scores, if it falls significantly outside of the scores other students have assigned.
PAF – View and Moderate Result — This defines and demonstrates the various calculations from the perspective of the University of Queensland’s group peer assessment tool.
Agile Project Management for Student Teams
from Rob Anson
Having practiced and taught traditional project management for a number of years around IT projects, I was beyond ready for a new methodology that was better fitted to the reality of these projects. Agile Scrum was that methodology. But in addition, it was well-suited to large student projects in the classroom.
I must point out however that successful Agile requires discipline and structure. It is not at all “anything goes”, if you want appropriate, high quality solutions to come of it.
- Retrospectives–A Best Practice for Team Improvement–The Retrospective is a key Agile practice to encourage continuous improvement of team processes. I wrote this up to show how PeerAssessment.Com can help make Retrospectives more effective.
- An Agile Framework for Teaching with Scrum in the IT Project Management Classroom–A clear outline of how to apply the core principles and practices of Agile to a project management course.
- Implementation of Agile Project Management in the Classroom–.
- A Capstone Course on Agile Software Development Using Scrum–Another course application of Agile. This is helpful for its attempts to quantify behaviors and reactions of student teams as they applied Scrum, and for the excellent Lessons Learned section.
- Scrum Reference Card–This is one of the best quick reference sources of Agile/Scrum practices I’ve found. It is always in the materials I give to students.
- Teaching Tip Play Ball: Bringing Scrum into the Classroom–This is a classic scrum game will get your students up and active to learn about a key Agile concept–how do self-organizing teams work.
Retrospectives — A Best Practice for Team Improvement
(by Rob Anson)
Whether or not you encourage your student teams to use Agile Project Management, there is an Agile technique, called the Retrospective, that works very well for student groups. It is a best practice drawn from high performing work teams that students should learn to use for their careers.
The Retrospective is a simple concept actually. The team should set aside some time to meet on a regular basis to examine how their team is working together–individuals, interactions, processes, communications, tools, etc. In Agile PM, this Retrospective meeting should take place at the end of each Sprint (often a 2-3 week period.) Or you may schedule when teams submit a major deliverable. But the point is to build it into the project where ideas for improvement can be immediately applied and re-evaluated in subsequent teamwork.
PeerAssessment.Com can help gather input for the Retrospective. First, its anonymity encourages students to be more forthcoming in their assessment of teamwork. Second, it organizes member inputs for the Retrospective meeting discussion. A byproduct is that the meeting itself is more efficient with the initial inputs in hand; I find 15-20 minutes is usually sufficient for students to complete the agenda below. The assessment should be conducted just prior to the meeting.
In the assessment, you should include some rating and open ended questions in the Team Assessment section. The default question sets are designed for this, including sets: T-01, T-02, T-03a. (You can substitute T-04 to T-01 and T-03b for T-03a.)
Retrospective Meeting Agenda
- Ask students to hold their Retrospective meeting soon after the assessment report is sent.
- The team should discuss and record answers to three questions:
- What 2-3 key things does our team do really well?
- What 2-3 challenges are impeding our team?
- Identify 2-3 action items that our team will commit to doing over the next work period to improve our teamwork.
- Ask the team to record these items, save a copy, and submit one to you.
- You may wish to ask teams, at their next Retrospective meeting, to include an assessment of implementing their action items from the prior Retrospective.
Citation: Rush, D. E. & Connolly, A. J. (2020). An Agile Framework for Teaching with Scrum in the IT Project Management Classroom. Journal of Information Systems Education, 31(3), 196-207
Abstract: This paper presents a framework for teaching a complete, semester-long IT project management course with traditional PMI-based content (sans software development) while featuring Scrum as the organizing logic for accomplishing coursework. This framework adapts widely-used Scrum practices from industry for use in the classroom, including how to organize student teams, homework, and activities. Organizing an existing course with Scrum is intended to maximize student learning of traditional project management content, as well as the difficult-to-teach, socially-complex, “soft” skills that lead to Scrum team success. This deep integration of Scrum into a traditional, predictive IT project management course goes well beyond single activities or units without crowding out valuable time and material. A brief overview of the agile philosophy and examples of teaching Scrum in the classroom situate this work in the teaching and learning literature. Classroom-tested Scrum rituals and example artifacts are provided to illustrate how to apply the framework. This group-based, iterative, and hands-on approach equips students to better internalize and understand the complex social interactions involved with a self-organizing team, concepts that are difficult to learn without first-hand experience. The proposed framework will help IS educators implement Scrum practices in their own courses, further addressing industry’s
increasing demand for IS professionals with Scrum experience.
Citation: Myers, M. J.(2016). Implementation Of Agile Project Management In The Classroom. (Master’s thesis). Retrieved from https://scholarcommons.sc.edu/etd/3953
Abstract: The world of engineering and engineering practices is advancing rapidly. In response to this rapid change, engineering education practices have to advance to ensure students are properly trained for the workforce. The purpose of this report is to address and substantiate the hypothesis that if engineering instruction incorporated Agile project management methods, then students will be challenged by professors to accomplish course objectives with a systematic and timely approach that will improve assessment performance metrics and present the framework of how agile methods of project management can be integrated into the classroom. The agile methods incorporated will also encourage the use of industry-related soft skills; emphasizing accountability, resourcefulness, team building, and interpersonal skills. From this framework, the idea that instructors have the ability to manage their students and accomplish course objectives in a timely manner, similar to the engineering industry practices, even when presented with impromptu absentees or cancellations, is plausible. The proposed method to substantiate this hypothesis was the implementation of a flipped classroom and using scrumban agile methods within a General and Honors classroom setting. Due to design and time limitations, only the Kanban Board was implemented into the Honors section for study. The results of the study showed the Honors section performance metrics decrease. With the limitations of the experiment, the hypothesis was rendered inconclusive. In moving forward, obstacles that were present (hurricane cancellations and guest instructors) are still believed to be mitigated with full experiment implementation
Citation: Viljan Mahni (2012). A Capstone Course on Agile Software Development Using Scrum. IEEE Transactions on Education, Vol. 55, No. 1, February 2012, p 99-106.
Abstract: In this paper, an undergraduate capstone course in software engineering is described that not only exposes students to agile software development, but also makes it possible to observe the behavior of developers using Scrum for the first time. The course requires students to work as Scrum Teams, responsible for the implementation of a set of user stories defined by a project domain expert playing the role of the Product Owner. During the course, data on project management activities are collected in order to analyze the amount of work completed, compliance with the release and iteration plans, productivity, ability in effort estimation, and the like. The paper discusses the achievement of teaching goals and provides empirical evaluation of students’ progress in estimation and planning skills. A summary of lessons learned and recommendations is given, reflecting the issues to be considered when teaching courses in agile software development. Surveys of students have shown that they were overwhelmingly positive about the course, indicating that the course fully met or even exceeded their expectations.
Download: Scrum Reference Card (print version)
Citation: Michael James and Luke Walter (2019). Scrum Reference Card. Seattle Scrum Company.
Abstract: This 6 pager is a surprisingly comprehensive and refreshingly brief description of all the major Scrum and Agile terms and concepts. It is broken down into the following sections (linked to online version):
Citation: May, J., York, J., and Lending, D. (2016) Play Ball: Bringing Scrum into the Classroom. Journal of Information Systems Education, Vol. 27(2) Spring 2016, 87-92.
Abstract: Scrum has become a widely-used framework for technology development in both private industry and the government. As a result, Information Systems recruiters and executives have recently been placing a focus on students with Scrum knowledge. Unfortunately, current System Analysis and Design textbooks provide cursory attention to Scrum. Thus, the purpose of this paper is to suggest a starting point for teaching Scrum at the university level by presenting a classroom exercise (Ball Game) that can be used as a means for learning Scrum in more detail. This tip accomplishes three things: (1) introduces students to Scrum concepts with an engaging and memorable exercise, (2) provides a means for teaching students about estimation, and (3) offers an approach that allows students to witness firsthand how self-organized teams inspect, adapt, and evolve