Skip to content

Assessing Students on Open Source Internship Programs §

Pattern Summary §

Design a holistic assessment approach that accounts for the diversity of open source internship projects alongside traditional measures of academic performance.

Problem / Challenge §

  • Traditional academic grading approaches are poorly suited to open source internship programs, where projects vary significantly in scope, complexity, and the time required for onboarding and contribution.

  • Faculty and program staff may find it difficult to assess student performance consistently across teams working on very different projects with different clients, community partners and timelines.

  • Assessment approaches that focus narrowly on technical outputs risk undervaluing the collaborative, communicative and professional skills that are central to open source practice.

  • Students may be discouraged from taking on ambitious or unfamiliar projects if they feel their grade depends solely on delivering a finished product.

Context §

  • A research or teaching university.

  • An OSPO has been established and is running a structured student open source internship or capstone program.

  • Students are working on real open source projects with genuine clients or community partners, often in teams, over a fixed semester or summer internship period.

  • Projects vary widely in scope, technical complexity and the level of prior work students can build on.

Forces §

  • Students contribute in different ways depending on their role, skills and the nature of their project. A 'one-size-fits-all' assessment framework risks penalising students whose contributions are valuable but less visible.

  • Grades need to be defensible within the academic institution's assessment framework while also reflecting the professional norms of the open source ecosystem.

  • Assessment should motivate students to engage deeply with their work and with the open source community rather than optimising for easily measurable outputs.

Solution §

Design an assessment approach that is holistic, evidence-based and flexible enough to account for the diversity of projects and student contributions. Assessment should draw on multiple sources of feedback and evidence and should be clearly communicated to students from the outset so that expectations are transparent.

Key design considerations include:

Holistic rather than points-based assessment §

Rather than summing individual assignment scores into a final grade, design assessment around a holistic evaluation of each student's overall performance and growth across the program. Build in assignments, checkpoints and deadlines as evidence for this holistic assessment rather than as direct inputs to a grade calculation.

Multiple sources of evidence §

Draw on a range of evidence sources to build a rounded picture of each student's contribution and development.

These may include:

  • Client and stakeholder feedback
  • Mentor and tech lead reviews
  • Team 360 reviews
  • Learning journals or reflective assignments
  • Self-evaluation
  • Observational evidence of communication and collaboration patterns

Weighting client and stakeholder feedback appropriately ensures that the real-world impact of student work is reflected in the assessment.

Assessment of process as well as output §

In open source internship programs, how students work is as important as what they deliver.

Assessment should explicitly include dimensions such as:

  • Quality of communication
  • Responsiveness to feedback
  • Engagement with the open source community
  • Consistency of contribution over time
  • The ability to collaborate effectively within a team

Weekly delivery indicators, such as demo videos, merged pull requests and checkpoint documents can provide useful evidence.

Transparency and early communication §

Clearly communicate assessment criteria and expectations to students at the start of the program.

Stretch components §

Consider incorporating a stretch component into the assessment framework to reward demonstrations of professional excellence that go beyond the baseline expectations.

Strategic feedback §

Assessment should not be purely retrospective.

Use the assessment process as an opportunity to provide students with strategic guidance about their professional development and career trajectory. Feedback should help students understand their strengths and areas for growth in the context of the wider open source ecosystem and technology sector.

Resulting Context §

Students receive a fair and meaningful assessment of their performance that reflects the real-world nature of their contribution and the professional skills they have developed, rather than a narrow measure of technical output.

The assessment process provides program staff with a defensible and consistent basis for grading across a diverse cohort of students working on very different projects.

Clients and partners feel that their feedback is valued and has a genuine influence on outcomes, strengthening their engagement with the program.

Students are motivated to engage deeply with their work, their team and the open source community - rather than optimising for easily measurable outputs.

Additional Learning from Carnegie Mellon University Libraries Open Source Program Office §

We assessed students on the Semesters of Code program by drawing on mentor reviews from industry partners including Red Hat Linux, the Eclipse Foundation, and local Pittsburgh startups.

The quality of student contributions was reflected in the strong interest shown by these mentors in offering students not only internships but also jobs upon graduation, providing a powerful real-world validation of student performance that complemented formal assessment processes.

Additional Learning from Georgia Tech Open Source Program Office §

We adopted an evolving assessment approach on the Virtual Summer Internship Program (VSIP) that prioritised increasing awareness and engagement with open source practices among participants.

We recognised that assessment needed to account for the diversity of projects and the different requirements in terms of onboarding, learning, and contributions. This approach acknowledges that completion of a project may not always be possible within the internship timeframe.

We also explored the potential to develop the program into a micro-badging framework, which would allow student achievements to be recognised and shared in an asynchronous and openly accessible way, extending the reach and impact of the program's assessment model beyond the immediate cohort of participants.

Additional Learning from Open Source with SLU §

We assess students on a holistic basis with a focus on final outcomes and overall growth trajectory rather than on individual assignment scores.

The team provides feedback on all members including the client. The client's voice carries additional weight in the overall assessment.

Observational evidence of how students actually do the work; their engagement on Slack; their conduct in meetings; and their responsiveness to peers and clients is also treated as a vital part of the assessment.

Weekly delivery indicators including demo videos, merged pull requests and checkpoint documents provide evidence of sustained engagement and progress.

A 10% stretch component rewards demonstrations of professional excellence. This may include public demonstrations of professional leadership such as conference presentations or contributions to major open source projects.

Additional Learning from the University of Vermont VERSO Open Source Program Office §

We design assessment of students on the ORCA program to account for the diversity of projects and the different requirements in terms of onboarding, learning and contributions.

We explicitly recognise that completion of a project may not always be possible within the program timeframe and assessment is designed accordingly.

We assess students through a combination of customer and stakeholder feedback, mentor reviews, learning journals, self-evaluation and 360 reviews in the case of student teams.

This multi-source approach ensures that the full range of student contributions, technical, collaborative and communicative, is captured in the assessment process.

Known Instances §

References §

Contributors & Acknowledgement §

A note on AI use: In addition to working from Deep Dive transcripts, capturing learning from our community discussions and other patterns from our members, this pattern was drafted with the help of AI. As a small organization, tools like this help us turn rich conversations into written resources without losing the ideas along the way. As always, there were plenty of human eyes reviewing, editing and improving the content before this pattern made it to publication. Thanks go to our community for the insights. If you do spot any errors, please let us know so we can correct them!