Actions

Proposals/MNet replacement/LTI implementation: Difference between revisions

From Mahara Wiki

< Proposals‎ | MNet replacement
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 86: Line 86:
= Proposal for initial implementation =
= Proposal for initial implementation =


The initial implementation is similar to how the Mahara assignment submission plugin works at the moment with a few modifications to account for assessment requirements.
The initial implementation is similar to how the Mahara assignment submission plugin works at the moment with a few modifications to account for assessment requirements. A few general remarks:


== Overall work flow ==
* Design the functionality to work standard LTI calls so it is more universally applicable and usable.
* LTI does not offer all the functionality that the custom Moodle plugin has. Therefore, rethinking was necessary in order to come up with a solution.
* As much as possible has been outsourced to Mahara as that is typically how LTI functions. Only a grade can be pushed back to the LMS via LTI.


* '''Lecturer''' sets up an assignment and chooses to have Mahara as source for the students to submit from (in contrast to just a file upload, writing text, Google document etc.). All regular assignment features like deadlines, grading methods, gradebook, notifications are available as per standard LMS assignment (for Moodle this would then probably mean to set up the assignment still as assignment rather than "External tool" of possible).
== LTI activity setup (Tutor / Teacher / Assessor) ==
* '''Student''' selects the Mahara portfolio (page or collection) from their personal Mahara account to submit to the assignment (for a preview of the selected portfolio, see "Later phases").
* '''Lecturer and student''' can view the submitted portfolio in an iframe / separate window.
** Iframe option: Only the page or collection should be displayed but not the regular navigation so that it looks like they really only have access to that portfolio. This would require changing Mahara's security header settings to allow the embedding, but can make it easier for the lecturer as they can see their grading options to the side like they do with other embedded content / PDFs and makes it easier for the lecturer to get back to their assignment as they never leave it.
** Separate window option: Could also strip out navigation and just display the portfolio. It's harder for the lecturer to handle the grading, in particular when using a rubric ore pre-made feedback text.
* '''Lecturer''' reviews the portfolio and leaves feedback either directly on the Mahara artefacts and pages or in the Moodle gradebook.  (no change in functionality)
* '''Student''' can see the grade for their portfolio in the LMS gradebook.  (no change in functionality)


== Submission options ==
* The tutor creates a new LTI activity (assignment/external tool depending for which LTI can be configured) in the LMS and selects Mahara as the provider.  After saving the activity in the LMS, (a stripped down version of) Mahara will be displayed in an iframe or new window to the Tutor depending on their configuration of the activity.
* As this is the first time the tutor has viewed the activity, Mahara will display a dialogue, in which the tutor can set the submission options for the activity in Mahara (see submission options section).
* On saving the options, the tutor will see the LTI tutor landing page. At this point, it will simply display that no students have submitted their portfolio.


During the assignment setup, the lecturer can choose how portfolios are submitted:
== Portfolio submission (Student) ==


* '''Always draft:''' Portfolios are never locked, i.e. they continue to stay in draft status and student can always make changes to their portfolio also while the portfolio is being assessed. No archive would be made when grading is done because there is no final submission that would lock the portfolio.
* The student will browse to the LTI activity in the LMS.
* '''Draft + submission:''' Portfolios in draft status can be changed, but once they are being submitted for assessment, they cannot be changed anymore, i.e. they are locked (like when a portfolio is submitted to a Mahara group). Once the grading is finished, the portfolio is unlocked again and the student can make changes. The portfolio would also be archived.
* Mahara will be displayed as per the presentation options setup in the LMS.
* Mahara will display the list of portfolios that the student has created and will allow the student to browse the portfolios.
* The student will then click the ‘Submit for Grading’ button on the portfolio.
* If applicable, the portfolio will be locked at this point.
* Maybe: This will optionally trigger an email to the tutor(s). [at this stage, only the tutor who set up the activity would be known to Mahara and not other potential markers; more investigation needed if others need to be notified as well]
* The student will then be redirected back to their landing page. This page will say that their portfolio has been successfully submitted.


Portfolios are never locked forever – like it is possible at the moment in Moodle – but rather archived.
== Portfolio grading (Tutor) ==


A portfolio can only be added to one assignment at a time. For a more advanced scenario, see "Later phases").
* The tutor browses to the LTI activity in the LMS.
* Mahara will be displayed as per the presentation options setup in the LMS.
* The list of submitted portfolios will be displayed as well as the list of graded portfolios.
* The tutor will click on a submitted portfolio and add feedback or perform SmartEvidence assessment as normal directly on the portfolio.
* The tutor will then click the ‘Assign grade’ button at the top of the portfolio. That button is only visible when the portfolio is viewed via LTI.
* The tutor will be presented with a dialogue with an input for the grade (0 to 100%; that's the only option available via LTI at the moment) and option to archive the portfolio after grading (selected by default).
* On submission of the dialogue the grade will be posted to the LMS via LTI and the portfolio optionally archived.
* If the portfolio was locked during the submission, it will be unlocked after archiving.
* The tutor will be free to re-grade any of the already graded portfolios.
* The archive is placed on the Mahara server and accessible via Administration -> Groups -> Archived portfolios.


There was the discussion about using a snapshot of the portfolio rather than locking a portfolio during assessment, but brings up more questions than it would solve because an infrastructure would need to be found to view the snapshot (currently snapshots are ZIP files) and then potentially make it available to students again to continue working on it. Furthermore, feedback that students should keep in their portfolio on the submitted portfolio would not reach it as it would be stored with the snapshot only or in the LMS gradebook rather than visible on the portfolio itself.
== Portfolio view after grading (student) ==
 
* The student browses to the LTI activity in the LMS.
== Assessment ==
* Mahara will be displayed as per the presentation options setup in the LMS.
 
* The student will see the portfolio they submitted with the grade they were assigned displayed at the top of the page. This will be pulled from the LMS via LTI.
* Lecturer has their regular assessment tools on hand (except plagiarism checker; see "Later phases") because the assessment takes place in the LMS. (no change in functionality)
* They will be free to make changes as the portfolio is now unlocked.
* Lecturer receives temporary “staff” permissions on Mahara while viewing the student portfolio even if their account does not have staff permissions when they just log in to Mahara. This will allow them to assess SmartEvidence collections as it is required to have staff permissions in the same institution as the student to perform an assessment without needing to have their account changed by an admin.
 
== Archiving of portfolios ==
 
Currently, the Mahara assignment submission plugin does not archive portfolios submitted via Moodle, which is a large grievance for institutions because they need to keep all assessed data.
 
* Portfolios that received a grade are archived automatically. There are two options:
** in Leap2A format only (easier as we already do archiving on Mahara in Leap2A)
** in HTML and Leap2A format on Mahara via the Mahara archiving functionality (preferred option). That functionality would be expanded to support the HTML export rather than only the Leap2A export (ideally: Both HTML and Leap2A files access the same files rather than duplicating them in the archive and if the Leap2A is needed, the XML is called and if someone wants to view their site, they call the index.html).  
Having both export formats available satisfies two scenarios:
*** Long-term storage as HTML: No Mahara instance is required to view the portfolio.
*** Re-import into an account via Leap2A: In case the student needs to make vast changes or wants to have an archived of the submitted portfolio or the lecturer wishes to create a sampler and use the portfolio for other purposes, e.g. department or program review.
* The archived portfolios are accessible in the gradebook via a link to the archive file that sits on the Mahara server.
* The archived portfolio also shows up in the Mahara administration in '''Admin menu''' -> '''Users''' -> '''[http://manual.mahara.org/en/17.04/administration/users.html#export-queue Export queue]''' while the archiving is in progress and under '''Admin menu''' -> '''Groups''' -> '''[http://manual.mahara.org/en/17.04/administration/groups.html#archived-submissions Archived submissions]''' once the archive has been created. The file name and display accounts for some information having passed through from the LMS to identify the portfolio by course (and asssignment).
* If a student needs to re-submit their portfolios, the LMS logic around displaying re-submissions is applied. Mahara would still store all submissions, but if the LMS only displays the final file rather than all files that were submitted, then only the link to the final portfolio archive is displayed in the LMS once a grade has been given.


= Later phases =
= Later phases =


* In Mahara, don't just mention that the portfolio was submitted to a particular assignment, but mention which assignment it was and link there rather than just the LMS if possible.
* Release a locked submission if the assignment doesn't exist anymore in the LMS.
* Bring along feedback from the LMS, incl. the grading on a rubric and store it as feedback in the portfolio.  
* Bring along feedback from the LMS, incl. the grading on a rubric and store it as feedback in the portfolio.  
* See a preview of my portfolio that I select so I can ensure that I'm selecting the correct portfolio.  
* See a preview of my portfolio that I select so I can ensure that I'm selecting the correct portfolio.  
Line 142: Line 141:
* Submit portfolios to a plagiarism checker.
* Submit portfolios to a plagiarism checker.
* Lock the portfolio automatically on the assignment due date.
* Lock the portfolio automatically on the assignment due date.
= Alternative approach not taken =
An alternative approach would be to store gradebook information directly in Mahara. This is an interesting idea for the long-term that deserves more discussion in view also of the direction Mahara is going. At the moment though, that undertaking is beyond what would be possible to do for the next release as there is a lot of infrastructure that would be required. Some points to consider for that discussion in the future:
* Gradebooks are highly complicated and should match what a regular LMS gradebook can do to allow for some sort of parity.
* Most institutions that work with gradebooks are schools and tertiaries that already have a gradebook in their LMS and want to keep data in one place. LTI could put the grade from Mahara into the LMS.
* Rather expand the assessment features in Mahara for SmartEvidence than focussing on gradebook.
* Rather allow to create a rubric in the first instance.
* If grade information is needed, find a way to push it back from the LMS via web services and display that information.
* People have been asking more for Turnitin integration than having a gradebook in Mahara as Mahara is not seen as assessment platform but as a place to get evidence ready for assessment that takes place in the LMS.
* Possibility (low-end): Have a “grade” field (turn on / off for a page / group) that teacher needs to fill in based on the grade they need to give as per the LMS. We would treat it as simple text entry field and have a report for users that shows them all their assessed portfolios and the grade they got without any calculation, summation etc. but would still need differetn grading methods, in particular rubric.
* Would need to figure out what to assess: An artefact? A page? One grade for an entire collection? How does it feed into SmartEvidence?

Latest revision as of 17:43, 11 July 2020

Mahara 17.04 supports LTI for authentication purposes. This was the first step into removing MNet functionality. There are a few more steps needed to replace MNet:

  1. Assignment submission
  2. Moving content from LMS to Mahara

It would also be beneficial if Mahara files could be moved to or displayed in the LMS.

We invite comments on the functionality on this page to learn more which features are the most important ones and what the work flows are that should be considered during the development.

Forum post going with this work.

Assignment submission functionality

Goal: Allow portfolios to be submitted for assessment to an LMS via LTI

Background

Mahara can connect to a learning management system (LMS) via LTI for authentication purposes allowing students and lecturers to jump from the LMS straight into their Mahara account. It is not necessary to set up a separate single sign-on between the two systems.

Mahara has been built as personal learning environment primarily and does come with rudimentary assessment capabilities. These are often not enough though for schools and universities as they wish to:

  • Have all assessment information in one place rather than storing grade information in two systems.
  • Use the grading methods lecturers are used to when assessing portfolios.
  • Link portfolios to assignments.
  • Archive portfolio submissions to satisfy assessment record keeping requirements.

Moodle has the third-party plugin “Mahara assignment submission” which allows students to submit their portfolios into Moodle and have them graded there. This plugin is not generic enough to be used with other LMSs and requires Moodle Networking.

In order to make Moodle compatible with more LMSs, the assignment submission functionality should be built using the Learning Tools Interoperability (LTI) standard, which is an industry standard in use by popular LMSs.

Here we collect ideas for functionalities and requirements to consider for and discuss in regard to the submitting of a portfolio to an LMS. The information is purposely situated at the user level rather than providing detailed technical implementation details so we look at the desired functionality first. LMS assignment functionality is only mentioned in so far as required for the connection to Mahara. However, enhancements to how grading is done on the LMS or what grading methods are available within the LMS are not discussed and are out of scope.

Ideas

As an assessor, I want to

  • Allow students to select their Mahara portfolio in my assignment in the LMS.
  • Assess SmartEvidence portfolios and change the assessment status (for which I require the "staff" role in Mahara).
  • Assess portfolios created in a group and give all students who contributed to the portfolio a grade.
  • Have the option to set up a group in Mahara automatically so I can share portfolio templates with all students in the group who have access to the assignment.
  • View the portfolio alongside my standard grading options so I don't need to click a link to view the portfolio and don't have to arrange multiple windows next to each other.
  • Save a snapshot of the submitted portfolio to store for the required number of years for my institution. Note: If space is not an issue, the snapshot should be saved as HTML and Leap2A export for flexibility. If only one export format should be selected, HTML is preferred as that is independent from the current version of Mahara in use and can simply be viewed in a web browser once the HTML export file is decompressed.
  • Use my standard grading options in the LMS, e.g. rubric, feedback comments that are saved in the gradebook, and have the grade put into the LMS gradebook.
  • Be able to leave comments on the student portfolio directly in Mahara, e.g. to comment on specific artefacts on the page.
  • Bring along feedback from the LMS, incl. the grading on a rubric and store it as feedback in the portfolio.
  • Have students submit portfolios that they can continue to work on during the assessment process for formative assessment.
  • Have students submit portfolios that they cannot edit while it's been submitted for assessment. Please refer to the open question “Lock submission or make snapshot?”
  • Access a snapshot of the submitted portfolio after the grading is finished via a link in case I need to review it again. The portfolio should not be locked though at this stage anymore so that students can re-purpose it.

As a student, I want to

  • Select my Mahara portfolio (either a page or collection) in an LMS assignment.
  • See a preview of my portfolio that I select so I can ensure that I'm selecting the correct portfolio.
  • Select a file that is in my Mahara file storage for submission to an LMS assignment. Note: The file should be saved in the LMS assignment so it can be stored along with other assessment information for the required number of years.
  • Select a portfolio created in a group.
  • Submit my portfolio to Turnitin as part of the submission process.

Open questions

Where to save snapshot?

Does it matter where the snapshot is saved? It might be easier or potentially the only way with LTI to save the snapshot in Mahara rather than in the LMS. Then it could be accessed like other archived assignment submissions in Mahara already in the admin interface and directly on the server for archiving of all submissions at the end of a term. See the user manual for what archived submissions look like in the admin interface in Mahara.

Lock submission or make snapshot?

Currently, you have three options in Moodle:

  • A portfolio is not locked;
  • A portfolio is locked for the duration of the grading but then released so students can continue working on it;
  • A portfolio is locked forever.

The locking of the portfolio during the assessment process is beneficial as students can continue working on the same portfolio after it's been released again. However, if they wanted to work on that portfolio already in the meantime, they could not. This is good and bad depending on how you view submissions and how you want to work with the portfolios in your class.

Another advantage of the locking is that assessors can leave comments on the actual portfolio (not just in the gradebook) that continue to be visible in the portfolio after grading and thus become a part of the portfolio. If we only saved a snapshot, none of the feedback would make it into the portfolio, but always only be in the gradebook.

It is possible to look into making a snapshot of the portfolio instead when it needs to be “locked” and not changed while the assessor looks at it. In that case, what should happen when a student needs to make changes to the submitted portfolio based on your feedback, but has already made other changes to the portfolio and it has evolved into a different direction? Should the student be able to download a saved Leap2A snapshot and add it back into their portfolio if they had already made significant changes to the submitted portfolio?

Other ideas

Ideas ideas to explore further at a later stage.

  • Save content from the LMS in Mahara
  • Link to or upload files from Mahara into the LMS
  • Share competency frameworks across Mahara and LMS

Proposal for initial implementation

The initial implementation is similar to how the Mahara assignment submission plugin works at the moment with a few modifications to account for assessment requirements. A few general remarks:

  • Design the functionality to work standard LTI calls so it is more universally applicable and usable.
  • LTI does not offer all the functionality that the custom Moodle plugin has. Therefore, rethinking was necessary in order to come up with a solution.
  • As much as possible has been outsourced to Mahara as that is typically how LTI functions. Only a grade can be pushed back to the LMS via LTI.

LTI activity setup (Tutor / Teacher / Assessor)

  • The tutor creates a new LTI activity (assignment/external tool depending for which LTI can be configured) in the LMS and selects Mahara as the provider. After saving the activity in the LMS, (a stripped down version of) Mahara will be displayed in an iframe or new window to the Tutor depending on their configuration of the activity.
  • As this is the first time the tutor has viewed the activity, Mahara will display a dialogue, in which the tutor can set the submission options for the activity in Mahara (see submission options section).
  • On saving the options, the tutor will see the LTI tutor landing page. At this point, it will simply display that no students have submitted their portfolio.

Portfolio submission (Student)

  • The student will browse to the LTI activity in the LMS.
  • Mahara will be displayed as per the presentation options setup in the LMS.
  • Mahara will display the list of portfolios that the student has created and will allow the student to browse the portfolios.
  • The student will then click the ‘Submit for Grading’ button on the portfolio.
  • If applicable, the portfolio will be locked at this point.
  • Maybe: This will optionally trigger an email to the tutor(s). [at this stage, only the tutor who set up the activity would be known to Mahara and not other potential markers; more investigation needed if others need to be notified as well]
  • The student will then be redirected back to their landing page. This page will say that their portfolio has been successfully submitted.

Portfolio grading (Tutor)

  • The tutor browses to the LTI activity in the LMS.
  • Mahara will be displayed as per the presentation options setup in the LMS.
  • The list of submitted portfolios will be displayed as well as the list of graded portfolios.
  • The tutor will click on a submitted portfolio and add feedback or perform SmartEvidence assessment as normal directly on the portfolio.
  • The tutor will then click the ‘Assign grade’ button at the top of the portfolio. That button is only visible when the portfolio is viewed via LTI.
  • The tutor will be presented with a dialogue with an input for the grade (0 to 100%; that's the only option available via LTI at the moment) and option to archive the portfolio after grading (selected by default).
  • On submission of the dialogue the grade will be posted to the LMS via LTI and the portfolio optionally archived.
  • If the portfolio was locked during the submission, it will be unlocked after archiving.
  • The tutor will be free to re-grade any of the already graded portfolios.
  • The archive is placed on the Mahara server and accessible via Administration -> Groups -> Archived portfolios.

Portfolio view after grading (student)

  • The student browses to the LTI activity in the LMS.
  • Mahara will be displayed as per the presentation options setup in the LMS.
  • The student will see the portfolio they submitted with the grade they were assigned displayed at the top of the page. This will be pulled from the LMS via LTI.
  • They will be free to make changes as the portfolio is now unlocked.

Later phases

  • In Mahara, don't just mention that the portfolio was submitted to a particular assignment, but mention which assignment it was and link there rather than just the LMS if possible.
  • Release a locked submission if the assignment doesn't exist anymore in the LMS.
  • Bring along feedback from the LMS, incl. the grading on a rubric and store it as feedback in the portfolio.
  • See a preview of my portfolio that I select so I can ensure that I'm selecting the correct portfolio.
  • Have the option to set up a group in Mahara automatically so I can share portfolio templates with all students in the group who have access to the assignment. This may need to be done via web services instead.
  • Assess portfolios created in a group and give all students who contributed to the portfolio a grade.
  • Select a file that is in my Mahara file storage for submission to an LMS assignment. Note: The file should be saved in the LMS assignment so it can be stored along with other assessment information for the required number of years.
  • Submit 1 portfolio to multiple LMS assignments.
    • There may need to be rules around the type of assignment, e.g. if the portfolio is to be locked, it can't be added to another assignment, but if it's not locked, it can be added.
    • A scenario where it's necessary to add 1 portfolio to multiple assignments: Students work on a science research and have individual assignments. During all assignments they compile an appendix of resources that is shared across all assignments and part of the overall project. Because the appendix is in the portfolio that satisfies the requirements for multiple assignments, the portfolio needs to be added to multiple assignments.
  • Submit portfolios to a plagiarism checker.
  • Lock the portfolio automatically on the assignment due date.
Retrieved from ‘https://wiki.mahara.org/index.php?title=Proposals/MNet_replacement/LTI_implementation&oldid=11465’