Actions

Proposals/SmartEvidence/SmartEvidence: Technical considerations for the MVP

From Mahara Wiki

< Proposals‎ | SmartEvidence
Revision as of 20:36, 18 July 2016 by Anitsirk (talk | contribs) (Created page with "* Use the [https://usecanvas.com/ottonomy/learning-pathway-modelsapi/47JdBeSCqzKJ1v3H8Oswlo Open Badges alliance framework] for storing competency framework data because it wo...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
  • Use the Open Badges alliance framework for storing competency framework data because it would be extensible with badges (we wouldn't have to use them right away or all of them, but at least we'd have a framework in place that supports it)
  • We are looking at a first implementation of SmartEvidence and thus know that some things are created manually / hard-coded on the backend.

Setting up a competency framework

  • Initially, competency frameworks with up to 2 levels (standard and sub-standard) are set up directly in the code and made available to the institutions for choosing in collections so that users can be assigned sharing and copying permissions as per default Mahara.
  • Administrator interface functionality would come at a later stage.
  • Institution admins have a “Yes/No” toggle (similarly to skins) where they can decide whether to use SmartEvidence or not. If they don't, they won't get the toggle for associating a competency framework with a collection when viewing the collection settings page.
  • Each standard / sub-standard gets
    • a short name which is used to identify the standard / sub-standard on the overview page; no longer than 10 or 15 characters;
    • a title which along with the short name is used to identify it in the annotation block; up to 255 characters;
    • an optional description which contains the key indicators or other narratives describing the standard / sub-standard; no character limit
  • For each SmartEvidence collection the admin selects whether the portfolio author or an assessor (with the role of staff or admin) can “rate” evidence.
  • It's also determined what the “rating” should be. The initial one would be: Meets, Partially meets, Doesn't meet. These strings are hard-coded at the moment for each framework. Thus, if someone wanted to have the same SmartEvidence collection in a different language, they could copy the code and change the language strins. As there is no global agree-on terminology for the “ratings”, language strings in core Mahara would be too hard, but we could look into offering a couple default ones.
  • NOTE: A SmartEvidence collection would need to be set up in each language though in which it is to be used as we do not yet have the functionality to translate content on the fly / offer two versions.
  • ? Provide a couple of optional scales that could be provided via Mahara to get people started?
  • The table of the sub-standard has a “parent” column so it can easily be associated with a standard and then also exapnded to a third level for example.
  • Only the top standard would get the grey background / be shown across, but not any other subsequent levels.
  • When a framework only has one level / no further categorization, then the grey level is dropped.
  • Make the overview table accessible: http://webaim.org/techniques/tables/data
  • We are not looking into a smartphone interface at the moment, but want to get something to work for 10” plus screens as the matrix is already complicated enough.

Associating a collection with a competency framework

  • SmartEvidence collections can only be set up on the institution or site level.
  • In contrast to the initial idea of using categories to distinguish SmartEvidence collections, we are implementing a “Yes/No” selector on the “Title/description” settings page for the collection which is visible only on institution and site collections.
  • Then the admin selects the competency framework that is available for the institution in which the collection is created from a select2 drop-down menu where the available frameworks are shown but the admin can also start typing to get the one they want.
  • Once the framework has been chosen, the SmartEvidence overview page is added automatically as first page in the collection. At a later stage, users / admin would be able to decide whether to place it at the end, but right now the beginning would do.
  • If it were easier and the SmartEvidence overview page should be treated more like a regular page (esp. seeing that our collections are still accessible via page URLs), then it should not be possible to remove it from the collection (it should only be able to be moved in the order of pages) and a user should not be able to delete that overview page as long as the entire collection still exists. Since the user can't edit the page itself, it should not get an edit button.
  • At a later stage we could look into offering to change the title of that overview page, but for right now I would say an admin should be the one to decide what to call it as part of the hard-coded setup.
  • The administrator can add pages to the SmartEvidence collection to indicate how many different pieces of evidence he expects. The creating and adding of pages functions in the standard Mahara way as is the deletion.
  • Now, administrators can share the collection as usual.
  • Aaron had mentioned to put some “Ajaxy” stuff on the SmartEvidence overview page, but I don't remember whether that was for the page navigation of for the imagery on the page.
  • A user can add more pages to the SmartEvidence collection than are provided by the admin the same way they add pages to a collection right now.
  • When a page is about to be deleted from a SmartEvidence collection, the learner will need to get a special notification to be alerted to the imact, e.g. losing annotations, not seeing the page in the competency framework anymore... The annotation blocks and all their data will be deleted.
  • Student working with SmartEvidence overview page
  • When a student clicks on one of the standards and wants to associate that with a page, a modal is opened in which the student can provide an annotation to explain the reasons for associating it with this standard / substandard. Since the user clicked on a particular standard from the overview page, the block would already have selected the appropriate standard in the drop-down menu.
  • The student cannot change the annotation (except for 10 minutes right after finishing editing and before any feedback has been left).
  • The annotation block is put in the bottom row of the evidence page and the block is retracted automatically.
  • When a student is on a page in the collection that she wants to associate with a standard, she can put the annotation block into the page anywhere and again, “Automatically retract” is selected. The title of the block is automatically set according to the selection of the competency standard and the short name is added, e.g. “Annotation for 3.1” or “Annotation for Innovative Design”.
  • As soon as a student put the annotation block onto a page and added her annotation, the overview page shows the icon for the standard having been associated with that page. Currently, that is the ring icon.
  • Students can associate more than one standard with a page. For each standard, an annotation block is created automatically if the student selected the standard from the overview page or added manually when done from the page itself. That way, when a student decides to remove a page from a standard because it doesn't fit anymore, all other annotations would not be affected.
  • Once an annotation block has been deleted, all its content (annotation and feedback) is gone and cannot be retrieved anymore. Students will receive a warning message when they are about to delete the block to know of the consequences.
  • At a later stage, we could look into changing the annotations from the a block to something else, but for the time being, this solution might be the easiest.
  • When a student puts an annotation block onto the page herself, she will need to select the appropriate standard. Since the page is in a particular SmartEvidence collection, a select2 drop-down menu in the block only displays the standards available in this framework. The student cannot change the selection of the standard once the block has been saved (except for up to 10 minutes and before any feedback has been left).
  • LATER: We also discussed the idea to have a section on the page in which all annotations are displayed with tabs to easily move between them (might be tricky on mobile as there could potentially be yet another hamburger unless we don't use one). Or to have an overlay so that the annotations can be placed on the side. Again, mobile might be tricky here and it requires more investigation.

Assessor working with SmartEvidence overview page

  • When an assessor has been given permission to view the collection (either by regular permissions or by portfolio submission), he can view the overview page as well.
  • An assessor can only leave feedback for a particular competency standard when the student has already placed an annotation for that standard. Otherwise, the assessor can use the regular commenting functionality on pages and artefacts, but cannot yet assess the evidence using the framework.
  • Once a standard has an annotation (the ring icon), he can click that standard, view the annotation by the student and leave feedback. The student then receives a notification about the feedback and the assessor is automatically added to the list of people who get notified when more feedback is left on that annotation (this should be very similar to how notifications work on comments). The assessor can also edit his feedback for up to 10 minutes or until the first additional feedback (this should be similar to comments; I can't remember if you could edit until another person left feedback or if it was for a maximum of 10 minutes).
  • The assessor needs the staff or admin role in order to “rate” evidence.
  • In addition, the assessor can “rate” the evidence and select from a drop-down menu which attribute it should get: Meets, partially meets, doesn't meet. These language strings are hard-coded into the framework in the beginning. A corresponding symbol indicating this status is then displayed on the overview page.
  • The assessor can change the “rating” at any time and doesn't necessarily have to leave feedback.

Others viewing a SmartEvidence collection

The regular sharing permissions for collections apply to a SmartEvidence collection. As soon as someone has access to view the collection, they can also see the overview page and add feedback to an annotation. Regular users, i.e. those without staff or admin permissions, are not able to “rate” evidence. They can only leave feedback.