Actions

Difference between revisions of "Developer Area/Bug Status"

From Mahara Wiki

< Developer Area
(Bug importance: refactor this section to make it more readable)
 
(15 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The status, priority, milestone, tags in Launchpad are used by Mahara developers for effective bug management. This page describe the current use of these features.
+
The status, priority, milestone, tags in [https://bugs.launchpad.net/mahara Launchpad] are used by the Mahara core team for effective bug management. This page describe the current use of these features.
  
== Bug status ==
+
== Status ==
  
The bug status reflects the current state of the bug. When the new bug is reported it obtains "New" status automatically (unless specified otherwise) and then usually goes through a number other statuses depending on developers actions and decisions. The list below describe each of the statuses and provides useful hints on status chnages.
+
The bug status reflects the current state of the bug. When the new bug is reported it obtains "New" status automatically (unless specified otherwise) and then usually goes through a number other statuses depending on developer actions and decisions. The list below describes each of the statuses and provides useful hints on status changes.
  
 
*'''New''' - The status field is automatically set to "new" when a bug is created.
 
*'''New''' - The status field is automatically set to "new" when a bug is created.
 
*'''Triaged''' - Once we've looked at a bug and decided on a priority and a milestone, it's changed to "triaged". Triaged is for bugs we haven't tried to reproduce.
 
*'''Triaged''' - Once we've looked at a bug and decided on a priority and a milestone, it's changed to "triaged". Triaged is for bugs we haven't tried to reproduce.
*'''Confirmed''' - Status is changed to "confirmed" once we've been able to reproduce it, or to "incomplete" if we need more information to be able to reproduce it.
+
*'''Confirmed''' - Status is changed to "confirmed" once we've been able to reproduce it or to "incomplete" if we need more information to be able to reproduce it.
* '''Invalid , Wont fix, Opinion''' - are all statuses that effectively close the bug.  "Invalid" is for things we don't really think are bugs at all, or maybe they're bugs but not bugs in Mahara.  "Won't fix" is self-explanatory.  "Opinion" is a softer version of "won't fix", and indicates to the reporter that there is reasonable disagreement about whether the bug is a bug or should be fixed (and invites more comments).
+
* '''Invalid , Wont fix, Opinion''' - These are statuses that effectively close the bug.  "Invalid" is for things we don't really think are bugs at all, or maybe they're bugs but not bugs in Mahara. "Won't fix" is self-explanatory. "Opinion" is a softer version of "won't fix" and indicates to the reporter that there is reasonable disagreement about whether the bug is a bug or should be fixed (and invites more comments).
*'''In progress''' - is for bugs which are currently being worked on.
+
*'''In progress''' - The bug is currently being worked on.
*'''Fix committed''' - is for when a fix is committed to git. Bugs which have been fixed only in the commit author's local branch or personal clone repository, or [[Developer_Area/Contributing_Code|submitted for revision]] should not be marked as "Fix Committed" until they appear in the master (or stable) branch from which releases are created (which will happen when the change on [https://reviews.mahara.org/ revision system] is marked as merged).  '''Any time a bug is marked as "fix committed", the milestone field must also be updated''' to the next planned release milestone on the appropriate branch, and if the bug is still unassigned, it should be assigned to whoever provided the patch.
+
*'''Fix committed''' - A fix is committed to git. Bugs which have been fixed only in the commit author's local branch or personal clone repository, or [[Developer_Area/Contributing_Code|submitted for revision]] should not be marked as "Fix Committed" until they appear in the master (or stable) branch from which releases are created (which will happen when the change on [https://reviews.mahara.org/ the revision system] is marked as merged).  '''Any time a bug is marked as "fix committed", the milestone field must also be updated''' to the next planned release milestone on the appropriate branch, and if the bug is still unassigned, it should be assigned to whoever provided the patch.
*'''Fix Released''' - Status is flipped to "Fix Released" once there is a stable release (i.e. a tarball) which includes the fix (alphas, betas and release candidates don't count).
+
*'''Fix Released''' - Status is flipped to "Fix Released" once there is a stable release (i.e. a tarball), which includes the fix (alphas, betas and release candidates don't count).
  
Periodically the core development team will go through the open bugs, decide what should be fixed before the next release, and update the status and milestone fields accordingly.
+
Periodically, the core development team will go through the open bugs, decide what should be fixed before the next release and update the status and milestone fields accordingly.
  
== Bug milestone ==
+
== Importance ==
  
The milestone should always point to the next planned release milestone on the appropriate branch. Say, if the bug affects 1.4_STABLE release,the milestone should point to branch 1.4.x; if the the bug is actually a new feature on the master that is not relevant to previous release, then the milestone should point to next upcoming release. For the both of these cases above the bug status should remain as "Fix committed" once the bug is merged to master. When release is out, the bug status will be flicked to "Fix released". '''The situation when the milestone is missing but the bug status is "Fix committed" should be avoided'''. It makes the bug not listed anywhere and at the time of the actual release, its status will not be updated and the bug will not be listed in the release notes.
+
Importance is highly subjective, but here are some rough guidelines for how we use them in Mahara. A bug with the following importance might have some of these characteristics:
  
However, there is a single case when the bug may be omitted from the milestone. When the bug fix is not applied to the stable releases, but fixes the feature that has been previously introduced to master branch and is not relevant to previous stable release. In this case the bug can be marked as "Fix released" and milestone can be omitted. The feature itself will retain "Fix committed" status until next release and should point to next release milestone. This will reduce the noise in the release notes, as the new feature will be listed in the notes, but there is no much sense to list bug fixes applicable to this new feature until it is released.
+
* '''Critical''': Use only when we should drop everything and fix this ASAP
 +
** Prevents Mahara from functioning entirely
 +
** Causes data loss or other irreversible problems
 +
** Likely to be encountered by many users
 +
** Security flaw that doesn't require any user account on the site
 +
** Something that '''must''' get done before the next release
  
To sum up what is said above:
+
* '''High'''
* We must mark ALL new features and ALL bug fixes that apply to previous stable releases against the appropriate milestone.
+
** Obviously "broken" to end user. Visible error messages, wrong item being deleted, etc.
* All new features and bug fixes applicable to previous releases should have "Fix committed" status until the release will be out.
+
** Likely to be encountered by many users
* We omit from the milestone bugs that don't apply to the stable releases and fix the features that has been previously introduced to master branch only (after the last stable branch has been created), in which case once the fix is merged to master, the bug status must be updated directly to "Fix released" (with no milestone).
+
** Security flaw that requires a logged-in user account on the site
 +
** Usability problem that causes extreme inconvenience
  
== Bug importance ==
+
* '''Medium'''
 +
** Incorrect behavior, but a workaround is possible
 +
** Unlikely to be encountered by most users
 +
** Security flaw that requires an admin account and/or has limited consequences
 +
** Usability problem that causes major inconvenience
  
Importance is highly subjective, but here are some rough guidelines for how we use them in Mahara:
+
* '''Low'''
 +
** Works correctly, but doesn't follow best practices
 +
** Affects very few users
 +
** Easy workaround is available
 +
** A minor deviation from security "best practices", but not close to exploitable by itself
 +
** Usability problem that causes moderate or minor inconvenience
  
* '''Critical''': generally reserved for things that make the software unusable or pose a serious security risk for people running it. These are bugs where we drop everything we are doing to address them first.
+
* '''Wishlist'''
* '''High''': most of the bad bugs that inconvenience people.
+
** Not a bug
* '''Medium''': the default priority.
+
** Suggestions for new features & functionality
* '''Low''': minor bugs that will probably not get fixed unless someone picks it up and submits a patch.
 
* '''Wishlist''':  for new features or for enhancements (i.e. things that are not really bugs).
 
  
 
== Tags ==
 
== Tags ==
  
We encourage the use of tags to group bugs together!  If you triage a bug and think it's easy, but you don't plan to fix it immediately, tag it as "bite-sized". If we get enough bite-sized bugs, we can list them on [http://www.openhatch.org openhatch.org], or other sites that might encourage new contributors.
+
We encourage the use of tags to group bugs together! Some recommended tags:
 +
 
 +
* '''bite-sized''': A bug with a trivial fix. We use these during training to introduce new developers to Mahara's development process.
 +
 
 +
* '''snack-sized''': A bug that is slightly harder than bite-sized. We use these during more advanced Mahara development training events.
 +
 
 +
* '''ie9''', '''ie10''', '''ff''', '''chrome''', '''opera''', '''safari''', etc.: A tag that indicates a browser-specific bug.
 +
 
 +
* '''front-end''': A bug that requires a front-end developer, i.e. for CSS, HTML and Javascript changes.
 +
 
 +
* '''mysql''', '''postgres''': A bug that only affects one type of database.
 +
 
 +
* '''fastcgi''', '''nginx''', '''windows''': A bug that's only present when running Mahara in a non-standard environment.
 +
 
 +
* '''usability''': Indicates a problem area in Mahara's user interface; i.e. things that are difficult for users to figure out, or tasks that are annoying or difficult to accomplish.
 +
 
 +
* '''ldap''', '''note''', '''blogs''', '''tinymce''', '''groups''', etc.: Tags that indicate which plugin or part of Mahara a bug affects.
 +
 
 +
* '''regression''':
 +
** Functionality that used to work, but [https://en.wikipedia.org/wiki/Software_regression stopped working] after a known code change.
 +
** It's helpful to use '''[https://git-scm.com/docs/git-bisect git bisect]''' to find the exact code change that introduced the bug, so that you can correct the underlying problem.
 +
 
 +
== Security ==
 +
 
 +
Bugs with security implications should be marked as '''Private Security''' in the '''This report contains information that is...''' option. See this page for more info: [[Security]]
 +
 
 +
When the bug fix for a security bug is released, its status should be changed to '''Public Security'''. Or, if a bug is already widely known, or if it's a very low-priority security bug, it may be advisable to change it to '''Public Security''' even before its release.
 +
 
 +
== Milestone & "Affects" ==
 +
 
 +
=== Affected versions ===
 +
 
 +
The "Affects" column indicates which series' of Mahara the bug is present in. In most cases this will include the current development series. You should check to see whether the bug is also present in the current three [[SupportedVersions|supported releases]].
 +
 
 +
=== Milestone ===
 +
 
 +
The milestone is used for three main things:
 +
 
 +
# as a TODO list for an upcoming release
 +
# as a record of when a bug was fixed
 +
# as an extended changelog for a given release
 +
 
 +
You can decide whether & which milestone to apply based on the bug's status:
 +
 
 +
* '''Triaged''', '''Confirmed''', or '''In Progress''': Assign a milestone if:
 +
** We want to get it in the next possible release
 +
** And/or a developer has actually started implementation on it
 +
 
 +
* '''Fix committed''': Always assign a milestone, indicating the next stable release for the branch.
 +
 
 +
* '''Fix released'''
 +
** Usually a bug in this status will have passed through "Fix committed" first, so it will already have a milestone.
 +
** Exception: Sometimes an old bug in the tracker turns out to be no longer present in the latest release because some unrelated piece of code solved the problem. In those cases, you can change the bug's status to "Fix released" but give no milestone.
 +
 
 +
* '''Won't fix''': A bug in "Won't Fix" status should not have a milestone.
 +
 
 +
=== Backporting ===
 +
 
 +
We don't backport all bug fixes to all versions of Mahara. For the full details see: "[[SupportedVersions#Definition_of_support|Supported Versions: Definition of support]]"
 +
 
 +
Here's how we record backporting decisions in Launchpad. If you know a bug is present in older Mahara release:
 +
 
 +
* '''Supported''' release, '''Will''' backport: Mark the series as "affected" and give it a Milestone.
 +
* '''Supported''' release, '''Won't''' backport: Mark as "affected", no Milestone, status "Won't Fix". Marking it as affected and "Won't fix" lets us know we already made a backporting decision, so we don't accidentally rehash the same discussion.
 +
* '''Unsupported''' release: These are never backported. Sometimes, for informational purposes, it can be useful to mark the most recent unsupported & non-fixed branch as affected and "Won't Fix". But it's entirely optional. Example:
 +
** The current stable release is 16.10. Older supported releases are 16.04 and 15.10.
 +
** I find a bug that turns out to be a regression introduced in Mahara 1.6, and will be backported to 16.04 and 15.10.
 +
** Result: I may mark the 15.04 release as affected and "Won't Fix", to let people know at a glance that the bug goes back earlier than 15.10.
  
 
== Relevant links ==
 
== Relevant links ==
 +
 +
* Ubuntu's similar rules for prioritizing bugs: https://wiki.ubuntu.com/Bugs/Importance

Latest revision as of 12:05, 18 October 2016

The status, priority, milestone, tags in Launchpad are used by the Mahara core team for effective bug management. This page describe the current use of these features.

Status

The bug status reflects the current state of the bug. When the new bug is reported it obtains "New" status automatically (unless specified otherwise) and then usually goes through a number other statuses depending on developer actions and decisions. The list below describes each of the statuses and provides useful hints on status changes.

  • New - The status field is automatically set to "new" when a bug is created.
  • Triaged - Once we've looked at a bug and decided on a priority and a milestone, it's changed to "triaged". Triaged is for bugs we haven't tried to reproduce.
  • Confirmed - Status is changed to "confirmed" once we've been able to reproduce it or to "incomplete" if we need more information to be able to reproduce it.
  • Invalid , Wont fix, Opinion - These are statuses that effectively close the bug.  "Invalid" is for things we don't really think are bugs at all, or maybe they're bugs but not bugs in Mahara. "Won't fix" is self-explanatory. "Opinion" is a softer version of "won't fix" and indicates to the reporter that there is reasonable disagreement about whether the bug is a bug or should be fixed (and invites more comments).
  • In progress - The bug is currently being worked on.
  • Fix committed - A fix is committed to git. Bugs which have been fixed only in the commit author's local branch or personal clone repository, or submitted for revision should not be marked as "Fix Committed" until they appear in the master (or stable) branch from which releases are created (which will happen when the change on the revision system is marked as merged).  Any time a bug is marked as "fix committed", the milestone field must also be updated to the next planned release milestone on the appropriate branch, and if the bug is still unassigned, it should be assigned to whoever provided the patch.
  • Fix Released - Status is flipped to "Fix Released" once there is a stable release (i.e. a tarball), which includes the fix (alphas, betas and release candidates don't count).

Periodically, the core development team will go through the open bugs, decide what should be fixed before the next release and update the status and milestone fields accordingly.

Importance

Importance is highly subjective, but here are some rough guidelines for how we use them in Mahara. A bug with the following importance might have some of these characteristics:

  • Critical: Use only when we should drop everything and fix this ASAP
    • Prevents Mahara from functioning entirely
    • Causes data loss or other irreversible problems
    • Likely to be encountered by many users
    • Security flaw that doesn't require any user account on the site
    • Something that must get done before the next release
  • High
    • Obviously "broken" to end user. Visible error messages, wrong item being deleted, etc.
    • Likely to be encountered by many users
    • Security flaw that requires a logged-in user account on the site
    • Usability problem that causes extreme inconvenience
  • Medium
    • Incorrect behavior, but a workaround is possible
    • Unlikely to be encountered by most users
    • Security flaw that requires an admin account and/or has limited consequences
    • Usability problem that causes major inconvenience
  • Low
    • Works correctly, but doesn't follow best practices
    • Affects very few users
    • Easy workaround is available
    • A minor deviation from security "best practices", but not close to exploitable by itself
    • Usability problem that causes moderate or minor inconvenience
  • Wishlist
    • Not a bug
    • Suggestions for new features & functionality

Tags

We encourage the use of tags to group bugs together! Some recommended tags:

  • bite-sized: A bug with a trivial fix. We use these during training to introduce new developers to Mahara's development process.
  • snack-sized: A bug that is slightly harder than bite-sized. We use these during more advanced Mahara development training events.
  • ie9, ie10, ff, chrome, opera, safari, etc.: A tag that indicates a browser-specific bug.
  • front-end: A bug that requires a front-end developer, i.e. for CSS, HTML and Javascript changes.
  • mysql, postgres: A bug that only affects one type of database.
  • fastcgi, nginx, windows: A bug that's only present when running Mahara in a non-standard environment.
  • usability: Indicates a problem area in Mahara's user interface; i.e. things that are difficult for users to figure out, or tasks that are annoying or difficult to accomplish.
  • ldap, note, blogs, tinymce, groups, etc.: Tags that indicate which plugin or part of Mahara a bug affects.
  • regression:
    • Functionality that used to work, but stopped working after a known code change.
    • It's helpful to use git bisect to find the exact code change that introduced the bug, so that you can correct the underlying problem.

Security

Bugs with security implications should be marked as Private Security in the This report contains information that is... option. See this page for more info: Security

When the bug fix for a security bug is released, its status should be changed to Public Security. Or, if a bug is already widely known, or if it's a very low-priority security bug, it may be advisable to change it to Public Security even before its release.

Milestone & "Affects"

Affected versions

The "Affects" column indicates which series' of Mahara the bug is present in. In most cases this will include the current development series. You should check to see whether the bug is also present in the current three supported releases.

Milestone

The milestone is used for three main things:

  1. as a TODO list for an upcoming release
  2. as a record of when a bug was fixed
  3. as an extended changelog for a given release

You can decide whether & which milestone to apply based on the bug's status:

  • Triaged, Confirmed, or In Progress: Assign a milestone if:
    • We want to get it in the next possible release
    • And/or a developer has actually started implementation on it
  • Fix committed: Always assign a milestone, indicating the next stable release for the branch.
  • Fix released
    • Usually a bug in this status will have passed through "Fix committed" first, so it will already have a milestone.
    • Exception: Sometimes an old bug in the tracker turns out to be no longer present in the latest release because some unrelated piece of code solved the problem. In those cases, you can change the bug's status to "Fix released" but give no milestone.
  • Won't fix: A bug in "Won't Fix" status should not have a milestone.

Backporting

We don't backport all bug fixes to all versions of Mahara. For the full details see: "Supported Versions: Definition of support"

Here's how we record backporting decisions in Launchpad. If you know a bug is present in older Mahara release:

  • Supported release, Will backport: Mark the series as "affected" and give it a Milestone.
  • Supported release, Won't backport: Mark as "affected", no Milestone, status "Won't Fix". Marking it as affected and "Won't fix" lets us know we already made a backporting decision, so we don't accidentally rehash the same discussion.
  • Unsupported release: These are never backported. Sometimes, for informational purposes, it can be useful to mark the most recent unsupported & non-fixed branch as affected and "Won't Fix". But it's entirely optional. Example:
    • The current stable release is 16.10. Older supported releases are 16.04 and 15.10.
    • I find a bug that turns out to be a regression introduced in Mahara 1.6, and will be backported to 16.04 and 15.10.
    • Result: I may mark the 15.04 release as affected and "Won't Fix", to let people know at a glance that the bug goes back earlier than 15.10.

Relevant links