Actions

Difference between revisions of "Developer Area/Release Instructions"

From Mahara Wiki

< Developer Area
(37 intermediate revisions by 6 users not shown)
Line 8: Line 8:
 
* [[Developer Area/Release Instructions/Release Candidate|Release candidate]]
 
* [[Developer Area/Release Instructions/Release Candidate|Release candidate]]
  
 +
= Apply for CVE numbers (for security fixes) before publication of the release =
  
====Pre-requisites:====
+
At least a few days before the release (best as soon as a bug report exists) apply for one or more [https://cve.mitre.org/cve/request_id.html CVE numbers] for reported security issues. Each issue needs to have its own CVE number.
 +
 
 +
While CVE numbers are not a requirement, they are useful for administrators who monitor security announcements. It is a good practice thing to do.
 +
 
 +
Use the [https://cveform.mitre.org/ request form with MITRE] as we don't put Mahara into a distro anymore.
 +
 
 +
[[File:File_Siren.gif]]'''IMPORTANT !!!'''[[File:File_Siren.gif]]
 +
 
 +
Please note that this form can '''ONLY''' be used if the issue hasn't been made public yet.
 +
 
 +
* CVE numbers that we receive are embargoed and do not leak details until we let MITRE know that they can be published.
 +
* Once you received a CVE number, please add it to the Launchpad bug for that issue so it can be tracked. It takes at least a couple of days after having received the CVE numbers until Launchpad accepts the CVE number via the CVE number field.
 +
* Add the description, vulnerability, impact (if needed) and the discoverer credit to the Launchpad bug so that the information is readily available for the security announcement. '''Catalyst NZ staff''' also have access to that information on Seafile along with a running record of which bugs have been reported to MITRE (or others if needed).
 +
 
 +
= 1 week prior to the release =
 +
 
 +
Let Mahara Partners know in the [https://mahara.org/interaction/forum/view.php?id=1136 Partner forum] that a release is upcoming. Details are not shared though as the reports are still embargoed.
 +
 
 +
=Pre-requisites for doing the release=
  
 
* You must have a [[Developer Area/Release Instructions/Creating a GPG key|valid GPG key]] and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys).
 
* You must have a [[Developer Area/Release Instructions/Creating a GPG key|valid GPG key]] and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys).
* You must have [http://www.piware.de/2009/09/automated-release-tarball-upload-to-launchpad/ lp-upload-project] (ubuntu-dev-tools in maverick or earlier) configured to '''write public information''' on Launchpad on your behalf:
+
* You must have lp-upload-project installed:
 +
 
 +
  apt-get install lptools
 +
 
 +
OLD INFORMATION - please ignore
 +
* Configure lptools to '''write public information''' on Launchpad on your behalf:
  
 
  manage-credentials create -c ubuntu-dev-tools -l 2
 
  manage-credentials create -c ubuntu-dev-tools -l 2
  
(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have Gone Away. You will need to install lp-tools to proceed with the script usage.)
+
(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.)
  
===1. Make sure your mahara-scripts checkout is up to date===
+
= Make sure your mahara-scripts checkout is up to date=
  
You need to have an up-to-date version of the [http://gitorious.org/mahara/mahara-scripts mahara-scripts] repo in order to get the latest copy of the [http://gitorious.org/mahara/mahara-scripts/blobs/master/release.sh release script]. That's the only repo you need to have since the release script will clone the gitorious repo all by itself.
+
You need to have an up-to-date version of the [https://git.mahara.org/scripts/mahara-scripts mahara-scripts] repo in order to get the latest copy of the [http://git.mahara.org/scripts/mahara-scripts/blobs/master/release.php release script]. That's the only repo you need to have since the release script will clone the git.mahara.org repo all by itself.
  
===2. Run the first part of the release script===
+
= Run the first part of the release script=
 +
 
 +
[[File:File_Siren.gif]]'''IMPORTANT !!!'''[[File:File_Siren.gif]]
 +
 
 +
(ignore for release candidates)'''Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.'''
  
 
Using the script you got in the previous step, run the first step of it like this:
 
Using the script you got in the previous step, run the first step of it like this:
  
  ./release.sh X.Y.Z X.Y_STABLE
+
  ./release.php X.Y.Z X.Y_STABLE (For rc X.Yrc1)
 
   
 
   
  
 
This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.
 
This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.
  
Note that the repos that will be pushed back to gitorious are live in your /tmp. '''Do not reboot your machine''' in the middle of doing a release or you will have to run through the release script again.
+
Note that the repos that will be pushed back to git.mahara.org are live in your /tmp. '''Do not reboot your machine''' in the middle of doing a release or you will have to run through the release script again.
  
===3. Testing===
+
Note: For release candidates the commit message should be empty.
  
 +
= Testing=
 +
 +
As we do continuous behat testing throughout development we only need to do the basic manual testing.
 
Here is the testing that needs to be done using the tarballs you generated in the last step:
 
Here is the testing that needs to be done using the tarballs you generated in the last step:
  
# install the [http://seleniumhq.org/projects/ide/ Selenium firefox plugin] if you don't already have it
+
# untar the tar that release.php generated somewhere and make sure you can connect to it via web installer
# untar the tar that release.sh generated somewhere and make sure you can connect to it and login as admin
+
# drop the db and create a new blank one to test install
# in a mahara repository, check the right branch out (X.Y_STABLE for X.Y.Z)
 
# in that repository, cd test/selenium/ &amp;&amp; ./generate-testsuites.py
 
# in firefox: Tools | Selenium IDE
 
# in Selenium: File | Open Test Suite
 
# open test/selenium/TestSuite.html
 
# set the Base URL to where you can connect to that Mahara site
 
# drop the db and create a new blank one (the selenium tests start by running the install)
 
# hit the "play entire test suite" green "play" button
 
  
Note: avoid using firefox while it runs, it steals tabs and gets confused.
+
The install should be run 4 times, both on MySQL and Postgres and both via the web interface and CLI script.
  
The test suite should be run both on MySQL and Postgres. Then run these manual upgrade tests (no need to run the Selenium test suite):
+
Once we are all happy with that we need to run these manual upgrade tests:
  
# install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
+
# install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres, via web interface, and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
 
# repeat that last step on MySQL
 
# repeat that last step on MySQL
  
===4. Create the release on Launchpad===
+
= Create the release on Launchpad=
  
At https://launchpad.net/mahara/+milestone/X.Y.Z
+
Go to https://launchpad.net/mahara/+milestone/X.Y.Z
  
Click on "create release" and give it today's date.
+
First move any bugs that are not going to be part of this release to the next milestone in the series, eg if this is X.Y.3 then move to X.Y.4. You may need to make the milestone if it doesn't already exist via https://launchpad.net/mahara/X.Y and using the 'create milestone' link.
 +
 
 +
Then click on "create release" and give it today's date.
  
 
Leave "Keep the X.Y.Z milestone active." '''unchecked'''.
 
Leave "Keep the X.Y.Z milestone active." '''unchecked'''.
Line 65: Line 90:
 
Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)
 
Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)
  
===5. Sign and upload the tarballs===
+
= Sign and upload the tarballs=
  
 
Run the next part of the release script, i.e. the script that was spit out by the release script.
 
Run the next part of the release script, i.e. the script that was spit out by the release script.
Line 71: Line 96:
 
  ./release-X.Y.Z-cleanup.sh
 
  ./release-X.Y.Z-cleanup.sh
  
When prompted by lp-project-upload for release notes and a changelog, don't enter anything there since you've already put them on Launchpad in the last step. You will be prompted twice for each file upload (6 times in total).
+
You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage.
  
===6. Update bugs on the tracker===
+
= Update bugs on the tracker=
  
On [https://bugs.launchpad.net/mahara Launchpad], click on all of the bugs targetted for that milestone and move them from "Fix committed" to "Fix released".
+
On [https://bugs.launchpad.net/mahara Launchpad], click on all of the bugs targeted for that milestone and move them from "Fix committed" to "Fix released".
  
===7. Put the release notes on the wiki page===
+
Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.
 +
 
 +
= Put the release notes on the wiki page=
  
 
This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.
 
This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.
Line 83: Line 110:
 
[[Release Notes|Release notes page]]
 
[[Release Notes|Release notes page]]
  
===8. Quick announcements===
+
= Language packs=
  
* Change the topic of #mahara
+
[[Developer_Area/Language_Packs/Launchpad_Branching | Create a new translation branch on launchpad.net and update the language scripts]] for the new release so that they appear on http://langpacks.mahara.org
* Put a note on Twitter and Identica.
 
* Update [http://freecode.com/projects/mahara freecode] to report the new release
 
  
You may need to ask a previous release manager to get access to some of these accounts.
+
= Security forum posts=
 +
 
 +
If this release includes security fixes, then publish advisories on the [https://mahara.org/interaction/forum/view.php?id=43 security forum].
 +
 
 +
Here's a sample forum post:
 +
 
 +
Title: Security issue relating to (Category) <1.0.5, <1.1.3, <1.2.0
 +
 +
<description of issue>
 +
 +
Category: (XSS, Access control, Privacy, Spam, Privilege escalation, Session fixation, Etc.)
 +
Severity: (Critical, High, Medium, Low)
 +
Versions affected: <1.0.5, <1.1.3, <1.2.0
 +
Reported by:
 +
Bug reports: (launchpad URL)
 +
CVE reference: (If you have one; if not, this can be added later)
 +
 
 +
Sticky: No
 +
Closed: No
  
===9. Security advisories===
+
It can be tricky to decide what to put down for "Category". This should be the "category" in the security sense (like a category on the OWASP site) rather than category of Mahara functionality. Some categories we have used in the past:
  
If this release includes security fixes, then publish the advisories on the [http://mahara.org/interaction/forum/view.php?id=43 security forum].
+
* Injection
 +
* Password security
 +
* Cross-site scripting (XSS)
 +
* Cross-site request forgery (CSRF)
 +
* Access control (i.e., users being able to bypass content sharing permissions)
 +
* Privacy (i.e., we tell a user that a piece of information is private but then we accidentally display it publicly)
 +
* Session fixation
 +
* Privilege escalation
 +
* Disclosure of system information
 +
* Stored XSS
 +
* Session management
 +
* User Authorization
  
===10. Update the news at [http://www.mahara.org/ http://mahara.org/]===
+
= News forum posts =
  
Posting in the news forum does this.
+
Post announcement in the news forum at [http://www.mahara.org/ http://mahara.org/]
  
 
Include links to the security advisories and a link to the [https://launchpad.net/mahara/+download download page] on Launchpad.
 
Include links to the security advisories and a link to the [https://launchpad.net/mahara/+download download page] on Launchpad.
Line 103: Line 157:
 
Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.
 
Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.
  
===11. Update the demo.mahara.org install (currently Catalyst hosted)===
+
Here is a sample minor release announcement:
 +
 
 +
Title: Mahara security release: 15.04.1, 1.10.4, 1.9.6
 +
 +
Hi all,
 +
 +
Today we're announcing new minor releases for the Mahara 15.04, Mahara 1.10, and Mahara 1.9 series.
 +
 +
These releases include <X> security fix<es> relating to <list of issues>, as well as several bug
 +
fixes. Site administrators are advised to upgrade as soon as possible.
 +
 +
Release packages, as well as a full list of changes, are available on our Launchpad project:
 +
 +
    15.04.1: https://launchpad.net/mahara/+milestone/15.04.1
 +
    1.10.4: https://launchpad.net/mahara/+milestone/1.10.4
 +
    1.9.6: https://launchpad.net/mahara/+milestone/1.9.6
 +
 
 +
<if this is the last release for a series:>
 +
Note: the 1.9.6 release is the last supported release in the 1.9 series.
 +
 
 +
Special thanks, as always, to everyone who helped by reporting bugs, submitting patches, and testing
 +
the changes!
 +
 
 +
Sticky: No
 +
Closed: Yes
 +
 
 +
= Quick announcements=
 +
 
 +
* Change the topic of #mahara and include a link to forum topic on #mahara-dev
 +
* Put a note on:
 +
** [https://twitter.com/maharaproject Twitter]
 +
** [http://freecode.com/projects/mahara freecode] (formerly Freshmeat)
 +
** [https://www.linkedin.com/groups?home=&gid=2037561 LinkedIn]
 +
** Facebook [https://www.facebook.com/groups/maharausergroup/ MUG], [https://www.facebook.com/groups/MaharaDE13/ Mahara DE], [https://www.facebook.com/groups/moodlemaharameetup/ Moodle-Mahara Meetup], [https://www.facebook.com/groups/mahara.users/ Mahara in Japanese]
 +
 
 +
You may need to ask a previous release manager to get access to some of these accounts.
 +
 
 +
=Update community sites install (currently Catalyst hosted)=
 +
 
 +
* demo.mahara.org
 +
* mahara.org
 +
* master.dev.mahara.org
  
demo.mahara.org is in the client repository, on the demo-mahara-org branch.
+
They are all deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs on the demo site. As a guideline, language packs that are 90% or more translated should be installed.
  
git fetch origin
+
=Update MITRE about release for CVE number publication=
  git merge origin/1.0_STABLE
 
  dch -v 1.0.X~testing-1 (changelog: 1.0.X)
 
  make
 
  
Send it to the e-learning repo and install it.
+
If the release included security updates for which CVE numbers had been issued, [https://cveform.mitre.org/ request an update to the CVE number] so it can be published.
  
===Subpages===
+
=Subpages=
  
 
* [[Developer_Area/Release_Instructions/Creating_a_GPG_key|Creating a GPG key]]
 
* [[Developer_Area/Release_Instructions/Creating_a_GPG_key|Creating a GPG key]]

Revision as of 23:55, 30 May 2018

This document details the procedure to follow when building a release for general download. Other documents related to this one include the Version Numbering policy and the Release Policy.

For the purposes of these instructions, assume we are releasing version X.Y.Z of Mahara.

Also see these pages for more instructions:

Apply for CVE numbers (for security fixes) before publication of the release

At least a few days before the release (best as soon as a bug report exists) apply for one or more CVE numbers for reported security issues. Each issue needs to have its own CVE number.

While CVE numbers are not a requirement, they are useful for administrators who monitor security announcements. It is a good practice thing to do.

Use the request form with MITRE as we don't put Mahara into a distro anymore.

File Siren.gifIMPORTANT !!!File Siren.gif

Please note that this form can ONLY be used if the issue hasn't been made public yet.

  • CVE numbers that we receive are embargoed and do not leak details until we let MITRE know that they can be published.
  • Once you received a CVE number, please add it to the Launchpad bug for that issue so it can be tracked. It takes at least a couple of days after having received the CVE numbers until Launchpad accepts the CVE number via the CVE number field.
  • Add the description, vulnerability, impact (if needed) and the discoverer credit to the Launchpad bug so that the information is readily available for the security announcement. Catalyst NZ staff also have access to that information on Seafile along with a running record of which bugs have been reported to MITRE (or others if needed).

1 week prior to the release

Let Mahara Partners know in the Partner forum that a release is upcoming. Details are not shared though as the reports are still embargoed.

Pre-requisites for doing the release

 apt-get install lptools

OLD INFORMATION - please ignore

  • Configure lptools to write public information on Launchpad on your behalf:
manage-credentials create -c ubuntu-dev-tools -l 2

(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.)

Make sure your mahara-scripts checkout is up to date

You need to have an up-to-date version of the mahara-scripts repo in order to get the latest copy of the release script. That's the only repo you need to have since the release script will clone the git.mahara.org repo all by itself.

Run the first part of the release script

File Siren.gifIMPORTANT !!!File Siren.gif

(ignore for release candidates)Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.

Using the script you got in the previous step, run the first step of it like this:

./release.php X.Y.Z X.Y_STABLE (For rc X.Yrc1)

This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.

Note that the repos that will be pushed back to git.mahara.org are live in your /tmp. Do not reboot your machine in the middle of doing a release or you will have to run through the release script again.

Note: For release candidates the commit message should be empty.

Testing

As we do continuous behat testing throughout development we only need to do the basic manual testing. Here is the testing that needs to be done using the tarballs you generated in the last step:

  1. untar the tar that release.php generated somewhere and make sure you can connect to it via web installer
  2. drop the db and create a new blank one to test install

The install should be run 4 times, both on MySQL and Postgres and both via the web interface and CLI script.

Once we are all happy with that we need to run these manual upgrade tests:

  1. install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres, via web interface, and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
  2. repeat that last step on MySQL

Create the release on Launchpad

Go to https://launchpad.net/mahara/+milestone/X.Y.Z

First move any bugs that are not going to be part of this release to the next milestone in the series, eg if this is X.Y.3 then move to X.Y.4. You may need to make the milestone if it doesn't already exist via https://launchpad.net/mahara/X.Y and using the 'create milestone' link.

Then click on "create release" and give it today's date.

Leave "Keep the X.Y.Z milestone active." unchecked.

Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)

Sign and upload the tarballs

Run the next part of the release script, i.e. the script that was spit out by the release script.

./release-X.Y.Z-cleanup.sh

You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage.

Update bugs on the tracker

On Launchpad, click on all of the bugs targeted for that milestone and move them from "Fix committed" to "Fix released".

Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.

Put the release notes on the wiki page

This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.

Release notes page

Language packs

Create a new translation branch on launchpad.net and update the language scripts for the new release so that they appear on http://langpacks.mahara.org

Security forum posts

If this release includes security fixes, then publish advisories on the security forum.

Here's a sample forum post:

Title: Security issue relating to (Category) <1.0.5, <1.1.3, <1.2.0

<description of issue>

Category: (XSS, Access control, Privacy, Spam, Privilege escalation, Session fixation, Etc.)
Severity: (Critical, High, Medium, Low)
Versions affected: <1.0.5, <1.1.3, <1.2.0
Reported by:
Bug reports: (launchpad URL)
CVE reference: (If you have one; if not, this can be added later) 
Sticky: No
Closed: No

It can be tricky to decide what to put down for "Category". This should be the "category" in the security sense (like a category on the OWASP site) rather than category of Mahara functionality. Some categories we have used in the past:

  • Injection
  • Password security
  • Cross-site scripting (XSS)
  • Cross-site request forgery (CSRF)
  • Access control (i.e., users being able to bypass content sharing permissions)
  • Privacy (i.e., we tell a user that a piece of information is private but then we accidentally display it publicly)
  • Session fixation
  • Privilege escalation
  • Disclosure of system information
  • Stored XSS
  • Session management
  • User Authorization

News forum posts

Post announcement in the news forum at http://mahara.org/

Include links to the security advisories and a link to the download page on Launchpad.

Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.

Here is a sample minor release announcement:

Title: Mahara security release: 15.04.1, 1.10.4, 1.9.6

Hi all,

Today we're announcing new minor releases for the Mahara 15.04, Mahara 1.10, and Mahara 1.9 series.

These releases include <X> security fix<es> relating to <list of issues>, as well as several bug 
fixes. Site administrators are advised to upgrade as soon as possible.

Release packages, as well as a full list of changes, are available on our Launchpad project:

   15.04.1: https://launchpad.net/mahara/+milestone/15.04.1
   1.10.4: https://launchpad.net/mahara/+milestone/1.10.4
   1.9.6: https://launchpad.net/mahara/+milestone/1.9.6
<if this is the last release for a series:>
Note: the 1.9.6 release is the last supported release in the 1.9 series. 
Special thanks, as always, to everyone who helped by reporting bugs, submitting patches, and testing 
the changes!
Sticky: No
Closed: Yes

Quick announcements

You may need to ask a previous release manager to get access to some of these accounts.

Update community sites install (currently Catalyst hosted)

  • demo.mahara.org
  • mahara.org
  • master.dev.mahara.org

They are all deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs on the demo site. As a guideline, language packs that are 90% or more translated should be installed.

Update MITRE about release for CVE number publication

If the release included security updates for which CVE numbers had been issued, request an update to the CVE number so it can be published.

Subpages