Actions

Difference between revisions of "Developer Area/Release Instructions"

From Mahara Wiki

< Developer Area
(14 intermediate revisions by 3 users not shown)
Line 8: Line 8:
 
* [[Developer Area/Release Instructions/Release Candidate|Release candidate]]
 
* [[Developer Area/Release Instructions/Release Candidate|Release candidate]]
  
 +
= Apply for CVE numbers (for security fixes) =
  
====Pre-requisites:====
+
At least a few days before the release (best as soon as a bug report exists) apply for one or more [https://cve.mitre.org/cve/request_id.html CVE numbers] for reported security issues. Each issue needs to have its own CVE number.
 +
 
 +
While CVE numbers are not a requirement, they are useful for administrators who monitor security announcements. It is a good practice thing to do.
 +
 
 +
Use the [https://cveform.mitre.org/ request form with MITRE] as we don't put Mahara into a distro anymore.
 +
 
 +
[[File:File_Siren.gif]]'''IMPORTANT !!!'''[[File:File_Siren.gif]]
 +
 
 +
Please note that this form can '''ONLY''' be used if the issue hasn't been made public yet.
 +
 
 +
* CVE numbers that we receive are embargoed and do not leak details until we let MITRE know that they can be published.
 +
* Once you received a CVE number, please add it to the Launchpad bug for that issue so it cn be tracked.
 +
 
 +
=Pre-requisites for doing the release=
  
 
* You must have a [[Developer Area/Release Instructions/Creating a GPG key|valid GPG key]] and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys).
 
* You must have a [[Developer Area/Release Instructions/Creating a GPG key|valid GPG key]] and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys).
Line 16: Line 30:
 
   apt-get install lptools
 
   apt-get install lptools
  
 +
OLD INFORMATION - please ignore
 
* Configure lptools to '''write public information''' on Launchpad on your behalf:
 
* Configure lptools to '''write public information''' on Launchpad on your behalf:
  
Line 22: Line 37:
 
(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.)
 
(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.)
  
===1. Make sure your mahara-scripts checkout is up to date===
+
= Make sure your mahara-scripts checkout is up to date=
  
You need to have an up-to-date version of the [http://gitorious.org/mahara/mahara-scripts mahara-scripts] repo in order to get the latest copy of the [http://gitorious.org/mahara/mahara-scripts/blobs/master/release.sh release script]. That's the only repo you need to have since the release script will clone the gitorious repo all by itself.
+
You need to have an up-to-date version of the [https://git.mahara.org/scripts/mahara-scripts mahara-scripts] repo in order to get the latest copy of the [http://git.mahara.org/scripts/mahara-scripts/blobs/master/release.php release script]. That's the only repo you need to have since the release script will clone the git.mahara.org repo all by itself.
  
===2. Run the first part of the release script===
+
= Run the first part of the release script=
  
[[File:siren.gif]]'''IMPORTANT !!!'''[[File:siren.gif]]
+
[[File:File_Siren.gif]]'''IMPORTANT !!!'''[[File:File_Siren.gif]]
  
'''Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.'''
+
(ignore for release candidates)'''Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.'''
  
 
Using the script you got in the previous step, run the first step of it like this:
 
Using the script you got in the previous step, run the first step of it like this:
  
  ./release.sh X.Y.Z X.Y_STABLE
+
  ./release.php X.Y.Z X.Y_STABLE (For rc X.Yrc1)
 
   
 
   
  
 
This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.
 
This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.
  
Note that the repos that will be pushed back to gitorious are live in your /tmp. '''Do not reboot your machine''' in the middle of doing a release or you will have to run through the release script again.
+
Note that the repos that will be pushed back to git.mahara.org are live in your /tmp. '''Do not reboot your machine''' in the middle of doing a release or you will have to run through the release script again.
  
===3. Testing===
+
Note: For release candidates the commit message should be empty.
  
 +
= Testing=
 +
 +
As we do continuous behat testing throughout development we only need to do the basic manual testing.
 
Here is the testing that needs to be done using the tarballs you generated in the last step:
 
Here is the testing that needs to be done using the tarballs you generated in the last step:
  
# install the [http://seleniumhq.org/projects/ide/ Selenium firefox plugin] if you don't already have it
+
# untar the tar that release.php generated somewhere and make sure you can connect to it via web installer
# untar the tar that release.sh generated somewhere and make sure you can connect to it and login as admin
+
# drop the db and create a new blank one to test install
# in a mahara repository, check the right branch out (X.Y_STABLE for X.Y.Z)
 
# in that repository, cd test/selenium/ &amp;&amp; ./generate-testsuites.py
 
# in firefox: Tools | Selenium IDE
 
# in Selenium: File | Open Test Suite
 
# open test/selenium/TestSuite.html
 
# set the Base URL to where you can connect to that Mahara site
 
# drop the db and create a new blank one (the selenium tests start by running the install)
 
# hit the "play entire test suite" green "play" button
 
  
Note: Avoid using firefox while selenium runs, it steals tabs and gets confused.
+
The install should be run 4 times, both on MySQL and Postgres and both via the web interface and CLI script.
Note: Firebug can also confuse selenium, so it's best to turn it off while running the suite.
 
  
The test suite should be run both on MySQL and Postgres. Then run these manual upgrade tests (no need to run the Selenium test suite):
+
Once we are all happy with that we need to run these manual upgrade tests:
  
# install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
+
# install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres, via web interface, and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
 
# repeat that last step on MySQL
 
# repeat that last step on MySQL
  
===4. Create the release on Launchpad===
+
= Create the release on Launchpad=
  
At https://launchpad.net/mahara/+milestone/X.Y.Z
+
Go to https://launchpad.net/mahara/+milestone/X.Y.Z
  
Click on "create release" and give it today's date.
+
First move any bugs that are not going to be part of this release to the next milestone in the series, eg if this is X.Y.3 then move to X.Y.4. You may need to make the milestone if it doesn't already exist via https://launchpad.net/mahara/X.Y and using the 'create milestone' link.
 +
 
 +
Then click on "create release" and give it today's date.
  
 
Leave "Keep the X.Y.Z milestone active." '''unchecked'''.
 
Leave "Keep the X.Y.Z milestone active." '''unchecked'''.
Line 74: Line 85:
 
Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)
 
Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)
  
===5. Sign and upload the tarballs===
+
= Sign and upload the tarballs=
  
 
Run the next part of the release script, i.e. the script that was spit out by the release script.
 
Run the next part of the release script, i.e. the script that was spit out by the release script.
Line 82: Line 93:
 
You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage.
 
You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage.
  
===6. Update bugs on the tracker===
+
= Update bugs on the tracker=
  
On [https://bugs.launchpad.net/mahara Launchpad], click on all of the bugs targetted for that milestone and move them from "Fix committed" to "Fix released".
+
On [https://bugs.launchpad.net/mahara Launchpad], click on all of the bugs targeted for that milestone and move them from "Fix committed" to "Fix released".
  
 
Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.
 
Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.
  
===7. Put the release notes on the wiki page===
+
= Put the release notes on the wiki page=
  
 
This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.
 
This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.
Line 94: Line 105:
 
[[Release Notes|Release notes page]]
 
[[Release Notes|Release notes page]]
  
===8. Language packs===
+
= Language packs=
  
 
[[Developer_Area/Language_Packs/Launchpad_Branching | Create a new translation branch on launchpad.net and update the language scripts]] for the new release so that they appear on http://langpacks.mahara.org
 
[[Developer_Area/Language_Packs/Launchpad_Branching | Create a new translation branch on launchpad.net and update the language scripts]] for the new release so that they appear on http://langpacks.mahara.org
  
===9. Security advisories===
+
= Security forum posts=
 +
 
 +
If this release includes security fixes, then publish advisories on the [https://mahara.org/interaction/forum/view.php?id=43 security forum].
 +
 
 +
Here's a sample forum post:
 +
 
 +
Title: Security issue relating to (Category) <1.0.5, <1.1.3, <1.2.0
 +
 +
<description of issue>
 +
 +
Category: (XSS, Access control, Privacy, Spam, Privilege escalation, Session fixation, Etc.)
 +
Severity: (Critical, High, Medium, Low)
 +
Versions affected: <1.0.5, <1.1.3, <1.2.0
 +
Reported by:
 +
Bug reports: (launchpad URL)
 +
CVE reference: (If you have one; if not, this can be added later)
 +
 
 +
Sticky: No
 +
Closed: No
 +
 
 +
It can be tricky to decide what to put down for "Category". This should be the "category" in the security sense (like a category on the OWASP site) rather than category of Mahara functionality. Some categories we have used in the past:
  
If this release includes security fixes, then publish the advisories on the [https://mahara.org/interaction/forum/view.php?id=43 security forum].
+
* Injection
 +
* Password security
 +
* Cross-site scripting (XSS)
 +
* Cross-site request forgery (CSRF)
 +
* Access control (i.e., users being able to bypass content sharing permissions)
 +
* Privacy (i.e., we tell a user that a piece of information is private but then we accidentally display it publicly)
 +
* Session fixation
 +
* Privilege escalation
 +
* Disclosure of system information
 +
* Stored XSS
 +
* Session management
 +
* User Authorization
  
===10. Update the news at [http://www.mahara.org/ http://mahara.org/]===
+
= News forum posts =
  
Posting in the news forum does this.
+
Post announcement in the news forum at [http://www.mahara.org/ http://mahara.org/]
  
 
Include links to the security advisories and a link to the [https://launchpad.net/mahara/+download download page] on Launchpad.
 
Include links to the security advisories and a link to the [https://launchpad.net/mahara/+download download page] on Launchpad.
Line 110: Line 152:
 
Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.
 
Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.
  
===11. Quick announcements===
+
Here is a sample minor release announcement:
 +
 
 +
Title: Mahara security release: 15.04.1, 1.10.4, 1.9.6
 +
 +
Hi all,
 +
 +
Today we're announcing new minor releases for the Mahara 15.04, Mahara 1.10, and Mahara 1.9 series.
 +
 +
These releases include <X> security fix<es> relating to <list of issues>, as well as several bug
 +
fixes. Site administrators are advised to upgrade as soon as possible.
 +
 +
Release packages, as well as a full list of changes, are available on our Launchpad project:
 +
 +
    15.04.1: https://launchpad.net/mahara/+milestone/15.04.1
 +
    1.10.4: https://launchpad.net/mahara/+milestone/1.10.4
 +
    1.9.6: https://launchpad.net/mahara/+milestone/1.9.6
 +
 
 +
<if this is the last release for a series:>
 +
Note: the 1.9.6 release is the last supported release in the 1.9 series.
 +
 
 +
Special thanks, as always, to everyone who helped by reporting bugs, submitting patches, and testing
 +
the changes!
 +
 
 +
Sticky: No
 +
Closed: Yes
 +
 
 +
= Quick announcements=
  
* Create a topic in the Mahara news forum (and security forum if it's a security release) - see below for more information.
 
 
* Change the topic of #mahara and include a link to forum topic on #mahara-dev
 
* Change the topic of #mahara and include a link to forum topic on #mahara-dev
 
* Put a note on:
 
* Put a note on:
Line 122: Line 189:
 
You may need to ask a previous release manager to get access to some of these accounts.
 
You may need to ask a previous release manager to get access to some of these accounts.
  
===12. Update the demo.mahara.org install (currently Catalyst hosted)===
+
=Update community sites install (currently Catalyst hosted)=
 +
 
 +
* demo.mahara.org
 +
* mahara.org
 +
* master.dev.mahara.org
  
demo.mahara.org is deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs. As a guideline, language packs that are 90% or more translated should be installed.
+
They are all deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs on the demo site. As a guideline, language packs that are 90% or more translated should be installed.
  
===13. Close the Y.M.0 milestone on Launchpad===
+
=Update MITRE about release for CVE number publication=
  
Once the release is done, please close the milestone for the .0 release and create the one for the .1 release if it doesn't already exist so that bugs are targeted correctly.
+
If the release included security updates for which CVE numbers had been issued, [https://cveform.mitre.org/ request an update to the CVE number] so it can be published.
  
===Subpages===
+
=Subpages=
  
 
* [[Developer_Area/Release_Instructions/Creating_a_GPG_key|Creating a GPG key]]
 
* [[Developer_Area/Release_Instructions/Creating_a_GPG_key|Creating a GPG key]]

Revision as of 17:22, 30 October 2017

This document details the procedure to follow when building a release for general download. Other documents related to this one include the Version Numbering policy and the Release Policy.

For the purposes of these instructions, assume we are releasing version X.Y.Z of Mahara.

Also see these pages for more instructions:

Apply for CVE numbers (for security fixes)

At least a few days before the release (best as soon as a bug report exists) apply for one or more CVE numbers for reported security issues. Each issue needs to have its own CVE number.

While CVE numbers are not a requirement, they are useful for administrators who monitor security announcements. It is a good practice thing to do.

Use the request form with MITRE as we don't put Mahara into a distro anymore.

File Siren.gifIMPORTANT !!!File Siren.gif

Please note that this form can ONLY be used if the issue hasn't been made public yet.

  • CVE numbers that we receive are embargoed and do not leak details until we let MITRE know that they can be published.
  • Once you received a CVE number, please add it to the Launchpad bug for that issue so it cn be tracked.

Pre-requisites for doing the release

 apt-get install lptools

OLD INFORMATION - please ignore

  • Configure lptools to write public information on Launchpad on your behalf:
manage-credentials create -c ubuntu-dev-tools -l 2

(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.)

Make sure your mahara-scripts checkout is up to date

You need to have an up-to-date version of the mahara-scripts repo in order to get the latest copy of the release script. That's the only repo you need to have since the release script will clone the git.mahara.org repo all by itself.

Run the first part of the release script

File Siren.gifIMPORTANT !!!File Siren.gif

(ignore for release candidates)Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.

Using the script you got in the previous step, run the first step of it like this:

./release.php X.Y.Z X.Y_STABLE (For rc X.Yrc1)

This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.

Note that the repos that will be pushed back to git.mahara.org are live in your /tmp. Do not reboot your machine in the middle of doing a release or you will have to run through the release script again.

Note: For release candidates the commit message should be empty.

Testing

As we do continuous behat testing throughout development we only need to do the basic manual testing. Here is the testing that needs to be done using the tarballs you generated in the last step:

  1. untar the tar that release.php generated somewhere and make sure you can connect to it via web installer
  2. drop the db and create a new blank one to test install

The install should be run 4 times, both on MySQL and Postgres and both via the web interface and CLI script.

Once we are all happy with that we need to run these manual upgrade tests:

  1. install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres, via web interface, and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
  2. repeat that last step on MySQL

Create the release on Launchpad

Go to https://launchpad.net/mahara/+milestone/X.Y.Z

First move any bugs that are not going to be part of this release to the next milestone in the series, eg if this is X.Y.3 then move to X.Y.4. You may need to make the milestone if it doesn't already exist via https://launchpad.net/mahara/X.Y and using the 'create milestone' link.

Then click on "create release" and give it today's date.

Leave "Keep the X.Y.Z milestone active." unchecked.

Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)

Sign and upload the tarballs

Run the next part of the release script, i.e. the script that was spit out by the release script.

./release-X.Y.Z-cleanup.sh

You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage.

Update bugs on the tracker

On Launchpad, click on all of the bugs targeted for that milestone and move them from "Fix committed" to "Fix released".

Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.

Put the release notes on the wiki page

This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.

Release notes page

Language packs

Create a new translation branch on launchpad.net and update the language scripts for the new release so that they appear on http://langpacks.mahara.org

Security forum posts

If this release includes security fixes, then publish advisories on the security forum.

Here's a sample forum post:

Title: Security issue relating to (Category) <1.0.5, <1.1.3, <1.2.0

<description of issue>

Category: (XSS, Access control, Privacy, Spam, Privilege escalation, Session fixation, Etc.)
Severity: (Critical, High, Medium, Low)
Versions affected: <1.0.5, <1.1.3, <1.2.0
Reported by:
Bug reports: (launchpad URL)
CVE reference: (If you have one; if not, this can be added later) 
Sticky: No
Closed: No

It can be tricky to decide what to put down for "Category". This should be the "category" in the security sense (like a category on the OWASP site) rather than category of Mahara functionality. Some categories we have used in the past:

  • Injection
  • Password security
  • Cross-site scripting (XSS)
  • Cross-site request forgery (CSRF)
  • Access control (i.e., users being able to bypass content sharing permissions)
  • Privacy (i.e., we tell a user that a piece of information is private but then we accidentally display it publicly)
  • Session fixation
  • Privilege escalation
  • Disclosure of system information
  • Stored XSS
  • Session management
  • User Authorization

News forum posts

Post announcement in the news forum at http://mahara.org/

Include links to the security advisories and a link to the download page on Launchpad.

Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.

Here is a sample minor release announcement:

Title: Mahara security release: 15.04.1, 1.10.4, 1.9.6

Hi all,

Today we're announcing new minor releases for the Mahara 15.04, Mahara 1.10, and Mahara 1.9 series.

These releases include <X> security fix<es> relating to <list of issues>, as well as several bug 
fixes. Site administrators are advised to upgrade as soon as possible.

Release packages, as well as a full list of changes, are available on our Launchpad project:

   15.04.1: https://launchpad.net/mahara/+milestone/15.04.1
   1.10.4: https://launchpad.net/mahara/+milestone/1.10.4
   1.9.6: https://launchpad.net/mahara/+milestone/1.9.6
<if this is the last release for a series:>
Note: the 1.9.6 release is the last supported release in the 1.9 series. 
Special thanks, as always, to everyone who helped by reporting bugs, submitting patches, and testing 
the changes!
Sticky: No
Closed: Yes

Quick announcements

You may need to ask a previous release manager to get access to some of these accounts.

Update community sites install (currently Catalyst hosted)

  • demo.mahara.org
  • mahara.org
  • master.dev.mahara.org

They are all deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs on the demo site. As a guideline, language packs that are 90% or more translated should be installed.

Update MITRE about release for CVE number publication

If the release included security updates for which CVE numbers had been issued, request an update to the CVE number so it can be published.

Subpages