Difference between revisions of "Developer Area/Release Instructions"
From Mahara Wiki
< Developer Area
(45 intermediate revisions by 8 users not shown) | |||
Line 3: | Line 3: | ||
For the purposes of these instructions, assume we are releasing version X.Y.Z of Mahara. | For the purposes of these instructions, assume we are releasing version X.Y.Z of Mahara. | ||
− | ''' | + | '''Also see these pages for more instructions:''' |
− | + | * [[Developer Area/Release Instructions/Major Release|Major release]] | |
+ | * [[Developer Area/Release Instructions/Release Candidate|Release candidate]] | ||
− | ====Pre-requisites | + | = Apply for CVE numbers (for security fixes) before publication of the release = |
+ | |||
+ | At least a few days before the release (best as soon as a bug report exists) apply for one or more [https://cve.mitre.org/cve/request_id.html CVE numbers] for reported security issues. Each issue needs to have its own CVE number. | ||
+ | |||
+ | While CVE numbers are not a requirement, they are useful for administrators who monitor security announcements. It is a good practice thing to do. | ||
+ | |||
+ | Use the [https://cveform.mitre.org/ request form with MITRE] as we don't put Mahara into a distro anymore. | ||
+ | |||
+ | [[File:File_Siren.gif]]'''IMPORTANT !!!'''[[File:File_Siren.gif]] | ||
+ | |||
+ | Please note that this form can '''ONLY''' be used if the issue hasn't been made public yet. | ||
+ | |||
+ | * CVE numbers that we receive are embargoed and do not leak details until we let MITRE know that they can be published. | ||
+ | * Once you received a CVE number, please add it to the Launchpad bug for that issue so it can be tracked. It takes at least a couple of days after having received the CVE numbers until Launchpad accepts the CVE number via the CVE number field. | ||
+ | * Add the description, vulnerability, impact (if needed) and the discoverer credit to the Launchpad bug so that the information is readily available for the security announcement. '''Catalyst NZ staff''' also have access to that information on Seafile along with a running record of which bugs have been reported to MITRE (or others if needed). | ||
+ | |||
+ | = 1 week prior to the release = | ||
+ | |||
+ | Let Mahara Partners know in the [https://mahara.org/interaction/forum/view.php?id=1136 Partner forum] that a release is upcoming. Details are not shared though as the reports are still embargoed. | ||
+ | |||
+ | =Pre-requisites for doing the release= | ||
* You must have a [[Developer Area/Release Instructions/Creating a GPG key|valid GPG key]] and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys). | * You must have a [[Developer Area/Release Instructions/Creating a GPG key|valid GPG key]] and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys). | ||
− | * You must have lp-upload-project | + | * You must have lp-upload-project installed: |
+ | |||
+ | apt-get install lptools | ||
+ | |||
+ | OLD INFORMATION - please ignore | ||
+ | * Configure lptools to '''write public information''' on Launchpad on your behalf: | ||
manage-credentials create -c ubuntu-dev-tools -l 2 | manage-credentials create -c ubuntu-dev-tools -l 2 | ||
− | + | (Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.) | |
+ | |||
+ | = Make sure your mahara-scripts checkout is up to date= | ||
− | + | You need to have an up-to-date version of the [https://git.mahara.org/scripts/mahara-scripts mahara-scripts] repo in order to get the latest copy of the [http://git.mahara.org/scripts/mahara-scripts/blobs/master/release.php release script]. That's the only repo you need to have since the release script will clone the git.mahara.org repo all by itself. | |
− | + | = Run the first part of the release script= | |
− | + | [[File:File_Siren.gif]]'''IMPORTANT !!!'''[[File:File_Siren.gif]] | |
− | + | (ignore for release candidates)'''Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.''' | |
Using the script you got in the previous step, run the first step of it like this: | Using the script you got in the previous step, run the first step of it like this: | ||
− | ./release. | + | ./release.php X.Y.Z X.Y_STABLE (For rc X.Yrc1) |
− | + | This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later. | |
− | Note that the repos that will be pushed back to | + | Note that the repos that will be pushed back to git.mahara.org are live in your /tmp. '''Do not reboot your machine''' in the middle of doing a release or you will have to run through the release script again. |
− | + | Note: For release candidates the commit message should be empty. | |
− | = | + | = Testing= |
+ | As we do continuous behat testing throughout development we only need to do the basic manual testing. | ||
Here is the testing that needs to be done using the tarballs you generated in the last step: | Here is the testing that needs to be done using the tarballs you generated in the last step: | ||
− | + | # untar the tar that release.php generated somewhere and make sure you can connect to it via web installer | |
− | # untar the release | + | # drop the db and create a new blank one to test install |
− | + | ||
− | + | The install should be run 4 times, both on MySQL and Postgres and both via the web interface and CLI script. | |
− | |||
− | |||
− | |||
− | |||
− | # drop the db and create a new blank one | ||
− | |||
− | + | Once we are all happy with that we need to run these manual upgrade tests: | |
− | # install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post) | + | # install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres, via web interface, and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post) |
# repeat that last step on MySQL | # repeat that last step on MySQL | ||
− | + | = Create the release on Launchpad= | |
− | + | Go to https://launchpad.net/mahara/+milestone/X.Y.Z | |
− | + | First move any bugs that are not going to be part of this release to the next milestone in the series, eg if this is X.Y.3 then move to X.Y.4. You may need to make the milestone if it doesn't already exist via https://launchpad.net/mahara/X.Y and using the 'create milestone' link. | |
− | + | Then click on "create release" and give it today's date. | |
Leave "Keep the X.Y.Z milestone active." '''unchecked'''. | Leave "Keep the X.Y.Z milestone active." '''unchecked'''. | ||
Line 67: | Line 90: | ||
Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :) | Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :) | ||
− | + | = Sign and upload the tarballs= | |
− | |||
− | |||
Run the next part of the release script, i.e. the script that was spit out by the release script. | Run the next part of the release script, i.e. the script that was spit out by the release script. | ||
Line 75: | Line 96: | ||
./release-X.Y.Z-cleanup.sh | ./release-X.Y.Z-cleanup.sh | ||
− | + | You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage. | |
− | + | = Update bugs on the tracker= | |
− | + | On [https://bugs.launchpad.net/mahara Launchpad], click on all of the bugs targeted for that milestone and move them from "Fix committed" to "Fix released". | |
− | + | Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed. | |
− | + | = Put the release notes on the wiki page= | |
− | + | This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki. | |
− | [[ | + | [[Releases|Release notes page]] |
− | + | = Language packs= | |
− | + | [[Developer_Area/Language_Packs/Launchpad_Branching | Create a new translation branch on launchpad.net and update the language scripts]] for the new release so that they appear on http://langpacks.mahara.org | |
− | + | = Security forum posts= | |
− | |||
− | |||
− | + | If this release includes security fixes, then publish advisories on the [https://mahara.org/interaction/forum/view.php?id=43 security forum]. | |
− | + | Here's a sample forum post: | |
− | + | Title: Security issue relating to (Category) <1.0.5, <1.1.3, <1.2.0 | |
+ | |||
+ | <description of issue> | ||
+ | |||
+ | Category: (XSS, Access control, Privacy, Spam, Privilege escalation, Session fixation, Etc.) | ||
+ | Severity: (Critical, High, Medium, Low) | ||
+ | Versions affected: <1.0.5, <1.1.3, <1.2.0 | ||
+ | Reported by: | ||
+ | Bug reports: (launchpad URL) | ||
+ | CVE reference: (If you have one; if not, this can be added later) | ||
+ | |||
+ | Sticky: No | ||
+ | Closed: No | ||
− | + | It can be tricky to decide what to put down for "Category". This should be the "category" in the security sense (like a category on the OWASP site) rather than category of Mahara functionality. Some categories we have used in the past: | |
− | + | * Injection | |
+ | * Password security | ||
+ | * Cross-site scripting (XSS) | ||
+ | * Cross-site request forgery (CSRF) | ||
+ | * Access control (i.e., users being able to bypass content sharing permissions) | ||
+ | * Privacy (i.e., we tell a user that a piece of information is private but then we accidentally display it publicly) | ||
+ | * Session fixation | ||
+ | * Privilege escalation | ||
+ | * Disclosure of system information | ||
+ | * Stored XSS | ||
+ | * Session management | ||
+ | * User Authorization | ||
− | = | + | = News forum posts = |
− | + | Post announcement in the news forum at [http://www.mahara.org/ http://mahara.org/] | |
Include links to the security advisories and a link to the [https://launchpad.net/mahara/+download download page] on Launchpad. | Include links to the security advisories and a link to the [https://launchpad.net/mahara/+download download page] on Launchpad. | ||
Line 115: | Line 157: | ||
Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum. | Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum. | ||
− | </ | + | Here is a sample minor release announcement: |
+ | |||
+ | Title: Mahara security release: 15.04.1, 1.10.4, 1.9.6 | ||
+ | |||
+ | Hi all, | ||
+ | |||
+ | Today we're announcing new minor releases for the Mahara 15.04, Mahara 1.10, and Mahara 1.9 series. | ||
+ | |||
+ | These releases include <X> security fix<es> relating to <list of issues>, as well as several bug | ||
+ | fixes. Site administrators are advised to upgrade as soon as possible. | ||
+ | |||
+ | Release packages, as well as a full list of changes, are available on our Launchpad project: | ||
+ | |||
+ | 15.04.1: https://launchpad.net/mahara/+milestone/15.04.1 | ||
+ | 1.10.4: https://launchpad.net/mahara/+milestone/1.10.4 | ||
+ | 1.9.6: https://launchpad.net/mahara/+milestone/1.9.6 | ||
+ | |||
+ | <if this is the last release for a series:> | ||
+ | Note: the 1.9.6 release is the last supported release in the 1.9 series. | ||
+ | |||
+ | Special thanks, as always, to everyone who helped by reporting bugs, submitting patches, and testing | ||
+ | the changes! | ||
+ | |||
+ | Sticky: No | ||
+ | Closed: Yes | ||
+ | |||
+ | = Quick announcements= | ||
+ | |||
+ | * Change the topic of #mahara and include a link to forum topic on #mahara-dev | ||
+ | * Put a note on: | ||
+ | ** [https://twitter.com/maharaproject Twitter] | ||
+ | ** [http://freecode.com/projects/mahara freecode] (formerly Freshmeat) | ||
+ | ** [https://www.linkedin.com/groups?home=&gid=2037561 LinkedIn] | ||
+ | ** Facebook [https://www.facebook.com/groups/maharausergroup/ MUG], [https://www.facebook.com/groups/MaharaDE13/ Mahara DE], [https://www.facebook.com/groups/moodlemaharameetup/ Moodle-Mahara Meetup], [https://www.facebook.com/groups/mahara.users/ Mahara in Japanese] | ||
+ | |||
+ | You may need to ask a previous release manager to get access to some of these accounts. | ||
− | = | + | =Update community sites install (currently Catalyst hosted)= |
− | demo.mahara.org | + | * demo.mahara.org, including prep site and change the version number and release date on the homepage and dashboard page (via Admin menu -> Configure site -> Static pages) |
+ | * mahara.org (at minimum to latest minor point release of the supported version it runs on) | ||
+ | * master.dev.mahara.org | ||
− | + | They are all deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs on the demo site. As a guideline, language packs that are 90% or more translated should be installed. | |
− | |||
− | |||
− | |||
− | + | =Update MITRE about release for CVE number publication= | |
− | + | If the release included security updates for which CVE numbers had been issued, [https://cveform.mitre.org/ request an update to the CVE number] so it can be published. | |
− | + | =Subpages= | |
* [[Developer_Area/Release_Instructions/Creating_a_GPG_key|Creating a GPG key]] | * [[Developer_Area/Release_Instructions/Creating_a_GPG_key|Creating a GPG key]] |
Revision as of 14:48, 2 November 2019
This document details the procedure to follow when building a release for general download. Other documents related to this one include the Version Numbering policy and the Release Policy.
For the purposes of these instructions, assume we are releasing version X.Y.Z of Mahara.
Also see these pages for more instructions:
Apply for CVE numbers (for security fixes) before publication of the release
At least a few days before the release (best as soon as a bug report exists) apply for one or more CVE numbers for reported security issues. Each issue needs to have its own CVE number.
While CVE numbers are not a requirement, they are useful for administrators who monitor security announcements. It is a good practice thing to do.
Use the request form with MITRE as we don't put Mahara into a distro anymore.
Please note that this form can ONLY be used if the issue hasn't been made public yet.
- CVE numbers that we receive are embargoed and do not leak details until we let MITRE know that they can be published.
- Once you received a CVE number, please add it to the Launchpad bug for that issue so it can be tracked. It takes at least a couple of days after having received the CVE numbers until Launchpad accepts the CVE number via the CVE number field.
- Add the description, vulnerability, impact (if needed) and the discoverer credit to the Launchpad bug so that the information is readily available for the security announcement. Catalyst NZ staff also have access to that information on Seafile along with a running record of which bugs have been reported to MITRE (or others if needed).
1 week prior to the release
Let Mahara Partners know in the Partner forum that a release is upcoming. Details are not shared though as the reports are still embargoed.
Pre-requisites for doing the release
- You must have a valid GPG key and you need to add it to Launchpad (see https://launchpad.net/~username/+editpgpkeys).
- You must have lp-upload-project installed:
apt-get install lptools
OLD INFORMATION - please ignore
- Configure lptools to write public information on Launchpad on your behalf:
manage-credentials create -c ubuntu-dev-tools -l 2
(Note: This won't work for Ubuntus after Maverick, as manage-credentials appears to have gone away. You will need to install lptools to proceed with the script usage.)
Make sure your mahara-scripts checkout is up to date
You need to have an up-to-date version of the mahara-scripts repo in order to get the latest copy of the release script. That's the only repo you need to have since the release script will clone the git.mahara.org repo all by itself.
Run the first part of the release script
(ignore for release candidates)Make sure that any security reviews/patches are merged before proceeding. THESE PATCHES ARE MARKED STATUS: Draft. If you do not have the ability to see the security patches - ask someone on the security team to merge them for you.
Using the script you got in the previous step, run the first step of it like this:
./release.php X.Y.Z X.Y_STABLE (For rc X.Yrc1)
This will create tarballs as well as a changelog and some release notes to paste into Launchpad. It will also spit out another script (release-X.Y.Z-cleanup.sh) to be run later.
Note that the repos that will be pushed back to git.mahara.org are live in your /tmp. Do not reboot your machine in the middle of doing a release or you will have to run through the release script again.
Note: For release candidates the commit message should be empty.
Testing
As we do continuous behat testing throughout development we only need to do the basic manual testing. Here is the testing that needs to be done using the tarballs you generated in the last step:
- untar the tar that release.php generated somewhere and make sure you can connect to it via web installer
- drop the db and create a new blank one to test install
The install should be run 4 times, both on MySQL and Postgres and both via the web interface and CLI script.
Once we are all happy with that we need to run these manual upgrade tests:
- install a fresh X.Y.Z-1 site, then upgrade to Z.Y.Z on Postgres, via web interface, and quickly check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post)
- repeat that last step on MySQL
Create the release on Launchpad
Go to https://launchpad.net/mahara/+milestone/X.Y.Z
First move any bugs that are not going to be part of this release to the next milestone in the series, eg if this is X.Y.3 then move to X.Y.4. You may need to make the milestone if it doesn't already exist via https://launchpad.net/mahara/X.Y and using the 'create milestone' link.
Then click on "create release" and give it today's date.
Leave "Keep the X.Y.Z milestone active." unchecked.
Paste in the release notes and the changelog that were generated for you by the release script. You can remove the "bump version number" commits from the changelog since they aren't very useful :)
Sign and upload the tarballs
Run the next part of the release script, i.e. the script that was spit out by the release script.
./release-X.Y.Z-cleanup.sh
You will be prompted for enter release notes and a changelog. This will happen once for each archive file that is being uploaded, so you'll see six prompts total. Any data you enter here will go into the Launchpad release page. So, you can either enter it here, or leave it empty here and enter it directly into the Launchpad webpage.
Update bugs on the tracker
On Launchpad, click on all of the bugs targeted for that milestone and move them from "Fix committed" to "Fix released".
Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.
Put the release notes on the wiki page
This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.
Language packs
Create a new translation branch on launchpad.net and update the language scripts for the new release so that they appear on http://langpacks.mahara.org
Security forum posts
If this release includes security fixes, then publish advisories on the security forum.
Here's a sample forum post:
Title: Security issue relating to (Category) <1.0.5, <1.1.3, <1.2.0 <description of issue> Category: (XSS, Access control, Privacy, Spam, Privilege escalation, Session fixation, Etc.) Severity: (Critical, High, Medium, Low) Versions affected: <1.0.5, <1.1.3, <1.2.0 Reported by: Bug reports: (launchpad URL) CVE reference: (If you have one; if not, this can be added later)
Sticky: No Closed: No
It can be tricky to decide what to put down for "Category". This should be the "category" in the security sense (like a category on the OWASP site) rather than category of Mahara functionality. Some categories we have used in the past:
- Injection
- Password security
- Cross-site scripting (XSS)
- Cross-site request forgery (CSRF)
- Access control (i.e., users being able to bypass content sharing permissions)
- Privacy (i.e., we tell a user that a piece of information is private but then we accidentally display it publicly)
- Session fixation
- Privilege escalation
- Disclosure of system information
- Stored XSS
- Session management
- User Authorization
News forum posts
Post announcement in the news forum at http://mahara.org/
Include links to the security advisories and a link to the download page on Launchpad.
Now is a good time to notify translators about changed strings, if this is a stable release. I tend to do this with a forum post in the Translations forum.
Here is a sample minor release announcement:
Title: Mahara security release: 15.04.1, 1.10.4, 1.9.6 Hi all, Today we're announcing new minor releases for the Mahara 15.04, Mahara 1.10, and Mahara 1.9 series. These releases include <X> security fix<es> relating to <list of issues>, as well as several bug fixes. Site administrators are advised to upgrade as soon as possible. Release packages, as well as a full list of changes, are available on our Launchpad project: 15.04.1: https://launchpad.net/mahara/+milestone/15.04.1 1.10.4: https://launchpad.net/mahara/+milestone/1.10.4 1.9.6: https://launchpad.net/mahara/+milestone/1.9.6
<if this is the last release for a series:> Note: the 1.9.6 release is the last supported release in the 1.9 series.
Special thanks, as always, to everyone who helped by reporting bugs, submitting patches, and testing the changes!
Sticky: No Closed: Yes
Quick announcements
- Change the topic of #mahara and include a link to forum topic on #mahara-dev
- Put a note on:
- freecode (formerly Freshmeat)
- Facebook MUG, Mahara DE, Moodle-Mahara Meetup, Mahara in Japanese
You may need to ask a previous release manager to get access to some of these accounts.
Update community sites install (currently Catalyst hosted)
- demo.mahara.org, including prep site and change the version number and release date on the homepage and dashboard page (via Admin menu -> Configure site -> Static pages)
- mahara.org (at minimum to latest minor point release of the supported version it runs on)
- master.dev.mahara.org
They are all deployed using Catalyst's standard internal hosting deployment scripts. Also update the installed language packs on the demo site. As a guideline, language packs that are 90% or more translated should be installed.
Update MITRE about release for CVE number publication
If the release included security updates for which CVE numbers had been issued, request an update to the CVE number so it can be published.