Actions

Developer Area/Release Instructions/Release day: Difference between revisions

From Mahara Wiki

< Developer Area‎ | Release Instructions
No edit summary
Line 26: Line 26:


==📓 Prepare the changelog and release notes==
==📓 Prepare the changelog and release notes==
Use the templates below to structure your changelog and release notes.
Use the templates below to structure your changelog and release notes. To see an example, look at the [https://launchpad.net/mahara/+milestone/21.04.0 21.04.0].


====Changelog template:====
====Changelog template:====
Line 36: Line 36:
  Bug XXXXXXX: <Title of bug>
  Bug XXXXXXX: <Title of bug>


  ...<br />
  ...
====Release notes template:====
====Release notes template:====
Note: Replace the <code><XX.XX.X></code> with the milestone/version you are working on.
Note: Replace the <code><XX.XX.X></code> with the milestone/version you are working on. Include any information about backwards compatibility breaking.                                           
  Mahara <code><XX.XX.X></code> Release Notes
  Mahara <code><XX.XX.X></code> Release Notes


Line 52: Line 52:


  the upgrade on it first, to minimise the effect of any potential unforeseen problems.
  the upgrade on it first, to minimise the effect of any potential unforeseen problems.
                                                           
==⚡ The release script==
==⚡ The release script==


Line 60: Line 59:


===Run the script===
===Run the script===
Go into <code>mahara-scripts</code> and run <code>./release.php X.Y.Z X.Y_DEV</code> , i.e. <code>[version number][branch] For rc X.Yrc1</code>


#Go into <code>mahara-scripts</code>
#Run <code>./release.php X.Y.Z X.Y_DEV</code> , i.e. <code>[version number][branch] For rc X.Yrc1</code>
#When prompted, insert your changelog.
#When prompted, insert your changelog.
#(TEMPORARY: <code>sshphp</code>) <code>less /tmp/mahara.####/make.log</code> to watch for prompts.
#Respond to the prompts in your terminal with <code>y</code> .


Any data you enter for the changelog or release notes prompts will go into the Launchpad release page. You can enter it here, or leave it empty here and enter it directly into the Launchpad webpage.
Any data you enter for the changelog or release notes prompts will go into the Launchpad release page. You can enter it here, or leave it empty here and enter it directly into the Launchpad webpage.


===Check the terminal output in case there are GPG errors===
==== Notes about the script: ====
MAJOR RELEASE - A new <code>_STABLE</code> branch is created based off the <code>_DEV</code> branch as part of the <code>./release.php</code> script.
 
*You can check that it happens in the local <code>/tmp/</code> clone of mahara.
*You can check that the release cleanup script pushes things to the right branch.
 
====Check the terminal output in case there are GPG errors====
E.g. <code>Tag new version bump commit as 'X.YRC2_RELEASE' error: gpg failed to sign the data error: unable to sign the tag</code>
E.g. <code>Tag new version bump commit as 'X.YRC2_RELEASE' error: gpg failed to sign the data error: unable to sign the tag</code>


Line 86: Line 92:


==✅ Testing==
==✅ Testing==
With continuous Behat testing throughout development, we only need to some basic manual testing.
With continuous Behat testing throughout development, we only need to some basic manual testing.  
 
The following needs to be tested with '''MySQL''' as well as '''PostgreSQL'''.


===Extract the build and prep the database===
===Extract the build and prep the database===
Test that we can install Mahara with the pre-built packages.


#Extract the build (<code>.tar</code> or <code>.zip</code>) of your pre-built Mahara and make sure you can connect to it via web installer.
#Extract the build (<code>.tar</code> or <code>.zip</code>) of your pre-built Mahara and make sure you can connect to it via web installer.
Line 95: Line 102:


===Test case: Install the site with CLI and the web interface===
===Test case: Install the site with CLI and the web interface===
CLI
Test that we can install Mahara with the pre-built packages:


#Install Mahara using the CLI script <code>htdocs/admin/cli/install.php</code>.
#Install Mahara using the CLI script <code>htdocs/admin/cli/install.php</code>.
#Reset your database and repeat with MySQL/PostgreSQL.
#Reset your database.
 
Web interface
 
#Go to your Mahara site and click 'Install'.
#Go to your Mahara site and click 'Install'.
#Reset your database and repeat with MySQL/PostgreSQL.
#Reset your database.
#Repeat with MySQL/PostgreSQL.


'''Note:''' If we are doing a <code>.0</code> release the install will complain about missing <code>$versions->$currentmajorversion</code>->latest release number but don't panic this is because the local copy of the release has the series version that doesn't exist upstream yet so we can ignore this warning·
'''Note:''' If we are doing a <code>.0</code> release the install will complain about missing <code>$versions->$currentmajorversion</code>->latest release number but don't panic this is because the local copy of the release has the series version that doesn't exist upstream yet so we can ignore this warning·


===Test case: Manual upgrade with CLI and the web interface===
===Test case: Manual upgrade with CLI and the web interface===
CLI
Once we are all happy with that we need to run these manual upgrade tests


#Install a fresh site on version <code>X.Y.Z(-1)</code>, i.e. (an earlier version).
#Install a fresh <code>X.Y.Z-1</code> site (one version before your working one).
#Switch the code to your current release and upgrade to <code>X.Y.Z</code> with the CLI script <code>htdocs/admin/cli/upgrade.php</code>.
#Switch the code to your current release and upgrade to <code>X.Y.Z</code> with the CLI script <code>htdocs/admin/cli/upgrade.php</code>.
#Check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post).
#Check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post).
#Repeat with PostgreSQL/MySQL.
#Reset your database.
 
#Install a fresh <code>X.Y.Z-1</code>
Web interface
#Switch the code to your current release and upgrade to <code>X.Y.Z</code> by going to the site and clicking 'Upgrade'.
 
#Install a fresh site on version <code>X.Y.Z(-1)</code>, i.e. (an earlier version).
#Switch the code to your current release.
#Go to the site and click 'Upgrade' to get the site version to <code>X.Y.Z</code> .
#Repeat with PostgreSQL/MySQL.
#Repeat with PostgreSQL/MySQL.


Line 162: Line 163:


Note: The number of language packs we support can be more than the supported Mahara versions as some people want to update old lang packs.
Note: The number of language packs we support can be more than the supported Mahara versions as some people want to update old lang packs.
== 📊 GIT stats ==
Prep the git contributor stats  to go in the release announcement on the Mahara News forum.
# Clone our version of <code>gitdm</code> from the [https://git.mahara.org/scripts/mahara-scripts mahara-scripts repo]: <code>git clone [email protected]:scripts/mahara-scripts.git</code>
# Get the name of the current release branch, e.g. <code>21.10_DEV</code>
# Get the previous branch's first release tag, e.g. <code>21.04.0_RELEASE</code>
# Generate the stats:
## <code>cd</code> into your local mahara repo
## <code>git log -p -M --no-merges 21.04.0_RELEASE..21.10_DEV > ~/mahara.log</code>
## <code>cd ~/code/mahara-scripts/gitdm/</code>
## <code>cat ~/mahara.log | ./gitdm -c mahara.config -u -s -z -o results -h results.html</code>
# Check that the <code>results</code> text file only list developers once (otherwise add them to the <code>mahara.aliases</code> file).
#Make sure that there is no "(unknown)" company by checking all the necessary mappings are in <code>mahara.domain-map</code>.
#If there are "(unknown)" company results, look through the generated text file <code>database.dump</code> to locate the unmapped individuals/emails.


==📣 Announcements==
==📣 Announcements==
Line 168: Line 184:
You will need to get access to some of these accounts.
You will need to get access to some of these accounts.


Change the topic of #mahara and include a link to forum topic on #mahara-dev
Change the topic of <code>#mahara</code> and include a link to forum topic on <code>#mahara-dev</code>


*[https://twitter.com/maharaproject Twitter]
*[https://twitter.com/maharaproject Twitter]
Line 180: Line 196:
**[https://www.facebook.com/groups/maharausergroup/ MUG]
**[https://www.facebook.com/groups/maharausergroup/ MUG]


=Update MITRE about release for CVE number publication=
==== [Major release] ====
If the release included security updates for which CVE numbers had been issued, [https://cveform.mitre.org/ request an update to the CVE number] so it can be published.
 
* Release notes posted in the News forum (copy the format from the release notes for the previous release).
* Identi.ca post which will also immediately post to Twitter.
* Any other places where you want to announce the release.
 
== 🔒 Update MITRE about release for CVE number publication ==
If the release has security updates for with issued CVE numbers, request an update to the CVE number so it can be published.
 
== 📝 Major release tasks ==
 
=== Bump the <code>_STABLE VERSION</code> on [https://mahara.org/ mahara.org] ===
This should now be done automatically via the <code>htdocs/admin/cli/create_version.php</code> script which is controlled by cron and will update once a day.
 
If during major upgrade you can't wait for the cron you can log into the server and run the CLI script manually, e.g.
<code>sudo -u www-data php create_version.php</code>
If that is failing you can always do it the old way - see below
 
'''OLD INFO - should not need to do this any more '''
 
This is what sets the right download link in the header. And also sets the 'Latest version' info on the Mahara Administration dashboard for "Site information"
 
To do this you need to do 2 things:
 
1. Edit the <code>htdocs/local/upgrade.php</code> file and add a upgrade block, eg:
  if ($oldversion < 2018102500) {
    set_config('mahara_stable_version', '18.10');
  }
2. Edit the <code>htdocs/local/version.php</code> file and change the version and release variables, eg:
  $config->version = 2018102500;
  $config->release = '18.10.0';
Note: Only a deployment to production will show a change.
===Launchpad series===
On completing a .0 release, update the status of our series in Launchpad for both [https://launchpad.net/mahara Mahara] and [https://launchpad.net/mahara-lang Mahara-Lang]:
 
*Change the status of the future / trunk series to '''"Active development"''' and the description to '''"Release of Mahara X.Y, scheduled for April/October YYYY"'''
*Changed the status of the series just released to '''"Current stable release"'''
*Changed the status of the series of all the supported series to '''"Supported"'''
*Changed the status of the series that has just fallen out of support to '''"Obsolete"'''


<br />
<br />

Revision as of 19:02, 26 April 2022

Not ready? > Go to 'Pre-release investigation and tasks'

📋 Final checks before starting release day

🚨 Make sure that any security reviews/patches are merged before proceeding 🚨

🟡 Security team (if you don't have +2 review status, you won't be able to see these patches)

  • Merge the 'Private' security patches.
  • Update the related Launchpad bugs to 'Public security'.

🟡 Release manager

  • CVE #'s - Security issues have an assigned CVE number?
  • Backporting - security fixes backported to all supported versions that need them?
  • Bugs fixed - all bugs going into his release have their fixes merged for all supported versions?
  • Comms - to Mahara partners + drafted the forum posts?

🟡 All devs

  • What minor point release are you going to do?
  • Have you created a release on the milestone on Launchpad to receive the release generated tarballs?
  • git pull mahara-scripts
  • Valid GPG - do you have a valid GPG key added to your Launchpad account? See https://launchpad.net/~username/+editpgpkeys
  • lptools - do you have the lp-upload-project installed for uploading the release tars? apt-get install lptools

📓 Prepare the changelog and release notes

Use the templates below to structure your changelog and release notes. To see an example, look at the 21.04.0.

Changelog template:

Note: You can copy the list of bugs from the milestone page and re-format the list. Remove the 'bump version number' commits text.

Bug XXXXXXX: <Title of bug>
Bug XXXXXXX: <Title of bug>
Bug XXXXXXX: <Title of bug>
...

Release notes template:

Note: Replace the <XX.XX.X> with the milestone/version you are working on. Include any information about backwards compatibility breaking.

Mahara <XX.XX.X> Release Notes
This is a stable release of Mahara <XX.XX>. Stable release are fit for general use. 
If you find a bug, please report it to the tracker:
https://bugs.launchpad.net/mahara/+filebug
This release includes an upgrade path from 1.1.0. 
If you wish to upgrade, we encourage you to make a copy of your website and test 
the upgrade on it first, to minimise the effect of any potential unforeseen problems.

⚡ The release script

  • A git repo will be created at /tmp/mahara.####/mahara/ cloned from https://git.mahara.org/mahara/mahara .
  • Pre-built Mahara archives are generated and put into your current folder.
  • You will get a release-X.Y.Z-cleanup.sh script to run later.

Run the script

Go into mahara-scripts and run ./release.php X.Y.Z X.Y_DEV , i.e. [version number][branch] For rc X.Yrc1

  1. When prompted, insert your changelog.
  2. (TEMPORARY: sshphp) less /tmp/mahara.####/make.log to watch for prompts.
  3. Respond to the prompts in your terminal with y .

Any data you enter for the changelog or release notes prompts will go into the Launchpad release page. You can enter it here, or leave it empty here and enter it directly into the Launchpad webpage.

Notes about the script:

MAJOR RELEASE - A new _STABLE branch is created based off the _DEV branch as part of the ./release.php script.

  • You can check that it happens in the local /tmp/ clone of mahara.
  • You can check that the release cleanup script pushes things to the right branch.

Check the terminal output in case there are GPG errors

E.g. Tag new version bump commit as 'X.YRC2_RELEASE' error: gpg failed to sign the data error: unable to sign the tag

So this is what I did to fix it, all commands are on command line:

  1. Update the gpg program with: git config --global gpg.program gpg2
  2. Test if it is working with: echo "test" | gpg2 --clearsign

It wasn't so I had to do: export GPG_TTY=$(tty)

then run again echo "test" | gpg2 --clearsign

After all that when I ran release.php command again it didn't give me the error.

Do not reboot your machine in the middle of doing a release or you will have to run through the release script again.

Note: For release candidates the commit message should be empty.

✅ Testing

With continuous Behat testing throughout development, we only need to some basic manual testing.

The following needs to be tested with MySQL as well as PostgreSQL.

Extract the build and prep the database

  1. Extract the build (.tar or .zip) of your pre-built Mahara and make sure you can connect to it via web installer.
  2. Drop the database and create a new blank one.

Test case: Install the site with CLI and the web interface

Test that we can install Mahara with the pre-built packages:

  1. Install Mahara using the CLI script htdocs/admin/cli/install.php.
  2. Reset your database.
  3. Go to your Mahara site and click 'Install'.
  4. Reset your database.
  5. Repeat with MySQL/PostgreSQL.

Note: If we are doing a .0 release the install will complain about missing $versions->$currentmajorversion->latest release number but don't panic this is because the local copy of the release has the series version that doesn't exist upstream yet so we can ignore this warning·

Test case: Manual upgrade with CLI and the web interface

Once we are all happy with that we need to run these manual upgrade tests

  1. Install a fresh X.Y.Z-1 site (one version before your working one).
  2. Switch the code to your current release and upgrade to X.Y.Z with the CLI script htdocs/admin/cli/upgrade.php.
  3. Check that everything works (create a portfolio, a blog with a blog post, a group with a forum and a forum post).
  4. Reset your database.
  5. Install a fresh X.Y.Z-1
  6. Switch the code to your current release and upgrade to X.Y.Z by going to the site and clicking 'Upgrade'.
  7. Repeat with PostgreSQL/MySQL.

🚀 Sign and upload the tarballs

Run upload the release and clean up any temporary files with ./release-X.Y.Z-cleanup.sh

Note I: If running this file stops before completion for any reason you will need to edit it and comment out the bits that did finish so that you don't run those buts twice then try the script again. For example the script does the bit where it pushes the tag to the git remote and signs the zipped files but doesn't upload them - you would need to comment out the 'git push' lines and the 'gpg --armor' lines.

Note II: If the zipped files fail to upload to Launchpad via the script then you will need to manually upload the zipped files and their corresponding .asc files. To do this go to the milestone page in Launchpad and click the 'Add download file' link. Set the description to: release tarball and choose the matching zipped and .asc file to upload

You will need to do this for all the three types: .zip, .tar.bz2, .tar.gz

🐛 Update bugs on the tracker

Go to Launchpad and click on all of the bugs targeted for that milestone and move them from 'Fix committed' to 'Fix released'.

OR

In Firefox, go to the milestone page and open all the bugs in new tabs at once by pasting the following into the browser console.

var result = document.evaluate("//tbody//tr[contains(., 'Committed')]//a[contains(@href, 'bugs')]", document, null, XPathResult.ANY_TYPE,null ); var mycount=0; while(node = result.iterateNext()) { window.open(node.href, '_blank'); mycount++; if (mycount == 50) { break; } }

You will ned to allow popups for the site for this to work and it will open up to 50 bugs where the status is 'Committed'. If you have more than 50 to do - mark the first ones 'released' - refresh the list and do again.

Now is also a good time to create the next release number in the series. Any remaining bugs that were not fixed for this release may be transferred to the next one, if they're still on the roadmap but were simply delayed.

📔 Put the release notes on the wiki page

This is just adding a link to the right page on Launchpad, we no longer have a duplicate copy of the release notes on the wiki.

Release notes page

📗 Update wiki.mahara.org releases page

🟡 Release manager

  1. Go to the releases page.
  2. Edit 'Current and upcoming releases.
  3. Update the text: "The latest stable version of Mahara is MAJOR_VERSION , released on DATE"
  4. Add a new entry to the table for the new release.
  5. Update the major version where 18 months has passed since its release to be 'unsupported'.

🌏 Update the language packs

🟡 Release manager

Create a new translation branch on launchpad.net and update the language scripts for the new release so that they appear on http://langpacks.mahara.org.

Note: The number of language packs we support can be more than the supported Mahara versions as some people want to update old lang packs.

📊 GIT stats

Prep the git contributor stats to go in the release announcement on the Mahara News forum.

  1. Clone our version of gitdm from the mahara-scripts repo: git clone [email protected]:scripts/mahara-scripts.git
  2. Get the name of the current release branch, e.g. 21.10_DEV
  3. Get the previous branch's first release tag, e.g. 21.04.0_RELEASE
  4. Generate the stats:
    1. cd into your local mahara repo
    2. git log -p -M --no-merges 21.04.0_RELEASE..21.10_DEV > ~/mahara.log
    3. cd ~/code/mahara-scripts/gitdm/
    4. cat ~/mahara.log | ./gitdm -c mahara.config -u -s -z -o results -h results.html
  5. Check that the results text file only list developers once (otherwise add them to the mahara.aliases file).
  6. Make sure that there is no "(unknown)" company by checking all the necessary mappings are in mahara.domain-map.
  7. If there are "(unknown)" company results, look through the generated text file database.dump to locate the unmapped individuals/emails.

📣 Announcements

🟡 Kristina | Release manager

You will need to get access to some of these accounts.

Change the topic of #mahara and include a link to forum topic on #mahara-dev

[Major release]

  • Release notes posted in the News forum (copy the format from the release notes for the previous release).
  • Identi.ca post which will also immediately post to Twitter.
  • Any other places where you want to announce the release.

🔒 Update MITRE about release for CVE number publication

If the release has security updates for with issued CVE numbers, request an update to the CVE number so it can be published.

📝 Major release tasks

Bump the _STABLE VERSION on mahara.org

This should now be done automatically via the htdocs/admin/cli/create_version.php script which is controlled by cron and will update once a day.

If during major upgrade you can't wait for the cron you can log into the server and run the CLI script manually, e.g.

sudo -u www-data php create_version.php

If that is failing you can always do it the old way - see below

OLD INFO - should not need to do this any more

This is what sets the right download link in the header. And also sets the 'Latest version' info on the Mahara Administration dashboard for "Site information"

To do this you need to do 2 things:

1. Edit the htdocs/local/upgrade.php file and add a upgrade block, eg:

 if ($oldversion < 2018102500) {
    set_config('mahara_stable_version', '18.10');
 }

2. Edit the htdocs/local/version.php file and change the version and release variables, eg:

 $config->version = 2018102500;
 $config->release = '18.10.0';

Note: Only a deployment to production will show a change.

Launchpad series

On completing a .0 release, update the status of our series in Launchpad for both Mahara and Mahara-Lang:

  • Change the status of the future / trunk series to "Active development" and the description to "Release of Mahara X.Y, scheduled for April/October YYYY"
  • Changed the status of the series just released to "Current stable release"
  • Changed the status of the series of all the supported series to "Supported"
  • Changed the status of the series that has just fallen out of support to "Obsolete"