Actions

Difference between revisions of "Developer Area/Developer Meetings/88"

From Mahara Wiki

< Developer Area‎ | Developer Meetings
Line 30: Line 30:
 
'''Attendees:''' Andreas, Dianne, Doris, Kristina, and Robert.
 
'''Attendees:''' Andreas, Dianne, Doris, Kristina, and Robert.
  
===A new Gerrit to be available for the community===
+
===Items from previous meeting===
This task is ongoing and needs a little bit more time due to the availability of people. The team is working hard on testing the site. We are excited at the modern look of the new site.
+
====A new Gerrit to be available for the community====
 +
This task is ongoing and needs a little bit more time due to the availability of people. The team is working hard on testing the site. We are excited about the modern look of the new site.
  
===How to stop people pushing to <code>_STABLE</code> branches===
+
'''Action item''': Catalyst team to make the new Gerrit available to the public.
  
======This task is ongoing. Robert mentioned that because people can get around the <code>make push</code> command, it's not a simple fix. We rely on the <code>_STABLE</code> branches for our releases, therefore we cannot lock those branches. This item will be looked into some more.======
+
====How to stop people pushing to <code>_STABLE</code> branches====
  
===Reduce security vulnerabilities by getting Composer to install only used plugins===
+
This task is ongoing. Robert mentioned that because people can get around the <code>make push</code> command, it's not an urgent fix that would need to be made. It's also not a simple fix. We rely on the <code>_STABLE</code> branches for our releases. Therefore, we cannot lock those branches. This item will be looked into some more.
Context: Not all plugins are installed on Mahara sites. We want to reduce the amount of plugin code on sites that don't need them so that when a plugin is found to have a vulnerability, sites who don't have it installed are not affected. The idea is to get our package manager, Composer, to ignore libraries that aren't frequently used, e.g. SimpleSAML.
 
  
Gold mentions that we are unable to use Composer to do this for us and that another option is to write our own script to clean up parts  we don't want in the final build package. This item is closed.
+
====Reduce security vulnerabilities by getting Composer to install only used plugins====
 +
Context: Not all plugins are installed on Mahara sites. We want to reduce the amount of plugin code on sites that don't need them so that when a plugin has a security vulnerability, sites who don't have it installed are not affected. The idea is to get our package manager, Composer, to ignore libraries that aren't frequently used, e.g. SimpleSAML.
  
===Fergus's Docker patch working with PHP8===
+
Gold mentions that we are unable to use Composer to do this for us and that another option is to write our own script to clean up parts we don't want in the final build package. This item is closed.
Gold and Robert has tested Fergus' patch and it works! Gold is working on getting Behat to work with the Docker patch so the tests can find our PHP 8 errors for us. The rest of the team has yet to test this Docker patch and get it working in their environment.
 
  
===Kristina's PHP compatibility announcement===
+
====Fergus's Docker patch working with PHP8====
She has made a [https://mahara.org/interaction/forum/topic.php?id=8989 forum post] . We are not close to having Mahara 22.04 to be PHP 8 compatible. Our aim is to have 22.10 to be available for transitioning from PHP 7 to 8. Next year we will only allow PHP 8 to be used with Mahara. The goal is to have PHP 8.1 compatibility as it will be there for the upcoming Ubuntu LTS.
+
Gold and Robert have tested Fergus' patch and it works! Kristina struggled getting it to work and may need to reinstall everything. Gold is working on getting Behat to work with the Docker patch so the tests can find our PHP 8 errors for us. The rest of the team has yet to test this Docker patch and get it working in their environment.
 +
 
 +
====Kristina's PHP compatibility announcement====
 +
She has made a [https://mahara.org/interaction/forum/topic.php?id=8989 forum post]. We are not close to having Mahara 22.04 to be PHP 8 compatible. Our aim is to have 22.10 to work with both PHP 7 to 8, with minimal PHP 8 support. After that, we will be working on full PHP 8 support, which will mean that PHP 7 will no longer be supported. The goal is to have PHP 8.1 compatibility as it will be there for the upcoming Ubuntu LTS. Whether that is Mahara 22.04 or 22.10 has not yet been decided as we are working on the transition at the moment.
  
 
===Doris' application to become an approved developer===
 
===Doris' application to become an approved developer===
Congratulations to Doris.
+
Congratulations to Doris. Her [[User:Doristam| application]] convinced all five approved developers who were present or had sent in their votes prior to the meeting. Eight approved developers were eligible to vote with one not being active any more and now removed and two others not available to vote.
 +
 
 +
'''Action item''': Robert to add Doris into all the Approved Developer groups on Gerrit and Launchpad.
  
 
===Improving of <code>get_user</code> for group submissions management===
 
===Improving of <code>get_user</code> for group submissions management===
Context: Recently, a change was made to use the <code>get_users</code> function to get account names. The performance of this function becomes very slow and affects user experience when the number of submissions increase, e.g. a with three evaluators, a site may have 2400 submissions in a course which make calls to the database 7200 times. The idea is that a direct DB query can provide better performance than the function.
+
Context: Recently, a change was made to use the <code>get_users</code> function to get account names in the group submission management. The performance of this function becomes very slow and affects UX when the number of submissions increase, e.g. with three evaluators, a site may have 2,400 submissions in a course which would make calls to the database 7,200 times slowing things down. The idea is that a direct DB query can provide better performance than the function. Would it be permissible to make these direct database calls instead of using the function?
  
 
<code>get_user</code> is used to get extra stuff about a person, e.g. account settings and activity preferences. Improving this will need to consider what is in on the submission output.  
 
<code>get_user</code> is used to get extra stuff about a person, e.g. account settings and activity preferences. Improving this will need to consider what is in on the submission output.  
Line 60: Line 65:
 
*If only the name needs to be displayed, one idea is to modify the function to get a 'bare bones' version by giving a flag.
 
*If only the name needs to be displayed, one idea is to modify the function to get a 'bare bones' version by giving a flag.
 
*The DB formatting might be a bottleneck that is slowing things down.
 
*The DB formatting might be a bottleneck that is slowing things down.
 +
 +
'''Action item''': Robert to look into improving the <code>get_user</code> function for submissions management, e.g. a bulk version and a bare-bones version.
  
 
===Gold's work on Elasticsearch 7===
 
===Gold's work on Elasticsearch 7===
Making progress and going through testing scenarios at the moment. For this to be ready for the 22.04 release as it is planned, a team effort is required to thoroughly test elasticsearch. Gold suggests a team of 2-3 devs working on for a couple days each week. We can use the <code>@manual</code> tag in Behat tests to help us with this when testing locally. There is is a site set up for this and ready for testing.
+
Making progress and going through testing scenarios at the moment. For this to be ready for the 22.04 release as it is planned, a team effort is required to thoroughly test elasticsearch. Gold suggests a team of 2-3 devs working on for a couple days each week. We can use the <code>@manual</code> tag in Behat tests to help us with this when testing locally. There is is a site set up for this and ready for internal Catalyst testing.
  
Gold recommends that we have Behat tests describing all the potential scenarios for known usage. e.g. a background standing up, the number of groups, institutions, and to index all the contents as well as runs tests to make sure that when a particular search is done, it checks what it should and shouldn’t see.  
+
Gold recommends that we have Behat tests describing the potential scenarios for known usage, e.g. a background standing up, the number of groups, institutions, and to index the content as well as run tests to make sure that when a particular search is done, it checks what it should and shouldn’t see.  
  
Elasticsearch has been hardcoded into the Mahara codebase and those all need to be eventually found and checked for ES7. The general search is there, works, and is being debugged.
+
Elasticsearch has been hardcoded into the Mahara codebase and those references need to be eventually found and checked for Elasticsearch 7. The general search is there, works, and is being debugged.
  
 
===Mahara 22.04===
 
===Mahara 22.04===
Line 72: Line 79:
  
 
In the first half of April, a preview version will be available for people to install and test. The release manager will update the details and documentation.
 
In the first half of April, a preview version will be available for people to install and test. The release manager will update the details and documentation.
 +
 +
'''Action item''': Kristina to discuss with the Catalyst team to organise devs to get Elasticsearch tested.
  
 
===Key decisions===
 
===Key decisions===
Line 81: Line 90:
 
*<code>get_user</code> is a bottle neck and needs improvements/modifications.
 
*<code>get_user</code> is a bottle neck and needs improvements/modifications.
  
===Tasks===
+
===Action items===
  
 
*Robert - Look into improving the <code>get_user</code> function for submissions management, e.g. a bulk version and a bare-bones version.
 
*Robert - Look into improving the <code>get_user</code> function for submissions management, e.g. a bulk version and a bare-bones version.
*Robert - Add Doris into all the Approved Developer groups.
+
*Robert - Add Doris into all the Approved Developer groups on Gerrit and Launchpad.
*Gold - Discuss with the team to organise devs to get elasticsearch tested.
+
*Kristina - Discuss with the team to organise devs to get Elasticsearch tested.
*To be continued: to have the new Gerrit available to the public.
+
*Catalyst team - Make the new Gerrit available to the public.
  
 
===The next developer meeting plan===
 
===The next developer meeting plan===
 
'''Date''': [https://www.timeanddate.com/worldclock/fixedtime.html?msg=89th+Mahara+developer+meeting&iso=20220413T11&p1=264 Tuesday, 12 April 2022 at 23:00 UTC]  
 
'''Date''': [https://www.timeanddate.com/worldclock/fixedtime.html?msg=89th+Mahara+developer+meeting&iso=20220413T11&p1=264 Tuesday, 12 April 2022 at 23:00 UTC]  
 
+
* Robert
'''Chair:''' Robert
+
* Gold
 
 
'''Minute taker:''' Gold<br />
 

Revision as of 17:20, 10 March 2022

Agenda for the 88th Mahara developer meeting on 2 March 2022 at 7:30 UTC (check your time zone).

We will meet using Big Blue Button (A Catalyst staff member will initiate the call).

Our Mahara channel on Matrix will be our backup in case there are problems with the web conferencing tool and we'll need to chat to resolve it. You can connect to our Matrix channel also using the #mahara channel on Freenode IRC.

  • Chair: Dianne Tennent
  • Minute taker: Doris Tam

Chair and minute taker duties explained

Agenda

  • Items from previous meetings:
    • Aim to have Gerrit available for the community by the next developer meeting
    • Robert to look into how to stop people pushing to stable branches
    • Gold to check if composer can access specific parts of 3rd party libraries only, to reduce security vulnerability
    • Test Fergus's docker patch for working with PHP8
    • Kristina to write announcement about PHP8 compatibility
  • Doris' application to become an approved developer
  • Direct SQL query vs. 'get_user' can speed things up, e.g. in group submissions management → Can we use it? Is there a better way? What about PHP8? (Kristina on behalf of Alex Del Ponte)
  • Elasticsearch - Gold
  • Mahara 22.04
  • Any other business
  • Next meeting and chair

Minutes

Recording

Attendees: Andreas, Dianne, Doris, Kristina, and Robert.

Items from previous meeting

A new Gerrit to be available for the community

This task is ongoing and needs a little bit more time due to the availability of people. The team is working hard on testing the site. We are excited about the modern look of the new site.

Action item: Catalyst team to make the new Gerrit available to the public.

How to stop people pushing to _STABLE branches

This task is ongoing. Robert mentioned that because people can get around the make push command, it's not an urgent fix that would need to be made. It's also not a simple fix. We rely on the _STABLE branches for our releases. Therefore, we cannot lock those branches. This item will be looked into some more.

Reduce security vulnerabilities by getting Composer to install only used plugins

Context: Not all plugins are installed on Mahara sites. We want to reduce the amount of plugin code on sites that don't need them so that when a plugin has a security vulnerability, sites who don't have it installed are not affected. The idea is to get our package manager, Composer, to ignore libraries that aren't frequently used, e.g. SimpleSAML.

Gold mentions that we are unable to use Composer to do this for us and that another option is to write our own script to clean up parts we don't want in the final build package. This item is closed.

Fergus's Docker patch working with PHP8

Gold and Robert have tested Fergus' patch and it works! Kristina struggled getting it to work and may need to reinstall everything. Gold is working on getting Behat to work with the Docker patch so the tests can find our PHP 8 errors for us. The rest of the team has yet to test this Docker patch and get it working in their environment.

Kristina's PHP compatibility announcement

She has made a forum post. We are not close to having Mahara 22.04 to be PHP 8 compatible. Our aim is to have 22.10 to work with both PHP 7 to 8, with minimal PHP 8 support. After that, we will be working on full PHP 8 support, which will mean that PHP 7 will no longer be supported. The goal is to have PHP 8.1 compatibility as it will be there for the upcoming Ubuntu LTS. Whether that is Mahara 22.04 or 22.10 has not yet been decided as we are working on the transition at the moment.

Doris' application to become an approved developer

Congratulations to Doris. Her application convinced all five approved developers who were present or had sent in their votes prior to the meeting. Eight approved developers were eligible to vote with one not being active any more and now removed and two others not available to vote.

Action item: Robert to add Doris into all the Approved Developer groups on Gerrit and Launchpad.

Improving of get_user for group submissions management

Context: Recently, a change was made to use the get_users function to get account names in the group submission management. The performance of this function becomes very slow and affects UX when the number of submissions increase, e.g. with three evaluators, a site may have 2,400 submissions in a course which would make calls to the database 7,200 times slowing things down. The idea is that a direct DB query can provide better performance than the function. Would it be permissible to make these direct database calls instead of using the function?

get_user is used to get extra stuff about a person, e.g. account settings and activity preferences. Improving this will need to consider what is in on the submission output.

Ideas:

  • If only the name needs to be displayed, one idea is to modify the function to get a 'bare bones' version by giving a flag.
  • The DB formatting might be a bottleneck that is slowing things down.

Action item: Robert to look into improving the get_user function for submissions management, e.g. a bulk version and a bare-bones version.

Gold's work on Elasticsearch 7

Making progress and going through testing scenarios at the moment. For this to be ready for the 22.04 release as it is planned, a team effort is required to thoroughly test elasticsearch. Gold suggests a team of 2-3 devs working on for a couple days each week. We can use the @manual tag in Behat tests to help us with this when testing locally. There is is a site set up for this and ready for internal Catalyst testing.

Gold recommends that we have Behat tests describing the potential scenarios for known usage, e.g. a background standing up, the number of groups, institutions, and to index the content as well as run tests to make sure that when a particular search is done, it checks what it should and shouldn’t see.

Elasticsearch has been hardcoded into the Mahara codebase and those references need to be eventually found and checked for Elasticsearch 7. The general search is there, works, and is being debugged.

Mahara 22.04

The release is usually due in the second half of April. This release will be similar to the 21.10 release where we have a few new features for portfolio work and lots of internal improvements to the infrastructure, codebase, and reduce technical debt, e.g. security, bug-fixing, Elasticsearch, and PHP 8.

In the first half of April, a preview version will be available for people to install and test. The release manager will update the details and documentation.

Action item: Kristina to discuss with the Catalyst team to organise devs to get Elasticsearch tested.

Key decisions

  • Doris is now an approved developer.
  • Composer is not the tool to hold back plugin code when sites have not installed them. A script will need to be written.
  • Fergus's docker patch with PHP 8 works!
  • Gold's work on elasticsearch is ready for testing and a team is needed to help to get it to the release.
  • get_user is a bottle neck and needs improvements/modifications.

Action items

  • Robert - Look into improving the get_user function for submissions management, e.g. a bulk version and a bare-bones version.
  • Robert - Add Doris into all the Approved Developer groups on Gerrit and Launchpad.
  • Kristina - Discuss with the team to organise devs to get Elasticsearch tested.
  • Catalyst team - Make the new Gerrit available to the public.

The next developer meeting plan

Date: Tuesday, 12 April 2022 at 23:00 UTC

  • Robert
  • Gold