Actions

Difference between revisions of "Developer Area/Developer Meetings/87"

From Mahara Wiki

< Developer Area‎ | Developer Meetings
m
m
 
(3 intermediate revisions by 2 users not shown)
Line 22: Line 22:
  
 
==Minutes==
 
==Minutes==
 +
 +
[https://seafile.catalyst.net.nz/f/bcea13a210a943909569/ Recording of the meeting]
 +
 
===Attendees===
 
===Attendees===
* Kristina Hoeppner, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
+
* Kristina Hoeppner, [https://www.catalyst.net.nz/ Catalyst], Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
* Dianne Tennent, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (minute taker)
+
* Dianne Tennent, [https://www.catalyst.net.nz/ Catalyst], Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (minute taker)
* Doris Tam, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
+
* Doris Tam, [https://www.catalyst.net.nz/ Catalyst], Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
* Gold, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
+
* Gold, [https://www.catalyst.net.nz/ Catalyst], Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
* Robert Lyon, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (chair)
+
* Robert Lyon, [https://www.catalyst.net.nz/ Catalyst], Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (chair)
 
* Adam, [http://Learnopen.tech Learnopen.tech], San Antonio, Texas
 
* Adam, [http://Learnopen.tech Learnopen.tech], San Antonio, Texas
  
Line 33: Line 36:
 
* Gold: [[Proposals/Management of third party libraries|Proposal on how to handle third-party libraries going forward]] and include looking at templating engine.
 
* Gold: [[Proposals/Management of third party libraries|Proposal on how to handle third-party libraries going forward]] and include looking at templating engine.
 
** Gold proposes that we strip out all our 3rd party libraries and modernise how we handle them. This would involve using composer to maintain the list and specific versions we use. We can use composer to check out the library into our local codebase rather than maintaining it in our codebase
 
** Gold proposes that we strip out all our 3rd party libraries and modernise how we handle them. This would involve using composer to maintain the list and specific versions we use. We can use composer to check out the library into our local codebase rather than maintaining it in our codebase
 
 
** How would be deal with our own customisations? There are modules in composer that allow for patching, e.g. if we need to change/fix a library, we push a patch to the git repo for that library and then reference that from the composer file. When we pull that down it will pull the patch down and patch our local copy with it.
 
** How would be deal with our own customisations? There are modules in composer that allow for patching, e.g. if we need to change/fix a library, we push a patch to the git repo for that library and then reference that from the composer file. When we pull that down it will pull the patch down and patch our local copy with it.
 
+
** What if people are not working with Git? They will get a zip file that we construct in our build process.
** What if people are not working with git? They will get a zip file that we construct in our build process.
 
 
 
 
** Some people can’t work with composer because they are behind a firewall? They can use the zip file.
 
** Some people can’t work with composer because they are behind a firewall? They can use the zip file.
 
 
**Note: our stable release builds will still include the third party libraries, but our git repo would be lighter
 
**Note: our stable release builds will still include the third party libraries, but our git repo would be lighter
 
** How to rework our codebase? We include them at the moment already, so, not much difference, just change how we handle them. Include an autoload.php file at the beginning of a file, and then reference the library with ‘use’ statements to pull them into context for accessing within the code.
 
** How to rework our codebase? We include them at the moment already, so, not much difference, just change how we handle them. Include an autoload.php file at the beginning of a file, and then reference the library with ‘use’ statements to pull them into context for accessing within the code.
 
 
** Note: good opportunity to do this refactoring at the same time as doing php8 updates
 
** Note: good opportunity to do this refactoring at the same time as doing php8 updates
 
** What if the upstream disappears? We have the build files/archives, we can always get access to old versions.
 
** What if the upstream disappears? We have the build files/archives, we can always get access to old versions.
 
 
** Discussion notes
 
** Discussion notes
*** With mahara readme files, we tend to fetch and install in many different ways, sometimes we use a copy and sometimes we use some parts from upstream, so in this case with composer we may not need to take all files, just ones we want, e.g. only 20%
+
*** With Mahara readme files, we tend to fetch and install in many different ways, sometimes we use a copy and sometimes we use some parts from upstream, so in this case with composer we may not need to take all files, just ones we want, e.g. only 20%
 
*** This could reduce security risks
 
*** This could reduce security risks
*** e.g. with our auth libraries, we only need SAML, so if composer can just pull saml and not the others, this avoids any issues in openidconnect that we don’t use. The same with ADOdb e.g. only need postgres and sql
+
*** e.g. with our auth libraries, we only need SAML, so if composer can just pull SAML and not the others, this avoids any issues in OpenID Connect that we don’t use. The same with ADOdb e.g. only need Postgres and SQL
 
*** But we don’t know if composer does that – Gold will check
 
*** But we don’t know if composer does that – Gold will check
 
*** We can use different version numbers of each library that are compatible with different php8 versions etc without having to change a bunch of code
 
*** We can use different version numbers of each library that are compatible with different php8 versions etc without having to change a bunch of code
 
 
* Kristina: Write announcement about PHP 8 compatibility. To be written this week.
 
* Kristina: Write announcement about PHP 8 compatibility. To be written this week.
 
 
* Gold and Dianne: Look into support of [https://wiki.mahara.org/wiki/Developer_Area/Plugins/Third_party third-party libraries and PHP 8 support] (after the meeting decided that Gold will look into third-party libraries and Dianne into frequently used Linux distributions)
 
* Gold and Dianne: Look into support of [https://wiki.mahara.org/wiki/Developer_Area/Plugins/Third_party third-party libraries and PHP 8 support] (after the meeting decided that Gold will look into third-party libraries and Dianne into frequently used Linux distributions)
 
** Gold: third-party libraries
 
** Gold: third-party libraries
*** Most 3rd party libraries should be working with php8. However, as an example, OAuth1 is not compatible, so we need to remember that if someone is using LTI 1.1, they can’t upgrade to PHP8. And there will be some other exceptions, e.g. if you’re using this you can’t use Mahara with php8
+
*** Most 3rd party libraries should be working with php8. However, as an example, OAuth1 is not compatible, so we need to remember that if someone is using LTI 1.1, they can’t upgrade to PHP8. And there will be some other exceptions, e.g. if you’re using this you can’t use Mahara with PHP 8.
*** We are taking a stepped approach – make Mahara work with php7 and 8 as best we can by April, ensuring it works with both, but not specifically built for PHP8.
+
*** We are taking a stepped approach – make Mahara work with PHP 7 and 8 as best we can by April, ensuring it works with both, but not specifically built for PHP 8.
 
*** Also, when we cannot yet fully upgrade to PHP8 for a third-party dependency and that dependency is not critical to Mahara but some people will need it, we will recommend they don't yet upgrade to PHP8.
 
*** Also, when we cannot yet fully upgrade to PHP8 for a third-party dependency and that dependency is not critical to Mahara but some people will need it, we will recommend they don't yet upgrade to PHP8.
*** There is a lot of deprecated stuff we need to fix, e.g. class reflection has been deprecated in PHP7, e.g. the ‘get’ class – that is just gone, so we need to replicate that and replace it with something that is compatible with 7 and 8. There were 101 occurrences of this. [https://www.amitmerchant.com/the-get-class-alternative-in-php8/#:~:text=Using%20%3A%3Aclass%20on%20objects,of%20get_class()%20like%20so. Potential solution.]
+
*** There is a lot of deprecated stuff we need to fix, e.g. class reflection has been deprecated in PHP 7, e.g. the ‘get’ class – that is just gone, so we need to replicate that and replace it with something that is compatible with 7 and 8. There were 101 occurrences of this. [https://www.amitmerchant.com/the-get-class-alternative-in-php8/#:~:text=Using%20%3A%3Aclass%20on%20objects,of%20get_class()%20like%20so. Potential solution.]
 
*** There is a [https://wiki.mahara.org/wiki/Developer_Area/PHP8/Upgrade_to_PHP8 wiki page] to list issues as they are discovered here:  
 
*** There is a [https://wiki.mahara.org/wiki/Developer_Area/PHP8/Upgrade_to_PHP8 wiki page] to list issues as they are discovered here:  
 
*** Another issue is passing null as an object is no longer allowed
 
*** Another issue is passing null as an object is no longer allowed
 
*** We need to start running PHP8 now and get Mahara working with it  
 
*** We need to start running PHP8 now and get Mahara working with it  
*** Fergus has set up docker to be able to switch between PHP8 and 7. Bob tried before xmas without any issues. [https://reviews.mahara.org/#/c/12322/ Patch is here].
+
*** Fergus has set up docker to be able to switch between PHP 8 and 7. Bob tried before Xmas without any issues. [https://reviews.mahara.org/#/c/12322/ Patch is here].
*** Adam to help with testing docker environment before merging with core
+
*** Adam to help with testing docker environment before merging with core.
 
** Dianne: PHP 8 support in frequently used Linux distributions
 
** Dianne: PHP 8 support in frequently used Linux distributions
*** [https://wiki.mahara.org/wiki/Developer_Area/PHP8 Wiki page] created that outlines which distros support which versions of php8.
+
*** [https://wiki.mahara.org/wiki/Developer_Area/PHP8 Wiki page] created that outlines which distros support which versions of PHP 8.
*** Ubuntu has native php8 in their 22.04 release which is an LTS release supported until April 2027
+
*** Ubuntu has native PHP 8 in their 22.04 release which is an LTS release supported until April 2027.
 
 
 
* N.N.: Include some sanity checks in the makefile, e.g. if someone tries to push to _STABLE to bring up an alert.
 
* N.N.: Include some sanity checks in the makefile, e.g. if someone tries to push to _STABLE to bring up an alert.
 
** This hasn’t been a problem so far, but it would be good to find a way to stop anyone from being able to push to the stable branches  
 
** This hasn’t been a problem so far, but it would be good to find a way to stop anyone from being able to push to the stable branches  
Line 76: Line 70:
 
===Gold: Move CLI scripts out of wwwroot===
 
===Gold: Move CLI scripts out of wwwroot===
 
* [https://wiki.mahara.org/wiki/Proposals/Mahara_Shell Proposal details]
 
* [https://wiki.mahara.org/wiki/Proposals/Mahara_Shell Proposal details]
* Creates a standardised method for exposing functions that can be run from cli, actions that can be run, and giving info about them
+
* Creates a standardised method for exposing functions that can be run from CLI, actions that can be run, and giving info about them
* Good to move out of htdocs/web root, because cli scripts are often less secure, they can do lots of things without having to log in as admin. A php vulnerability could expose them.
+
* Good to move out of htdocs/web root, because CLI scripts are often less secure, they can do lots of things without having to log in as admin. A php vulnerability could expose them.
 
* We should prevent them from being able to be accessed through a web browser
 
* We should prevent them from being able to be accessed through a web browser
* Cli commands would become similar to cron in that they live in the background.  
+
* CLI commands would become similar to cron in that they live in the background.  
* The new system is already halfway built by Gold. He grabbed the cli library and pulled it out into an external folder and created mash commands. He has put elastic search functions into classes. It’s mostly working for elastic search, but still needs a tidy up
+
* The new system is already halfway built by Gold. He grabbed the CLI library and pulled it out into an external folder and created mash commands. He has put elastic search functions into classes. It’s mostly working for elastic search, but still needs a tidy up
 
* Can still pass in variables to do certain things
 
* Can still pass in variables to do certain things
  
Line 92: Line 86:
 
* Some 3rd party library come with Twig stuff
 
* Some 3rd party library come with Twig stuff
 
* Supported by [https://symfony.com/ Symfony]
 
* Supported by [https://symfony.com/ Symfony]
* Symphony have lots of options for 3rd party libraries, they have code standards so they’re easy to understand and more compatible with each other
+
* Symfony have lots of options for 3rd party libraries, they have code standards so they’re easy to understand and more compatible with each other
 
* Next step is how to approach the changeover
 
* Next step is how to approach the changeover
 
* On hold until after the php8 transition, possible to do with bootstrap5
 
* On hold until after the php8 transition, possible to do with bootstrap5
Line 102: Line 96:
  
 
===To do===
 
===To do===
* Elastic search update for next meeting
+
* Elasticsearch update for next meeting
* Will have Gerrit available for the community by the next developer meeting
+
* Aim to have Gerrit available for the community by the next developer meeting
* Bob to look into how to stop people pushing to stable branches
+
* Robert to look into how to stop people pushing to stable branches
 
* Gold to check if composer can access specific parts of 3rd party libraries only, to reduce security vulnerability
 
* Gold to check if composer can access specific parts of 3rd party libraries only, to reduce security vulnerability
 
* Test [https://reviews.mahara.org/#/c/12322/ Fergus's docker patch] for working with PHP8
 
* Test [https://reviews.mahara.org/#/c/12322/ Fergus's docker patch] for working with PHP8
 
* Kristina to write announcement about PHP8 compatibility
 
* Kristina to write announcement about PHP8 compatibility

Latest revision as of 07:59, 20 January 2022

Agenda for the 87th Mahara developer meeting on 18 January 2022 at 10:30 NZDT (check your time zone).

We will meet using Big Blue Button (Kristina will initiate the call).

Our Mahara channel on Matrix will be our backup in case there are problems with the web conferencing tool and we'll need to chat to resolve it. You can connect to our Matrix channel also using the #mahara channel on Freenode IRC.

  • Chair: Robert Lyon
  • Minute taker: Dianne Tennent

Agenda

  • Items from previous meetings:
    • Gold: Proposal on how to handle third-party libraries going forward and include looking at templating engine.
    • Kristina: Write announcement about PHP 8 compatibility
    • Gold and Dianne: Look into support of third-party libraries and PHP 8 support (after the meeting decided that Gold will look into third-party libraries and Dianne into frequently used Linux distributions)
    • N.N.: Include some sanity checks in the makefile, e.g. if someone tries to push to _STABLE to bring up an alert.
  • Move CLI scripts out of wwwroot - Gold
  • Dwoo replacement - Doris
  • Your item here
  • Any other business
  • Next meeting and chair

Minutes

Recording of the meeting

Attendees

  • Kristina Hoeppner, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
  • Dianne Tennent, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (minute taker)
  • Doris Tam, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
  • Gold, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
  • Robert Lyon, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (chair)
  • Adam, Learnopen.tech, San Antonio, Texas

Items from previous meetings

  • Gold: Proposal on how to handle third-party libraries going forward and include looking at templating engine.
    • Gold proposes that we strip out all our 3rd party libraries and modernise how we handle them. This would involve using composer to maintain the list and specific versions we use. We can use composer to check out the library into our local codebase rather than maintaining it in our codebase
    • How would be deal with our own customisations? There are modules in composer that allow for patching, e.g. if we need to change/fix a library, we push a patch to the git repo for that library and then reference that from the composer file. When we pull that down it will pull the patch down and patch our local copy with it.
    • What if people are not working with Git? They will get a zip file that we construct in our build process.
    • Some people can’t work with composer because they are behind a firewall? They can use the zip file.
    • Note: our stable release builds will still include the third party libraries, but our git repo would be lighter
    • How to rework our codebase? We include them at the moment already, so, not much difference, just change how we handle them. Include an autoload.php file at the beginning of a file, and then reference the library with ‘use’ statements to pull them into context for accessing within the code.
    • Note: good opportunity to do this refactoring at the same time as doing php8 updates
    • What if the upstream disappears? We have the build files/archives, we can always get access to old versions.
    • Discussion notes
      • With Mahara readme files, we tend to fetch and install in many different ways, sometimes we use a copy and sometimes we use some parts from upstream, so in this case with composer we may not need to take all files, just ones we want, e.g. only 20%
      • This could reduce security risks
      • e.g. with our auth libraries, we only need SAML, so if composer can just pull SAML and not the others, this avoids any issues in OpenID Connect that we don’t use. The same with ADOdb e.g. only need Postgres and SQL
      • But we don’t know if composer does that – Gold will check
      • We can use different version numbers of each library that are compatible with different php8 versions etc without having to change a bunch of code
  • Kristina: Write announcement about PHP 8 compatibility. To be written this week.
  • Gold and Dianne: Look into support of third-party libraries and PHP 8 support (after the meeting decided that Gold will look into third-party libraries and Dianne into frequently used Linux distributions)
    • Gold: third-party libraries
      • Most 3rd party libraries should be working with php8. However, as an example, OAuth1 is not compatible, so we need to remember that if someone is using LTI 1.1, they can’t upgrade to PHP8. And there will be some other exceptions, e.g. if you’re using this you can’t use Mahara with PHP 8.
      • We are taking a stepped approach – make Mahara work with PHP 7 and 8 as best we can by April, ensuring it works with both, but not specifically built for PHP 8.
      • Also, when we cannot yet fully upgrade to PHP8 for a third-party dependency and that dependency is not critical to Mahara but some people will need it, we will recommend they don't yet upgrade to PHP8.
      • There is a lot of deprecated stuff we need to fix, e.g. class reflection has been deprecated in PHP 7, e.g. the ‘get’ class – that is just gone, so we need to replicate that and replace it with something that is compatible with 7 and 8. There were 101 occurrences of this. Potential solution.
      • There is a wiki page to list issues as they are discovered here:
      • Another issue is passing null as an object is no longer allowed
      • We need to start running PHP8 now and get Mahara working with it
      • Fergus has set up docker to be able to switch between PHP 8 and 7. Bob tried before Xmas without any issues. Patch is here.
      • Adam to help with testing docker environment before merging with core.
    • Dianne: PHP 8 support in frequently used Linux distributions
      • Wiki page created that outlines which distros support which versions of PHP 8.
      • Ubuntu has native PHP 8 in their 22.04 release which is an LTS release supported until April 2027.
  • N.N.: Include some sanity checks in the makefile, e.g. if someone tries to push to _STABLE to bring up an alert.
    • This hasn’t been a problem so far, but it would be good to find a way to stop anyone from being able to push to the stable branches
    • ‘make push’ will prevent this, but if not using ‘make push’, then makefile won’t be triggered, so we need to figure out another way to prevent people pushing directly to the stable branches (who are not supposed to)

Gold: Move CLI scripts out of wwwroot

  • Proposal details
  • Creates a standardised method for exposing functions that can be run from CLI, actions that can be run, and giving info about them
  • Good to move out of htdocs/web root, because CLI scripts are often less secure, they can do lots of things without having to log in as admin. A php vulnerability could expose them.
  • We should prevent them from being able to be accessed through a web browser
  • CLI commands would become similar to cron in that they live in the background.
  • The new system is already halfway built by Gold. He grabbed the CLI library and pulled it out into an external folder and created mash commands. He has put elastic search functions into classes. It’s mostly working for elastic search, but still needs a tidy up
  • Can still pass in variables to do certain things

Doris: Dwoo replacement

  • The Dwoo project has been archived, so there will be no more updates
  • Last year in Dec, we reached out to Catalyst for feedback on php templates
  • Looked at 5 libraries to compare: Twig, Smarty, Mustache, Plates and Latte
  • Twig was most supported and lots of people use it, there is a lot of support out there if we get stuck
  • Smarty has changed, we can’t do a lot of customisation because it’s quite closed off
  • Not much info on other three
  • Some 3rd party library come with Twig stuff
  • Supported by Symfony
  • Symfony have lots of options for 3rd party libraries, they have code standards so they’re easy to understand and more compatible with each other
  • Next step is how to approach the changeover
  • On hold until after the php8 transition, possible to do with bootstrap5

Next meeting

To do

  • Elasticsearch update for next meeting
  • Aim to have Gerrit available for the community by the next developer meeting
  • Robert to look into how to stop people pushing to stable branches
  • Gold to check if composer can access specific parts of 3rd party libraries only, to reduce security vulnerability
  • Test Fergus's docker patch for working with PHP8
  • Kristina to write announcement about PHP8 compatibility