Developer Area/Developer Meetings/87

Agenda for the 87th Mahara developer meeting on 18 January 2022 at 10:30 NZDT (check your time zone).

We will meet using Big Blue Button (Kristina will initiate the call).

Our Mahara channel on Matrix will be our backup in case there are problems with the web conferencing tool and we'll need to chat to resolve it. You can connect to our Matrix channel also using the #mahara channel on Freenode IRC.

• Chair: Robert Lyon
• Minute taker: Dianne Tennent

Agenda

• Items from previous meetings:
  • Gold: Proposal on how to handle third-party libraries going forward and include looking at templating engine.
  • Kristina: Write announcement about PHP 8 compatibility
  • Gold and Dianne: Look into support of third-party libraries and PHP 8 support (after the meeting decided that Gold will look into third-party libraries and Dianne into frequently used Linux distributions)
  • N.N.: Include some sanity checks in the makefile, e.g. if someone tries to push to _STABLE to bring up an alert.
• Move CLI scripts out of wwwroot - Gold
• Dwoo replacement - Doris
• Your item here
• Any other business
• Next meeting and chair

Minutes

Recording of the meeting

Attendees

• Kristina Hoeppner, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
• Dianne Tennent, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (minute taker)
• Doris Tam, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
• Gold, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand
• Robert Lyon, Catalyst, Te Whanganui-a-Tara Wellington, Aotearoa New Zealand (chair)
• Adam, Learnopen.tech, San Antonio, Texas

Items from previous meetings

• Gold: Proposal on how to handle third-party libraries going forward and include looking at templating engine.
  • Gold proposes that we strip out all our 3rd party libraries and modernise how we handle them. This would involve using composer to maintain the list and specific versions we use. We can use composer to check out the library into our local codebase rather than maintaining it in our codebase
  • How would be deal with our own customisations? There are modules in composer that allow for patching, e.g. if we need to change/fix a library, we push a patch to the git repo for that library and then reference that from the composer file. When we pull that down it will pull the patch down and patch our local copy with it.
  • What if people are not working with Git? They will get a zip file that we construct in our build process.
  • Some people can’t work with composer because they are behind a firewall? They can use the zip file.
  • Note: our stable release builds will still include the third party libraries, but our git repo would be lighter
  • How to rework our codebase? We include them at the moment already, so, not much difference, just change how we handle them. Include an autoload.php file at the beginning of a file, and then reference the library with ‘use’ statements to pull them into context for accessing within the code.
  • Note: good opportunity to do this refactoring at the same time as doing php8 updates
  • What if the upstream disappears? We have the build files/archives, we can always get access to old versions.
  • Discussion notes
    • With Mahara readme files, we tend to fetch and install in many different ways, sometimes we use a copy and sometimes we use some parts from upstream, so in this case with composer we may not need to take all files, just ones we want, e.g. only 20%
    • This could reduce security risks
    • e.g. with our auth libraries, we only need SAML, so if composer can just pull SAML and not the others, this avoids any issues in OpenID Connect that we don’t use. The same with ADOdb e.g. only need Postgres and SQL
    • But we don’t know if composer does that – Gold will check
    • We can use different version numbers of each library that are compatible with different php8 versions etc without having to change a bunch of code
• Kristina: Write announcement about PHP 8 compatibility. To be written this week.
• Gold and Dianne: Look into support of third-party libraries and PHP 8 support (after the meeting decided that Gold will look into third-party libraries and Dianne into frequently used Linux distributions)
  • Gold: third-party libraries
    • Most 3rd party libraries should be working with php8. However, as an example, OAuth1 is not compatible, so we need to remember that if someone is using LTI 1.1, they can’t upgrade to PHP8. And there will be some other exceptions, e.g. if you’re using this you can’t use Mahara with PHP 8.
    • We are taking a stepped approach – make Mahara work with PHP 7 and 8 as best we can by April, ensuring it works with both, but not specifically built for PHP 8.
    • Also, when we cannot yet fully upgrade to PHP8 for a third-party dependency and that dependency is not critical to Mahara but some people will need it, we will recommend they don't yet upgrade to PHP8.
    • There is a lot of deprecated stuff we need to fix, e.g. class reflection has been deprecated in PHP 7, e.g. the ‘get’ class – that is just gone, so we need to replicate that and replace it with something that is compatible with 7 and 8. There were 101 occurrences of this. Potential solution.
    • There is a wiki page to list issues as they are discovered here:
    • Another issue is passing null as an object is no longer allowed
    • We need to start running PHP8 now and get Mahara working with it
    • Fergus has set up docker to be able to switch between PHP 8 and 7. Bob tried before Xmas without any issues. Patch is here.
    • Adam to help with testing docker environment before merging with core.
  • Dianne: PHP 8 support in frequently used Linux distributions
    • Wiki page created that outlines which distros support which versions of PHP 8.
    • Ubuntu has native PHP 8 in their 22.04 release which is an LTS release supported until April 2027.
• N.N.: Include some sanity checks in the makefile, e.g. if someone tries to push to _STABLE to bring up an alert.
  • This hasn’t been a problem so far, but it would be good to find a way to stop anyone from being able to push to the stable branches
  • ‘make push’ will prevent this, but if not using ‘make push’, then makefile won’t be triggered, so we need to figure out another way to prevent people pushing directly to the stable branches (who are not supposed to)

Gold: Move CLI scripts out of wwwroot

• Proposal details
• Creates a standardised method for exposing functions that can be run from CLI, actions that can be run, and giving info about them
• Good to move out of htdocs/web root, because CLI scripts are often less secure, they can do lots of things without having to log in as admin. A php vulnerability could expose them.
• We should prevent them from being able to be accessed through a web browser
• CLI commands would become similar to cron in that they live in the background.
• The new system is already halfway built by Gold. He grabbed the CLI library and pulled it out into an external folder and created mash commands. He has put elastic search functions into classes. It’s mostly working for elastic search, but still needs a tidy up
• Can still pass in variables to do certain things

Doris: Dwoo replacement

• The Dwoo project has been archived, so there will be no more updates
• Last year in Dec, we reached out to Catalyst for feedback on php templates
• Looked at 5 libraries to compare: Twig, Smarty, Mustache, Plates and Latte
• Twig was most supported and lots of people use it, there is a lot of support out there if we get stuck
• Smarty has changed, we can’t do a lot of customisation because it’s quite closed off
• Not much info on other three
• Some 3rd party library come with Twig stuff
• Supported by Symfony
• Symfony have lots of options for 3rd party libraries, they have code standards so they’re easy to understand and more compatible with each other
• Next step is how to approach the changeover
• On hold until after the php8 transition, possible to do with bootstrap5

Next meeting

• Wednesday, 2 March 2022, 07:30 UTC
• Chair: Dianne
• Minutes: Doris

To do

• Elasticsearch update for next meeting
• Aim to have Gerrit available for the community by the next developer meeting
• Robert to look into how to stop people pushing to stable branches
• Gold to check if composer can access specific parts of 3rd party libraries only, to reduce security vulnerability
• Test Fergus's docker patch for working with PHP8
• Kristina to write announcement about PHP8 compatibility