Actions

Difference between revisions of "Developer Area/Dev Team Ideas"

From Mahara Wiki

< Developer Area
(Created page with "Current wishlist ideas from the Dev Team, these may be implemented at some point as time allows. Note: We should port stuff to here from the devteamideas page on the catalyst wi…")
 
Line 5: Line 5:
 
<div id="section_1">
 
<div id="section_1">
  
= Documentation =
+
=Documentation=
  
 
<div id="section_2">
 
<div id="section_2">
  
== API docs ==
+
==API docs==
  
 
* Developers like PHPDoc documentation, we should provide this on mahara.org (they'll be too big to ship with Mahara itself)
 
* Developers like PHPDoc documentation, we should provide this on mahara.org (they'll be too big to ship with Mahara itself)
Line 19: Line 19:
 
</div><div id="section_3">
 
</div><div id="section_3">
  
== Developer documentation ==
+
==Developer documentation==
  
 
We have none, we want some. API docs are helpful but aren't exactly task based.
 
We have none, we want some. API docs are helpful but aren't exactly task based.
Line 27: Line 27:
 
<div id="section_4">
 
<div id="section_4">
  
=== Writing themes ===
+
===Writing themes===
  
 
Things to cover:
 
Things to cover:
Line 37: Line 37:
 
</div><div id="section_5">
 
</div><div id="section_5">
  
=== Languages ===
+
===Languages===
  
 
Things to cover:
 
Things to cover:
Line 48: Line 48:
 
</div><div id="section_6">
 
</div><div id="section_6">
  
=== Blocktypes ===
+
===Blocktypes===
  
 
This might be best done as a tutorial on writing a blocktype, since that might cover most of the API methods (good link in with API methods throughout the docs is a good thing)
 
This might be best done as a tutorial on writing a blocktype, since that might cover most of the API methods (good link in with API methods throughout the docs is a good thing)
Line 54: Line 54:
 
</div><div id="section_7">
 
</div><div id="section_7">
  
=== Artefact plugins ===
+
===Artefact plugins===
  
 
The tutorial method might not work here because they're so big.. but maybe it will.
 
The tutorial method might not work here because they're so big.. but maybe it will.
Line 60: Line 60:
 
</div><div id="section_8">
 
</div><div id="section_8">
  
=== Search plugins ===
+
===Search plugins===
  
 
Probably a low priority thing to document, pending search subsystem rewrite.
 
Probably a low priority thing to document, pending search subsystem rewrite.
Line 66: Line 66:
 
</div><div id="section_9">
 
</div><div id="section_9">
  
=== Database schema docs ===
+
===Database schema docs===
  
 
* There are tools available to automatically generate a schema from a postgres database, we should investigate using those to provide schema docs
 
* There are tools available to automatically generate a schema from a postgres database, we should investigate using those to provide schema docs
Line 74: Line 74:
 
</div></div><div id="section_10">
 
</div></div><div id="section_10">
  
== End user documentation ==
+
==End user documentation==
  
 
Andy made a document which should be loaded into the wiki.
 
Andy made a document which should be loaded into the wiki.
Line 85: Line 85:
 
</div></div><div id="section_11">
 
</div></div><div id="section_11">
  
= Testing framework =
+
=Testing framework=
  
 
As Mahara gets bigger, it's going to be harder to make changes without breaking behaviour. Automatic testing helps improve confidence that nothing has been broken by a change.
 
As Mahara gets bigger, it's going to be harder to make changes without breaking behaviour. Automatic testing helps improve confidence that nothing has been broken by a change.
Line 93: Line 93:
 
<div id="section_12">
 
<div id="section_12">
  
== Selenium tests ==
+
==Selenium tests==
  
 
We currently have some but they're probably broken. Nigel organised their creation but hasn't run them for ages, mostly because he couldn't find a selenium IDE for firefox 3.
 
We currently have some but they're probably broken. Nigel organised their creation but hasn't run them for ages, mostly because he couldn't find a selenium IDE for firefox 3.
Line 101: Line 101:
 
</div><div id="section_13">
 
</div><div id="section_13">
  
== Unit tests ==
+
==Unit tests==
  
 
We're looking to draw on some Moodle experience here to help set up a framework.  Penny and Nigel had an IRC meeting with Nico Connault at around 1100 GMT on Dec 6 to see what information he could give us.
 
We're looking to draw on some Moodle experience here to help set up a framework.  Penny and Nigel had an IRC meeting with Nico Connault at around 1100 GMT on Dec 6 to see what information he could give us.
Line 116: Line 116:
 
</div></div><div id="section_14">
 
</div></div><div id="section_14">
  
= Major reworking =
+
=Major reworking=
  
 
Various subsystems of Mahara may or will need a rewrite over time.
 
Various subsystems of Mahara may or will need a rewrite over time.
Line 122: Line 122:
 
<div id="section_15">
 
<div id="section_15">
  
== Reworking of DML/Database abstraction ==
+
==Reworking of DML/Database abstraction==
  
 
DML, and ADODB under it, have severe API and architecture flaws that hurt and continue to hurt Mahara.
 
DML, and ADODB under it, have severe API and architecture flaws that hurt and continue to hurt Mahara.
Line 137: Line 137:
 
</div><div id="section_16">
 
</div><div id="section_16">
  
== New MNET library ==
+
==New MNET library==
  
 
Neil wrote a nice modern MNET library. Drupal has integration with it. Moodle might get it in Moodle 2.0, so Mahara should look to use it too.
 
Neil wrote a nice modern MNET library. Drupal has integration with it. Moodle might get it in Moodle 2.0, so Mahara should look to use it too.
Line 145: Line 145:
 
</div><div id="section_17">
 
</div><div id="section_17">
  
== Theme subsystem work ==
+
==Theme subsystem work==
  
 
* Reworking theme support to modularise CSS/JS files
 
* Reworking theme support to modularise CSS/JS files
Line 156: Line 156:
 
</div><div id="section_18">
 
</div><div id="section_18">
  
== Reworking things to remove the javascript dependency ==
+
==Reworking things to remove the javascript dependency==
  
 
We want Mahara to work without javascript enabled. This is important for accessibility. Currently, javascript is required for some parts of the system:
 
We want Mahara to work without javascript enabled. This is important for accessibility. Currently, javascript is required for some parts of the system:
Line 168: Line 168:
 
</div><div id="section_19">
 
</div><div id="section_19">
  
== Language subsystem work ==
+
==Language subsystem work==
  
 
The current system is modelled off of Moodle's - PHP files containing language strings. It works OK, but is very inflexible about how it allows variable interpolation.
 
The current system is modelled off of Moodle's - PHP files containing language strings. It works OK, but is very inflexible about how it allows variable interpolation.
Line 176: Line 176:
 
<div id="section_20">
 
<div id="section_20">
  
=== Investigate David Mudrak's language translator ===
+
===Investigate David Mudrak's language translator===
  
 
We should look at it and see how it works, what it provides, and if it's good we should advertise it/help with maintainership etc.
 
We should look at it and see how it works, what it provides, and if it's good we should advertise it/help with maintainership etc.
Line 182: Line 182:
 
</div></div><div id="section_21">
 
</div></div><div id="section_21">
  
== Maharactl ==
+
==Maharactl==
  
 
Maharactl is a hypothetical script that can drive a Mahara website.
 
Maharactl is a hypothetical script that can drive a Mahara website.
Line 192: Line 192:
 
</div><div id="section_22">
 
</div><div id="section_22">
  
== Web services api ==
+
==Web services api==
  
 
See Maharactl section - need core APIs.
 
See Maharactl section - need core APIs.
Line 200: Line 200:
 
</div><div id="section_23">
 
</div><div id="section_23">
  
== Making forums into an artefact plugin ==
+
==Making forums into an artefact plugin==
  
 
The concept of interaction plugins was born before the work in Mahara 1.1 to make shared group artefacts.<br /><br /> Now we have such artefacts, it makes sense to implement forums as an artefact plugin, which can only be used in groups. This is opposed to something like the file artefact plugin, where both users and groups have file sections. (Note - there is no API to say whether a given artefact type can be used in a group or not - it's up to the artefact plugin itself to export the appropriate pages and code to do this).<br /><br /> Under such a model, groups might own posts and discussions but users would get republish rights on their own posts and discussions they had participated in. This would allow users to embed forum discussions and posts into views.
 
The concept of interaction plugins was born before the work in Mahara 1.1 to make shared group artefacts.<br /><br /> Now we have such artefacts, it makes sense to implement forums as an artefact plugin, which can only be used in groups. This is opposed to something like the file artefact plugin, where both users and groups have file sections. (Note - there is no API to say whether a given artefact type can be used in a group or not - it's up to the artefact plugin itself to export the appropriate pages and code to do this).<br /><br /> Under such a model, groups might own posts and discussions but users would get republish rights on their own posts and discussions they had participated in. This would allow users to embed forum discussions and posts into views.
Line 206: Line 206:
 
</div><div id="section_24">
 
</div><div id="section_24">
  
== Profile section reworking ==
+
==Profile section reworking==
  
 
From an e-mail discussion:
 
From an e-mail discussion:
Line 216: Line 216:
 
</blockquote></div><div id="section_25">
 
</blockquote></div><div id="section_25">
  
== Upgrades to bundled third party libraries ==
+
==Upgrades to bundled third party libraries==
  
 
* [https://eduforge.org/tracker/index.php?func=detail&aid=2919&group_id=176&atid=739 Tinymce] 
 
* [https://eduforge.org/tracker/index.php?func=detail&aid=2919&group_id=176&atid=739 Tinymce] 

Revision as of 16:13, 11 May 2011

Current wishlist ideas from the Dev Team, these may be implemented at some point as time allows.

Note: We should port stuff to here from the devteamideas page on the catalyst wiki - N

Documentation

API docs

  • Developers like PHPDoc documentation, we should provide this on mahara.org (they'll be too big to ship with Mahara itself)
  • We should probably generate it per major version, suggestion: api.mahara.org/1.0, api.mahara.org/1.1... what about api.mahara.org/stable and api.mahara.org/unstable as symlinks?
  • This will require that we go through the files and change the @phpdoc tags until the structure is generated properly, and of course we will have to document all the functions that need it!
  • Functions like the pieform callbacks probably shouldn't appear in the docs
  • How and when shall they be generated? From git.mahara.org as a nightly cronjob?

Developer documentation

We have none, we want some. API docs are helpful but aren't exactly task based.

Question: how do we separate them out by version (for when APIs change)?

Writing themes

Things to cover:

  • Theme inheritance, including how to write an entirely new theme
  • Smarty
  • Pieform templates

Languages

Things to cover:

  • Translation system basics - including why files are all over the place, how to find them, how to translate (aka editing php files)
  • David's adminlang tool
  • How to get language packs upstream (tie in with mentioning language based groups on mahara.org?)
  • Git commands for when you do get a pack upstream

Blocktypes

This might be best done as a tutorial on writing a blocktype, since that might cover most of the API methods (good link in with API methods throughout the docs is a good thing)

Artefact plugins

The tutorial method might not work here because they're so big.. but maybe it will.

Search plugins

Probably a low priority thing to document, pending search subsystem rewrite.

Database schema docs

  • There are tools available to automatically generate a schema from a postgres database, we should investigate using those to provide schema docs
  • We might need to add comments to each table and field (xmldb supports this)
  • Not sure where they would be hosted? Probably generated in a similar way to the api docs - perhaps they should go on api.mahara.org?

End user documentation

Andy made a document which should be loaded into the wiki.

Other docs we should provide:

  • How to install new themes, language packs etc.
  • Administrator's guide (Andy's document only covers users, not admins)

Testing framework

As Mahara gets bigger, it's going to be harder to make changes without breaking behaviour. Automatic testing helps improve confidence that nothing has been broken by a change.

Testing a webapp is tricky, luckily there are many tools to help. We can use selenium tests for testing the frontend, and unit tests for core libraries. But whatever we do, we need to set up a good framework first.

Selenium tests

We currently have some but they're probably broken. Nigel organised their creation but hasn't run them for ages, mostly because he couldn't find a selenium IDE for firefox 3.

We should investigate this further.

Unit tests

We're looking to draw on some Moodle experience here to help set up a framework.  Penny and Nigel had an IRC meeting with Nico Connault at around 1100 GMT on Dec 6 to see what information he could give us.

We decided on the following

  1. We need to identify a stragegy for creating test data:
    • Moodle currently has a full generator (which can be used to populate the database for demo purposes as well), and each test tells it which data it needs before the test runs, and cleans it up aftewards. It also keeps a full schema copy with a different dbprefix, and when you try and run a unit test, it checks to see if the schema needs updating (very similar to the main install/upgrade strategy).
    • Another strategy is to ship an SQL dump.
  2. We need to identify a few basic low hanging fruit areas in Mahara that we can easily write tests for. This shouldn't be a large area of the codebase that would require a lot of refactoring to get it to a point we can write elegant tests, and it needent even be an area that we place highest priority on for having tests for, but a simple, well contained area that lets us get our hands dirty with a test framework to see how it works and can be used in the rest of the system. Some potential cases are:
    • Adding/removing/moving block instances within a view - View and BlockInstance are both objects and this code is quite well abstracted
  3. We should expect that in order to be able to write clean tests to cover a lot of Mahara, we will need to refactor a lot of code. A prime example of this is lib/groups.php which we should refactor into a class and smaller, more self contained methods, splitting out sql-generation and data-fetching methods from functional ones, as we write tests for it.

Major reworking

Various subsystems of Mahara may or will need a rewrite over time.

Reworking of DML/Database abstraction

DML, and ADODB under it, have severe API and architecture flaws that hurt and continue to hurt Mahara.

A new database API should:

  • Utilise PDO - this is what will be the standard way of accessing databases through PHP into the future
  • Work with whatever test framework we come up with
  • Not hide SQL
  • ??

This needs to be discussed in more detail

New MNET library

Neil wrote a nice modern MNET library. Drupal has integration with it. Moodle might get it in Moodle 2.0, so Mahara should look to use it too.

A good thing about this is that the new library can test itself and all MNET work can be focused in one place.

Theme subsystem work

  • Reworking theme support to modularise CSS/JS files
  • Cut down size of default theme
  • Make certain parts easier to override
    • favicon
  • Correct caching of static files
  • Documentation

Reworking things to remove the javascript dependency

We want Mahara to work without javascript enabled. This is important for accessibility. Currently, javascript is required for some parts of the system:

  • The file artefact plugin absolutely requires it
  • The blog artefact plugin absolutely requires it
  • ...

We should list all parts requiring javascript, and what we are doing/can do to fix them.

Language subsystem work

The current system is modelled off of Moodle's - PHP files containing language strings. It works OK, but is very inflexible about how it allows variable interpolation.

Nigel thinks we should investigate the possibility of moving to the open-source-software standard gettext method of translation. Failing that, we should look at how to provide better ways of turning data from the application into language strings. Note that this is not a simple problem of providing flexible string interpolation - please read this article for more information about translation system issues and potential solutions.

Investigate David Mudrak's language translator

We should look at it and see how it works, what it provides, and if it's good we should advertise it/help with maintainership etc.

Maharactl

Maharactl is a hypothetical script that can drive a Mahara website.

The first prototype was a perl script using perl modules to provide an API to the database. We would probably need a solution that utilised existing code if this idea was to work properly.

This depends on sane core APIs, and may need the core to be decoupled from the web based interface before this is even remotely possible.

Web services api

See Maharactl section - need core APIs.

Note: We have an XMLRPC dispatcher in place which is used for MNET.

Making forums into an artefact plugin

The concept of interaction plugins was born before the work in Mahara 1.1 to make shared group artefacts.

Now we have such artefacts, it makes sense to implement forums as an artefact plugin, which can only be used in groups. This is opposed to something like the file artefact plugin, where both users and groups have file sections. (Note - there is no API to say whether a given artefact type can be used in a group or not - it's up to the artefact plugin itself to export the appropriate pages and code to do this).

Under such a model, groups might own posts and discussions but users would get republish rights on their own posts and discussions they had participated in. This would allow users to embed forum discussions and posts into views.

Profile section reworking

From an e-mail discussion:

> >     o Profile section reworking (make profile data NOT artefacts)
>
> why do you want to do that?  i sell that all the time (everything is an
> artefact) and people really like it.

Well I think this comes about after discussions I've had with RichardM.
Basically, back when views were done as view templates, everything as an
artefact was good and made sense because that's how things got into views.
But the blocktype concept does a great job of abstracting that. You can see
blocks we have now that have 0, 1 or more than one artefact in them.

Given that, the concept of an "artefact" begins to fade a little - at least
from a user's point of view. We used to say "an artefact is something that
can be put in a view" - but that statement does not tell the whole story
any more. Many artefacts can only be put in views in certain ways (e.g.
profile fields). Some have a special blocktype for them but many do not.
Plenty of things that are _not_ artefacts can be put in views.

However, "artefact" still makes sense in the mahara codebase. It's a bit
like Drupal's "node". Lots of things in mahara are artefacts, and thus are
provided by artefact plugins. It makes sense, for example, that forum posts
and discussions are artefacts even. I think it would be great if we can
refactor forum to be an artefact plugin so that posts and discussions can
be put in views.

Anyways, that's a really long way of telling the background to why I think
profile data should not be artefacts. At the end of the day, they're really
one tiny bit of metadata that goes with a user's profile - they're not an
"evidence of learning" in the same way blogs, files, forum posts are. They
can only be put into views in a couple of blocktypes right now and arguably
a bunch of them shouldn't be able to be put in views at all. For example,
why have "first name" and "last name" in the profileinfo blocktype? "Name"
makes more sense - but people know their name and can put it in anyway...
similar arguments apply for many of the other profile fields.

Upgrades to bundled third party libraries

Subpages