Actions

Difference between revisions of "Testing Area/Behat Testing/Setup"

From Mahara Wiki

< Testing Area‎ | Behat Testing
(Updating instructions to cover what happens if you have OpenJDK 9+ installed which won't work with Selenium)
 
(10 intermediate revisions by 5 users not shown)
Line 4: Line 4:
  
 
2. Install Behat's dependencies:
 
2. Install Behat's dependencies:
<source lang="bash" enclose="div">
+
<syntaxhighlight lang="bash">
 
$ sudo apt-get install curl openjdk-8-jre-headless
 
$ sudo apt-get install curl openjdk-8-jre-headless
</source>
+
</syntaxhighlight>
  
 
3. Add the following config settings to the bottom of your Mahara config.php file inside the htdocs/ subdirectory of the Mahara codebase.  
 
3. Add the following config settings to the bottom of your Mahara config.php file inside the htdocs/ subdirectory of the Mahara codebase.  
Line 12: Line 12:
 
{{note|If /var/www/maharadata is not where you created your dataroot, you'll need to replace that with the correct path in the following commands}}
 
{{note|If /var/www/maharadata is not where you created your dataroot, you'll need to replace that with the correct path in the following commands}}
  
<source lang="bash" enclose="div">
+
<syntaxhighlight lang="bash">
 
// Behat config
 
// Behat config
 +
$cfg->behat_test = true; // Turn this off when you are not behat testing
 
$cfg->behat_dbprefix = 'behat_'; // must not empty
 
$cfg->behat_dbprefix = 'behat_'; // must not empty
$cfg->behat_dataroot = "/var/lib/maharadata/master_behat"; // Behat's copy of maharadata
+
$cfg->behat_dataroot = "/var/lib/maharadata/main_behat"; // The Behat copy of maharadata.
 
$cfg->behat_wwwroot = 'http://localhost:8000'; // Must be this
 
$cfg->behat_wwwroot = 'http://localhost:8000'; // Must be this
 
$cfg->behat_selenium2 = "http://127.0.0.1:4444/wd/hub"; // Must be this
 
$cfg->behat_selenium2 = "http://127.0.0.1:4444/wd/hub"; // Must be this
</source>
 
  
{{note|There should be a behat_dataroot for each Mahara instance that you set up. Here the one for the master branch / this Mahara instance is created.}}
+
// If you want failed step screenshots appear directly on screen while the step fails (only Ubuntu)
 +
$cfg->behat_view_screenshots = true;
 +
</syntaxhighlight>
 +
 
 +
{{note|There should be a behat_dataroot for each Mahara instance that you set up. Here the one for the main branch / this Mahara instance is created.}}
 
{{note|If the directory for $cfg->behat_dataroot does not exist then it will need to be created.}}
 
{{note|If the directory for $cfg->behat_dataroot does not exist then it will need to be created.}}
  
 
4. Make your Behat data directory (check this matches what you set in config.php):
 
4. Make your Behat data directory (check this matches what you set in config.php):
<source lang="bash" enclose="div">
+
<syntaxhighlight lang="bash">
$ sudo mkdir /var/lib/maharadata/master_behat
+
$ sudo mkdir /var/lib/maharadata/main_behat
</source>
+
</syntaxhighlight>
  
5. Make the directory and any subdirectories writeable by Mahara (check this is correct):  
+
5. Make the directory and any subdirectories writeable by Mahara (check the directory is correct):  
<source lang="bash" enclose="div">
+
<syntaxhighlight lang="bash">
$ sudo chmod 777 -R /var/lib/maharadata/master_behat
+
$ sudo chmod 777 -R /var/lib/maharadata/main_behat
</source>
+
</syntaxhighlight>
 +
<span id="chown"></span>
 +
6. Change the directory and any subdirectories permissions to be owned by apache user (check the directory is correct):
 +
<syntaxhighlight lang="bash">
 +
$ sudo chown -R www-data:www-data /var/lib/maharadata/main_behat
 +
</syntaxhighlight>
  
6. Change the directory and any subdirectories permissions to be owned by apache user (check this is correct):
+
*For Ubuntu, apache runs with user www-data
<source lang="bash" enclose="div">
+
*For Centos, apache runs with user apache
$ sudo chown -R www-data.www-data /var/lib/maharadata/master_behat
 
</source>
 
 
 
* For Ubuntu, apache runs with user www-data
 
* For Centos, apache runs with user apache
 
  
 
7. Run Behat tests (change into your Mahara code directory first) as the apache user:
 
7. Run Behat tests (change into your Mahara code directory first) as the apache user:
<source lang="bash" enclose="div">
+
<syntaxhighlight lang="bash">
 
$ cd
 
$ cd
 
$ cd code/mahara
 
$ cd code/mahara
 
./test/behat/mahara_behat.sh run
 
./test/behat/mahara_behat.sh run
</source>
+
</syntaxhighlight>
  
 
Or if you have sudo access:
 
Or if you have sudo access:
<source lang="bash" enclose="div">
+
<syntaxhighlight lang="bash">
 
$ cd
 
$ cd
 
$ cd code/mahara
 
$ cd code/mahara
 
sudo -u www-data ./test/behat/mahara_behat.sh run
 
sudo -u www-data ./test/behat/mahara_behat.sh run
</source>
+
</syntaxhighlight>
  
  
Line 71: Line 75:
 
[[Category:Behat]]
 
[[Category:Behat]]
  
== Running Behat tests with html report and screenshots on failed steps ==
+
==Running Behat tests with html report and screenshots on failed steps==
  
 
To run tests with html output, add the flag 'html' to your test run command, e.g:
 
To run tests with html output, add the flag 'html' to your test run command, e.g:
  
<source>./test/behat/mahara_behat.sh run create_page.feature html</source>
+
<syntaxhighlight lang="bash">./test/behat/mahara_behat.sh run create_page.feature html</syntaxhighlight>
  
 
This will automatically open an html report of the test run in your browser. If there were failed steps, the the html report plugin takes a screenshot of the page that failed.
 
This will automatically open an html report of the test run in your browser. If there were failed steps, the the html report plugin takes a screenshot of the page that failed.
You can find the screenshots in your behat dataroot (usually /var/lib/maharadata/master_behat) at the following path:
+
You can find the screenshots in your behat dataroot (usually /var/lib/maharadata/main_behat) at the following path:
 
behat/html_results/<feature title>/<scenario title>.png
 
behat/html_results/<feature title>/<scenario title>.png
 +
 +
Ubuntu only (you need the image viewer '[https://packages.ubuntu.com/search?keywords=eog eog]'): If you set
 +
 +
$cfg->behat_view_screenshots = true;
 +
 +
in your config.php file then the images will be opened up in image browser.
  
 
Hopefully we will rig up the report to link to the screenshot soon.
 
Hopefully we will rig up the report to link to the screenshot soon.
  
== How to Run Mahara behat test on Chrome ==
+
==How to Run Mahara behat test on Chrome for versions older than 17.04==
  
 
{{note|Mahara behat is running on chrome driver by default in 17:04+}}
 
{{note|Mahara behat is running on chrome driver by default in 17:04+}}
Line 97: Line 107:
 
3. Change the behat.yml file as follows,
 
3. Change the behat.yml file as follows,
  
Here is the file path var/lib/maharadata/master_behat/behat.yml
+
Here is the file path var/lib/maharadata/main_behat/behat.yml
  
when you open the beaht.yml file replace the following code with the lines that contain sessions to goutte=NUll( Niranjan or Lovesh can exactly tell you where to replcae the code if it is confusing) and save it
+
when you open the beaht.yml file replace the following code with the lines that contain sessions to go
  
 
javascript_session: selenium2
 
javascript_session: selenium2
Line 110: Line 120:
 
for example  ./test/behat/mahara_behat.sh run example.feature
 
for example  ./test/behat/mahara_behat.sh run example.feature
  
== Debugging ==
+
==Debugging==
 
 
In some cases, depending on the version of Ubuntu you have, the wrong version of OpenJDK will be installed. Selenium only runs with OpenJDK 8, but Ubuntu Bionic tends to install OpenJDK 11 (especially if 'default-jre' was installed)
 
 
 
If you run into issues where the script says it has started Selenium, but that Behat throws an exception because it thinks Selenium has not started, check the JRE installed:
 
 
 
<source>java -version</source>
 
 
 
If it says you're using OpenJDK 10 or 11 (which can ship with Ubuntu 18.04 depending on which service pack is used), this needs to be uninstalled first:
 
 
 
<source>sudo apt-get remove openjdk*</source>
 
 
 
Once this is removed, you will need to re-run the OpenJDK 8 installation:
 
  
<source lang="bash" enclose="div">
+
Go here for tips: https://wiki.mahara.org/wiki/Testing/Behat_Testing/Error_%26_Solutions
$ sudo apt-get install openjdk-8-jre-headless
 
</source>
 

Latest revision as of 12:21, 15 June 2023

Note: These instructions for the Behat setup are only for machines running Linux. They will not work for Windows machines.

1. Set up your developer environment if you haven't already done so.

2. Install Behat's dependencies:

$ sudo apt-get install curl openjdk-8-jre-headless

3. Add the following config settings to the bottom of your Mahara config.php file inside the htdocs/ subdirectory of the Mahara codebase.

Note: If /var/www/maharadata is not where you created your dataroot, you'll need to replace that with the correct path in the following commands
// Behat config
$cfg->behat_test = true; // Turn this off when you are not behat testing
$cfg->behat_dbprefix = 'behat_'; // must not empty
$cfg->behat_dataroot = "/var/lib/maharadata/main_behat"; // The Behat copy of maharadata.
$cfg->behat_wwwroot = 'http://localhost:8000'; // Must be this
$cfg->behat_selenium2 = "http://127.0.0.1:4444/wd/hub"; // Must be this

// If you want failed step screenshots appear directly on screen while the step fails (only Ubuntu)
$cfg->behat_view_screenshots = true;
Note: There should be a behat_dataroot for each Mahara instance that you set up. Here the one for the main branch / this Mahara instance is created.
Note: If the directory for $cfg->behat_dataroot does not exist then it will need to be created.

4. Make your Behat data directory (check this matches what you set in config.php):

$ sudo mkdir /var/lib/maharadata/main_behat

5. Make the directory and any subdirectories writeable by Mahara (check the directory is correct):

$ sudo chmod 777 -R /var/lib/maharadata/main_behat

6. Change the directory and any subdirectories permissions to be owned by apache user (check the directory is correct):

$ sudo chown -R www-data:www-data /var/lib/maharadata/main_behat
  • For Ubuntu, apache runs with user www-data
  • For Centos, apache runs with user apache

7. Run Behat tests (change into your Mahara code directory first) as the apache user:

$ cd
$ cd code/mahara
./test/behat/mahara_behat.sh run

Or if you have sudo access:

$ cd
$ cd code/mahara
sudo -u www-data ./test/behat/mahara_behat.sh run


For the first time of running Behat, you need to wait for the Behat environment initialisation. This can take a while.

To run all tests:

./test/behat/mahara_behat.sh run

To run your specific tests marked with @yourtags:

./test/behat/mahara_behat.sh run @yourtags

To run a particular feature file:

./test/behat/mahara_behat.sh run my_file.feature

The run / runheadless / rundebug / rundebugheadless / runfresh are interchangable with run in above commands

Running Behat tests with html report and screenshots on failed steps

To run tests with html output, add the flag 'html' to your test run command, e.g:

./test/behat/mahara_behat.sh run create_page.feature html

This will automatically open an html report of the test run in your browser. If there were failed steps, the the html report plugin takes a screenshot of the page that failed. You can find the screenshots in your behat dataroot (usually /var/lib/maharadata/main_behat) at the following path: behat/html_results/<feature title>/<scenario title>.png

Ubuntu only (you need the image viewer 'eog'): If you set

$cfg->behat_view_screenshots = true;

in your config.php file then the images will be opened up in image browser.

Hopefully we will rig up the report to link to the screenshot soon.

How to Run Mahara behat test on Chrome for versions older than 17.04

Note: Mahara behat is running on chrome driver by default in 17:04+

This is steps for older versions of Mahara

1. Download chromedriver here http://www.seleniumhq.org/download/ ( download the latest release)

2. Start selenium Server with chrome driver

$ java -jar selenium-server-standalone-2.53.1.jar -Dwebdriver.chrome.driver=path of chromedriver

3. Change the behat.yml file as follows,

Here is the file path var/lib/maharadata/main_behat/behat.yml

when you open the beaht.yml file replace the following code with the lines that contain sessions to go

javascript_session: selenium2

        selenium2:
          browser: chrome
           goutte: ~

4. Run the behat test without rundebug

for example ./test/behat/mahara_behat.sh run example.feature

Debugging

Go here for tips: https://wiki.mahara.org/wiki/Testing/Behat_Testing/Error_%26_Solutions