Actions

Difference between revisions of "Testing/Behat Testing/Setup"

From Mahara Wiki

< Testing/Behat Testing
(Behat installation)
(Updating instructions to cover what happens if you have OpenJDK 9+ installed which won't work with Selenium)
 
(27 intermediate revisions by 6 users not shown)
Line 1: Line 1:
=How to install Mahara and Behat=
+
{{note|These instructions for the Behat setup are only for machines running Linux. They will not work for Windows machines.}}
  
'''Note''': These instructions for Mahara and Behat setup are only for machines running Linux. They will not work for Windows machines
+
1. Set up your [[Developer_Area/Developer_Environment | developer environment]] if you haven't already done so.
  
 +
2. Install Behat's dependencies:
 +
<source lang="bash" enclose="div">
 +
$ sudo apt-get install curl openjdk-8-jre-headless
 +
</source>
  
 +
3. Add the following config settings to the bottom of your Mahara config.php file inside the htdocs/ subdirectory of the Mahara codebase.
  
== For dev's instructions to install a Mahara ==
+
{{note|If /var/www/maharadata is not where you created your dataroot, you'll need to replace that with the correct path in the following commands}}
Please use [[System_Administrator%27s_Guide/Installing_Mahara]].
 
  
 +
<source lang="bash" enclose="div">
 +
// Behat config
 +
$cfg->behat_dbprefix = 'behat_'; // must not empty
 +
$cfg->behat_dataroot = "/var/lib/maharadata/master_behat"; // Behat's copy of maharadata
 +
$cfg->behat_wwwroot = 'http://localhost:8000'; // Must be this
 +
$cfg->behat_selenium2 = "http://127.0.0.1:4444/wd/hub"; // Must be this
 +
</source>
  
 +
{{note|There should be a behat_dataroot for each Mahara instance that you set up. Here the one for the master branch / this Mahara instance is created.}}
 +
{{note|If the directory for $cfg->behat_dataroot does not exist then it will need to be created.}}
  
== Otherwise, if you do not have Mahara installed ==
+
4. Make your Behat data directory (check this matches what you set in config.php):
 +
<source lang="bash" enclose="div">
 +
$ sudo mkdir /var/lib/maharadata/master_behat
 +
</source>
  
{{note|If you do have Mahara already installed, skip to [[#Behat_installation]]}}
+
5. Make the directory and any subdirectories writeable by Mahara (check this is correct):
 +
<source lang="bash" enclose="div">
 +
$ sudo chmod 777 -R /var/lib/maharadata/master_behat
 +
</source>
  
 +
6. Change the directory and any subdirectories permissions to be owned by apache user (check this is correct):
 +
<source lang="bash" enclose="div">
 +
$ sudo chown -R www-data.www-data /var/lib/maharadata/master_behat
 +
</source>
  
 +
* For Ubuntu, apache runs with user www-data
 +
* For Centos, apache runs with user apache
  
Install Apache, PHP5, Database, Git:
+
7. Run Behat tests (change into your Mahara code directory first) as the apache user:
sudo apt-get install php5 php5-cli php5-pgsql php5-xmlrpc php5-intl php5-curl php5-gd php5-xdebug php5-dbg postgresql apache2 git
+
<source lang="bash" enclose="div">
 +
$ cd
 +
$ cd code/mahara
 +
./test/behat/mahara_behat.sh run
 +
</source>
  
Restart Apache
+
Or if you have sudo access:
sudo service apache2 restart
+
<source lang="bash" enclose="div">
 +
$ cd
 +
$ cd code/mahara
 +
sudo -u www-data ./test/behat/mahara_behat.sh run
 +
</source>
  
Make a Mahara directory in the same place as Moodle
 
sudo mkdir /var/www/html/mahara
 
  
Then change into the Mahara directory
+
For the first time of running Behat, you need to wait for the Behat environment initialisation. This can take a while.
cd /var/www/html/mahara
 
  
Then change the permission (insert the username of the machine into the username. Don't use the word username)
+
To run all tests:
  sudo chown username:username /var/www/html/mahara
+
  ./test/behat/mahara_behat.sh run
  
Get Mahara clone.
+
To run your specific tests marked with @yourtags:
  git clone https://github.com/MaharaProject/mahara.git .
+
  ./test/behat/mahara_behat.sh run @yourtags
  
Checkout the master(15.04) branch
+
To run a particular feature file:
  git checkout master
+
  ./test/behat/mahara_behat.sh run my_file.feature
  
Note: if this step doesn't work. You may have forgotten the fullstop off the end of the git clone command. It created a mahara directory inside the other mahara directory. Go in to the file explorer and delete the second instance.
+
The run / runheadless / rundebug / rundebugheadless / runfresh are interchangable with run in above commands
  
Become Postgres user:
+
[[Category:Behat]]
sudo su postgres
 
  
Open psql. After some preamble you should see the prompt postgres=#.
+
== Running Behat tests with html report and screenshots on failed steps ==
psql
 
  
You should now see postgres=# ready and awaiting your command.
+
To run tests with html output, add the flag 'html' to your test run command, e.g:
  
Create a user and don't use the PW that you use for your workstation:
+
<source>./test/behat/mahara_behat.sh run create_page.feature html</source>
CREATE USER maharauser WITH PASSWORD 'yourpassword';
 
 
 
As you previously did, you should see postgres=#
 
You are now going to create a database:
 
CREATE DATABASE mahara WITH OWNER maharauser;
 
  
Exit to leave psql by pressing Ctrl-D
+
This will automatically open an html report of the test run in your browser. If there were failed steps, the the html report plugin takes a screenshot of the page that failed.
 +
You can find the screenshots in your behat dataroot (usually /var/lib/maharadata/master_behat) at the following path:
 +
behat/html_results/<feature title>/<scenario title>.png
  
Exit back to your own user:
+
Hopefully we will rig up the report to link to the screenshot soon.
exit
 
  
Make a data directory:
+
== How to Run Mahara behat test on Chrome ==
sudo mkdir /var/www/maharadata
 
  
Make directory writeable by mahara:  
+
{{note|Mahara behat is running on chrome driver by default in 17:04+}}
sudo chmod 777 /var/www/maharadata
 
  
 +
This is steps for older versions of Mahara
  
In the code below make sure you change <your email address> to your actual work email address, without the <> signs.
+
1. Download chromedriver here http://www.seleniumhq.org/download/ ( download the latest release)
  
<pre>
+
2. Start selenium Server with chrome driver
<?php
 
$cfg = new stdClass();
 
  
$branch = 'master';
+
$ java -jar selenium-server-standalone-2.53.1.jar -Dwebdriver.chrome.driver=path of chromedriver
  
// database connection details
+
3. Change the behat.yml file as follows,
// valid values for dbtype are 'postgres8' and 'mysql5'
 
$cfg->dbtype  = 'postgres';
 
$cfg->dbhost  = 'localhost';
 
$cfg->dbuser  = 'maharauser';
 
$cfg->dbname  = "mahara";
 
$cfg->dbpass  = 'password';
 
  
$cfg->dataroot = "/var/www/maharadata";
+
Here is the file path var/lib/maharadata/master_behat/behat.yml
  
$cfg->sendemail = true;
+
when you open the beaht.yml file replace the following code with the lines that contain sessions to goutte=NUll( Niranjan or Lovesh can exactly tell you where to replcae the code if it is confusing) and save it
$cfg->sendallemailto = '<your email address>';
 
  
$cfg->productionmode = false;
+
javascript_session: selenium2
$cfg->perftofoot = true;
+
        selenium2:
</pre>
+
          browser: chrome
 +
            goutte: ~
  
 +
4. Run the behat test without rundebug
  
An example config.php should appear:
+
for example ./test/behat/mahara_behat.sh run example.feature
  
* Copy all of the code above
+
== Debugging ==
* open text editor from start menu
 
* paste code
 
* save file in the directory: /var/www/html/mahara/htdocs/ and call it config.php
 
* save
 
  
 +
In some cases, depending on the version of Ubuntu you have, the wrong version of OpenJDK will be installed. Selenium only runs with OpenJDK 8, but Ubuntu Bionic tends to install OpenJDK 11 (especially if 'default-jre' was installed)
  
 +
If you run into issues where the script says it has started Selenium, but that Behat throws an exception because it thinks Selenium has not started, check the JRE installed:
  
===Behat installation===
+
<source>java -version</source>
  
Install Behat's dependencies:
+
If it says you're using OpenJDK 10 or 11 (which can ship with Ubuntu 18.04 depending on which service pack is used), this needs to be uninstalled first:
sudo apt-get install curl openjdk-7-jre-headless
 
  
Add these config settings to the bottom of your Mahara config.php file, inside the htdocs/ subdirectory of the Mahara codebase.
+
<source>sudo apt-get remove openjdk*</source>
 
 
{{note|If /var/www/maharadata is not where you created your dataroot, you'll need to replace that with the correct path in the following commands}}
 
  
<pre>
+
Once this is removed, you will need to re-run the OpenJDK 8 installation:
// Behat config
 
$cfg->dbprefix = ''; // Behat complains without dbprefix being set
 
$cfg->wwwroot = 'http://your.mahara.domain/';  // Add this if you do not have wwwroot set already, Behat complains with out it
 
$cfg->behat_dbprefix = 'behat_'; // must not empty
 
$cfg->behat_dataroot = "/var/lib/maharadata/behat"; // Behat's copy of mahara data
 
$cfg->behat_wwwroot = 'http://localhost:8000'; // Must be this
 
</pre>
 
  
Make your data directory (check this matches what you set in config.php):
+
<source lang="bash" enclose="div">
sudo mkdir /var/lib/maharadata/behat
+
$ sudo apt-get install openjdk-8-jre-headless
 
+
</source>
Make directory writeable by mahara (check this is correct):
 
sudo chmod 777 /var/lib/maharadata/behat
 
 
 
Run Behat tests (change into your Mahara code directory first)
 
./test/behat/mahara_behat.sh run
 
 
 
For the first time of running behat, you need to wait for the Behat environment initialisation. This can take a while.
 
 
 
To run all tests
 
./test/behat/mahara_behat.sh run
 
 
 
To run your specific tests:
 
./test/behat/mahara_behat.sh run @yourtags
 
 
 
 
 
[[Category:Behat]]
 

Latest revision as of 06:15, 7 August 2019

Note: These instructions for the Behat setup are only for machines running Linux. They will not work for Windows machines.

1. Set up your developer environment if you haven't already done so.

2. Install Behat's dependencies:

$ sudo apt-get install curl openjdk-8-jre-headless

3. Add the following config settings to the bottom of your Mahara config.php file inside the htdocs/ subdirectory of the Mahara codebase.

Note: If /var/www/maharadata is not where you created your dataroot, you'll need to replace that with the correct path in the following commands
// Behat config
$cfg->behat_dbprefix = 'behat_'; // must not empty
$cfg->behat_dataroot = "/var/lib/maharadata/master_behat"; // Behat's copy of maharadata
$cfg->behat_wwwroot = 'http://localhost:8000'; // Must be this
$cfg->behat_selenium2 = "http://127.0.0.1:4444/wd/hub"; // Must be this
Note: There should be a behat_dataroot for each Mahara instance that you set up. Here the one for the master branch / this Mahara instance is created.
Note: If the directory for $cfg->behat_dataroot does not exist then it will need to be created.

4. Make your Behat data directory (check this matches what you set in config.php):

$ sudo mkdir /var/lib/maharadata/master_behat

5. Make the directory and any subdirectories writeable by Mahara (check this is correct):

$ sudo chmod 777 -R /var/lib/maharadata/master_behat

6. Change the directory and any subdirectories permissions to be owned by apache user (check this is correct):

$ sudo chown -R www-data.www-data /var/lib/maharadata/master_behat
  • For Ubuntu, apache runs with user www-data
  • For Centos, apache runs with user apache

7. Run Behat tests (change into your Mahara code directory first) as the apache user:

$ cd
$ cd code/mahara
./test/behat/mahara_behat.sh run

Or if you have sudo access:

$ cd
$ cd code/mahara
sudo -u www-data ./test/behat/mahara_behat.sh run


For the first time of running Behat, you need to wait for the Behat environment initialisation. This can take a while.

To run all tests:

./test/behat/mahara_behat.sh run

To run your specific tests marked with @yourtags:

./test/behat/mahara_behat.sh run @yourtags

To run a particular feature file:

./test/behat/mahara_behat.sh run my_file.feature

The run / runheadless / rundebug / rundebugheadless / runfresh are interchangable with run in above commands

Running Behat tests with html report and screenshots on failed steps

To run tests with html output, add the flag 'html' to your test run command, e.g:

./test/behat/mahara_behat.sh run create_page.feature html

This will automatically open an html report of the test run in your browser. If there were failed steps, the the html report plugin takes a screenshot of the page that failed. You can find the screenshots in your behat dataroot (usually /var/lib/maharadata/master_behat) at the following path: behat/html_results/<feature title>/<scenario title>.png

Hopefully we will rig up the report to link to the screenshot soon.

How to Run Mahara behat test on Chrome

Note: Mahara behat is running on chrome driver by default in 17:04+

This is steps for older versions of Mahara

1. Download chromedriver here http://www.seleniumhq.org/download/ ( download the latest release)

2. Start selenium Server with chrome driver

$ java -jar selenium-server-standalone-2.53.1.jar -Dwebdriver.chrome.driver=path of chromedriver

3. Change the behat.yml file as follows,

Here is the file path var/lib/maharadata/master_behat/behat.yml

when you open the beaht.yml file replace the following code with the lines that contain sessions to goutte=NUll( Niranjan or Lovesh can exactly tell you where to replcae the code if it is confusing) and save it

javascript_session: selenium2

        selenium2:
          browser: chrome
           goutte: ~

4. Run the behat test without rundebug

for example ./test/behat/mahara_behat.sh run example.feature

Debugging

In some cases, depending on the version of Ubuntu you have, the wrong version of OpenJDK will be installed. Selenium only runs with OpenJDK 8, but Ubuntu Bionic tends to install OpenJDK 11 (especially if 'default-jre' was installed)

If you run into issues where the script says it has started Selenium, but that Behat throws an exception because it thinks Selenium has not started, check the JRE installed:

java -version

If it says you're using OpenJDK 10 or 11 (which can ship with Ubuntu 18.04 depending on which service pack is used), this needs to be uninstalled first:

sudo apt-get remove openjdk*

Once this is removed, you will need to re-run the OpenJDK 8 installation:

$ sudo apt-get install openjdk-8-jre-headless