Actions

Difference between revisions of "Testing Area/Behat Testing/Error & Solutions"

From Mahara Wiki

< Testing Area‎ | Behat Testing
m
m
Line 7: Line 7:
  
  
Selenium upgrades causing firefox to try run headless and not open firefox. It can also throw an error when you try run the test. It will mention something about the Selenium server not being able to start up.
+
When Selenium upgrades causing firefox to try run headless and not open firefox. It can also throw an error when you try run the test. It will mention something about the Selenium server not being able to start up.
  
 
TO FIX:  
 
TO FIX:  
  
Go to the mahara_sh script. Currently located in mahara/test/behat and update the version. I can access that in the file explorer and then I just change the version number and then it auto updates when I run the ./test/behat/mahara_behat.sh rundebug @tag
+
Go to the mahara_sh script. Currently located in mahara/test/behat and update the version. You can access that via the file explorer and then I just change the version number and then it auto updates when I run the ./test/behat/mahara_behat.sh rundebug @tag
  
 
To find out if Selenium has updated to a new version, go to http://www.seleniumhq.org/download/ and see version of the Selenium Server is available.
 
To find out if Selenium has updated to a new version, go to http://www.seleniumhq.org/download/ and see version of the Selenium Server is available.
Line 35: Line 35:
 
Eg:
 
Eg:
  
<username>@boris:~$ ps aux | grep selenium<br>
+
<username>@<computername>:~$ ps aux | grep selenium<br>
 
<username> 6590  0.0  0.0  12960  936 pts/26  S+  10:31  0:00 grep --color=auto selenium
 
<username> 6590  0.0  0.0  12960  936 pts/26  S+  10:31  0:00 grep --color=auto selenium
 
   
 
   
Line 41: Line 41:
 
   kill 6590
 
   kill 6590
  
However don't kill the one that say's --color=auto selenium kill the other one.
+
However don't kill the one that say's --color=auto selenium you should kill the other one.
  
The run the command again to make sure that the server was killed
+
Then run the command again to make sure that the server was killed
 
   ps aux | grep selenium  
 
   ps aux | grep selenium  
  
Line 51: Line 51:
 
Once you have done that, if you are a dev take a look into the mahara_behat.sh script because there will be something up with the way that selenium is shutting down. Also check that this issue isn't happening with Jenkins. However, keep in mind that Jenkins runs headless.
 
Once you have done that, if you are a dev take a look into the mahara_behat.sh script because there will be something up with the way that selenium is shutting down. Also check that this issue isn't happening with Jenkins. However, keep in mind that Jenkins runs headless.
  
If you aren't a dev then go tell one of the devs that it's not shutting down properly. You can do this by writing a bug on Launchpad here: https://bugs.launchpad.net/mahara/+filebug
+
If you aren't a dev then go tell one of the devs that it's not shutting down properly. Or you can do this by writing a bug on Launchpad here: https://bugs.launchpad.net/mahara/+filebug
 
In the summary write Behat: tell them what the issue is here.  
 
In the summary write Behat: tell them what the issue is here.  
Just they can tell instantly it's a Behat issue you are having.
+
Just so they can tell instantly it's a Behat issue you are having.
  
  
Line 67: Line 67:
 
If all else fails then revert back to the previous stable version.   
 
If all else fails then revert back to the previous stable version.   
  
I change directory to downloads and run this in a separate terminal. This requires you to have the previous version of Firefox downloaded.
+
I change directory to downloads and run this in a separate terminal. This requires you to have the previous version of Firefox downloaded and still sitting in downloads.
 
java -jar selenium-server-standalone-2.44.0.jar -Dwebdriver.firefox.bin=/home/jinelleb/Downloads/firefox/firefox
 
java -jar selenium-server-standalone-2.44.0.jar -Dwebdriver.firefox.bin=/home/jinelleb/Downloads/firefox/firefox
  
Line 75: Line 75:
 
Same as the Selenium server issues, you can get errors with the PHP server not closing or running multiple servers. You can tell this because it will give a error message when you try run the tests. It will generally throw an error with the key words "localhost 8000" or "PHP server".
 
Same as the Selenium server issues, you can get errors with the PHP server not closing or running multiple servers. You can tell this because it will give a error message when you try run the tests. It will generally throw an error with the key words "localhost 8000" or "PHP server".
  
There could be multiple things causing this issue. Inc the config file not point to the right host name.
+
There could be multiple things causing this issue including the config file not point to the right host name.
  
  

Revision as of 12:30, 30 March 2015

Errors when using Behat


This is a place where you can log errors and come to fix yours. If you log an error please be sure to put down how you fixed it

Selenium issues


When Selenium upgrades causing firefox to try run headless and not open firefox. It can also throw an error when you try run the test. It will mention something about the Selenium server not being able to start up.

TO FIX:

Go to the mahara_sh script. Currently located in mahara/test/behat and update the version. You can access that via the file explorer and then I just change the version number and then it auto updates when I run the ./test/behat/mahara_behat.sh rundebug @tag

To find out if Selenium has updated to a new version, go to http://www.seleniumhq.org/download/ and see version of the Selenium Server is available.

You might need to do some more debugging. But this information should point you in the right direction.


Another bug with Selenium is if you are using Moodle or running Behat from two diff versions of Mahara is sometimes the Selenium Server doesn't shut down properly. You will know this is happening if you've run the tests and it says at the end "Selenium server not not shut down" or it just wont say that's it has shut down. It will be written the same way it does when Selenium starts up when you run the tests.

Eg: Composer installing... PHP server running, Selenium running. At the end of the tests running it says the same thing.. PHP server shut down... Selenium shut down.

When you try to run the tests it may also say "Selenium server is not running or could not be run" or something similar. This is because there are 2 running and it's getting confused


TO FIX:

Temporary fix, go to the terminal and type

ps aux | grep selenium

And then kill the process ID. This is the first number after your username.

Eg:

<username>@<computername>:~$ ps aux | grep selenium
<username> 6590 0.0 0.0 12960 936 pts/26 S+ 10:31 0:00 grep --color=auto selenium

So the command will be

 kill 6590

However don't kill the one that say's --color=auto selenium you should kill the other one.

Then run the command again to make sure that the server was killed

 ps aux | grep selenium 


This will just kill selenium running so you can finish what you are working on and run the test properly.

Once you have done that, if you are a dev take a look into the mahara_behat.sh script because there will be something up with the way that selenium is shutting down. Also check that this issue isn't happening with Jenkins. However, keep in mind that Jenkins runs headless.

If you aren't a dev then go tell one of the devs that it's not shutting down properly. Or you can do this by writing a bug on Launchpad here: https://bugs.launchpad.net/mahara/+filebug In the summary write Behat: tell them what the issue is here. Just so they can tell instantly it's a Behat issue you are having.


Firefox issues


Firefox upgrades and when you run the tests Firefox wont pop up with the Mahara instance in it. Another issue is the window opening but Behat not being able to run the test properly and it will say that you cant log in. It will run the first log in steps (say as an admin) but it wont pass the next step (to verify that the admin can be logged in) and if you actually look at Behat running (requires you to not be running it headless) then you will see it's not actually logging in.

TO FIX:

Generally Firefox and Selenium are heavily entwined. When Firefox breaks, go to http://www.seleniumhq.org/download/ and download the newer version of Selenium. When one upgarades it's usually best to upgrade the other at the same time so they are compatible.

If all else fails then revert back to the previous stable version.

I change directory to downloads and run this in a separate terminal. This requires you to have the previous version of Firefox downloaded and still sitting in downloads. java -jar selenium-server-standalone-2.44.0.jar -Dwebdriver.firefox.bin=/home/jinelleb/Downloads/firefox/firefox


PHP server issues

Same as the Selenium server issues, you can get errors with the PHP server not closing or running multiple servers. You can tell this because it will give a error message when you try run the tests. It will generally throw an error with the key words "localhost 8000" or "PHP server".

There could be multiple things causing this issue including the config file not point to the right host name.


TO FIX:

Grep search for php server and kill the process ID.

Go to your mahara config.php and read the Behat section and make sure that everything is pointing to the right place.

If these fail then if you aren't a dev file a bug here https://bugs.launchpad.net/mahara/+filebug with the summary Behat: description of what's happening when you try run tests.

If you are a dev then you need to look into the mahara_behat script and check the php server is closing properly. A way to debug is to run a test locally and check if the php server is starting up and shutting down properly. It will say in the terminal if you run "rundebug". I think it runs with just run as well.

Composer issue

Originally composer wasn't pegged. Now it has been you can often get a yellow error when you run Behat that mentions composer dependencies being out of date. It will say "Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies".


TO FIX:

Run update to update them. Or just delete the lock file and then run Behat again. The composer lock file is located under mahara/external it's called composerlock.php. The dependencies should update automatically when you run.


Debugg steps that are failing

When you are trying to write a test but the test keeps failing on a step and you don't know why it's failing. You need to be able to find a way to hold the window open so you can see what's happening.

TO FIX:

For a start when you run the test run using the debug option instead of just "run" this will give you a more human readable output. The step that is failing will go red. EG:

./test/behat/mahara_behat.sh rundebug @youtagthatyouwanttorun

To hold the window open so you can add a step right before the one breaking saying And I wait "30" seconds. Another way of holding the window open for a longer period of time is to click on the terminal in the and hit ctrl+z this will pause the test from running right where it is. You then need to type fg to bring the test back into the foreground and let it keep running till the end.

Stopping a test

A test has started running and you need to stop it in the middle of it.

TO FIX:

Press ctrl+c this will stop it from running