Actions

Difference between revisions of "Testing Area/Behat Testing"

From Mahara Wiki

< Testing Area
m (Useful Behat steps in Mahara)
 
(38 intermediate revisions by 4 users not shown)
Line 1: Line 1:
=== Setup ===
+
==Welcome to Behat testing with Mahara==
  
TODO...
+
https://docs.behat.org/en/latest/guides.html the behat home site!
 +
Behat testing is writing an automated test that tests the developers code by automatically, clicking around Mahara to check the devs submitted code doesn't break anything. This wiki section will teach you step by step how to write and submit your first Behat test.
  
=== How to run tests ===
+
Follow the steps below:
  
TODO...
+
# [[Testing_Area/Behat_Testing/Setup | Setting up your Mahara with Behat]]
 +
# [[Testing_Area/Behat_Testing/Running test |Running tests]]
 +
# Writing tests
 +
## [[Testing_Area/Behat_Testing/Basics |Basics]]
 +
## [[Testing_Area/Behat_Testing/Characteristics of a good test |Characteristics of a good test ]]
 +
## [[Testing_Area/Behat_Testing/Data set up | Data set up]]
 +
## [[Testing_Area/Behat_Testing/Steps |Steps]]
 +
## [[Testing_Area/Behat_Testing/Selectors and Elements |Selectors and elements ]]
 +
# [[Testing_Area/Behat_Testing/Error & Solutions |Errors and solutions]]
 +
# [[Testing_Area/Behat_Testing/Submitting tests |Submitting tests]]
 +
# [[Testing_Area/Behat_Testing/Jenkins Maintenance |Jenkins maintenance]]
  
=== How to write good tests ===
+
==Behat code snippets==
  
TODO...
+
If you want to get a human-readable(ish) list of all the existing Behat test scenarios:
  
=== Useful Behat steps in Mahara ===
+
<source lang="bash" enclose="div">
 +
cd /var/www/mahara
 +
find test/behat -name "*.feature" -exec grep -i -P "(Scenario)|(Feature)" {} \;
 +
</source>
  
These steps are specific to Mahara and are constantly changing. Behat testing is still new to Mahara and is in the early stages of development.
+
[[Category: Behat]]
 
 
Where ever it says "Selector" any of these forms of selectors can be used:
 
 
 
*dialogue
 
*block
 
*region
 
*table_row
 
*link
 
*button
 
*link_or_button
 
*select
 
*checkbox
 
*radio
 
*file
 
*filemanager
 
*optgroup
 
*option
 
*table
 
*field
 
*fieldset
 
*text
 
*css_element
 
*xpath_element
 
 
 
 
 
====Logging In====
 
 
 
Logs you in as Admin for testing purpose
 
  @Given I am logged in as the user "admin" with password "Password1"
 
 
 
====Opening/Loading/Closing Pages====
 
 
 
Checks, that current page PATH is equal to specified. eg; And I should be on "Profile" would be written as; And I should be on "artefact/internal/index.php"
 
  @Then I should be on "page path"
 
 
 
Opens user profile.
 
  @Given I am on user profile
 
 
 
Opens homepage.
 
  @When I go to the homepage
 
 
 
Opens Mahara homepage.
 
  @Given I am on homepage
 
 
 
   
 
Checks, that current page is the homepage.
 
  @Then I should be on the homepage
 
   
 
 
 
Reloads the current page.
 
  @Given I reload the page
 
 
 
Waits until the page is completely loaded. This step is auto-executed after every step.
 
  @Given I wait until the page is ready
 
   
 
Opens specified page.
 
  @Given I am on "page
 
 
 
Opens specified page.
 
  @When I go to "page"
 
 
 
Moves backward one page in history.
 
  @When I move backward one page
 
 
 
Moves forward one page in history
 
  @When I move forward one page
 
   
 
Checks, that current page response status is equal to specified.
 
  @Then the response status code should be 404
 
   
 
   
 
Checks, that current page response status is not equal to specified.
 
  @Then the response status code should not be 404
 
 
 
   
 
====Switching Frames/Windows====
 
 
 
 
 
 
 
 
 
Switches to the specified iframe.
 
  @Given I switch to "$iframename" iframe
 
 
 
 
 
Switches to the main Mahara frame.
 
  @Given I switch to the main frame
 
 
 
 
 
Switches to the specified window. Useful when interacting with popup windows.
 
  @Given I switch to “windowname” windows
 
 
 
 
 
Switches to the main Mahara window. Useful when you finish interacting with popup windows.
 
@Given I switch to the main window
 
 
 
 
====Actions Requiring Waiting====
 
 
 
   
 
Waits X seconds. Required after an action that requires data from an AJAX request.
 
  @Then I wait "$numberofseconds" seconds
 
 
 
 
 
 
Waits until the provided element selector exists in the DOM
 
  @Given I wait until "$element" "$selector" exists
 
 
 
   
 
Waits until the provided element does not exist in the DOM
 
  @Given I wait until "element_string" "selector_string" does not exist
 
 
 
 
 
====Click Actions====
 
 
 
Click on the specified element inside a table row containing the specified text.
 
  @Given I click on "element_string" "selector_string" in the "row_text_string" table row
 
 
 
 
 
@Given I drag "element_string" "selector1_string" and I drop it in "container_element_string" "selector2_string"
 
   
 
 
 
Generic mouse over action. Mouse over a element of the specified type.
 
  @When I hover "$element" "$selector"
 
   
 
 
 
 
 
Click on the element of the specified type which is located inside the second element.
 
  @When I click on "$element" "$selector"
 
 
 
 
 
 
Click on the specified element inside a table row containing the specified text.
 
  @Given I click on "$element" "$selector" in the table row
 
   
 
 
 
Clicks link with specified id|title|alt|text.
 
  @When I follow "id|title|alt|text"
 
 
 
 
 
Presses button with specified id|name|title|alt|value.
 
  @When I press "BUTTON_STRING"
 
 
 
Drags and drops the specified element to the specified container. This step does not work in all the browsers, consider it experimental.
 
  @Given I drag "$element" "$selector" and I drop it in "$containerelement" ”$selector"
 
 
 
 
 
====Filling in Forms====
 
 
 
   
 
Fills in form field with specified id|name|label|value.
 
  @When I fill in "field" with "value"
 
 
 
 
 
Fills in form field with specified id|name|label|value.
 
  @When I fill in "value" for "field"
 
   
 
   
 
Fills in form fields with provided table.
 
  @When I fill in the following:
 
   
 
 
 
Selects option in select field with specified id|name|label|value.
 
  @When I select "option" from "select"
 
   
 
   
 
Selects additional option in select field with specified id|name|label|value.
 
  @When I additionally select "option" from "select"
 
   
 
 
 
Checks checkbox with specified id|name|label|value.
 
  @When I check "option"
 
   
 
   
 
Unchecks checkbox with specified id|name|label|value.
 
  @When I uncheck "option"
 
   
 
 
 
Checks, that form field with specified id|name|label|value has specified value.
 
  @Then the "field" field should contain "value"
 
   
 
   
 
Checks, that form field with specified id|name|label|value doesn't have specified value.
 
  @Then the "field" field should not contain "value"
 
   
 
   
 
Checks, that checkbox with specified in|name|label|value is checked. 
 
  @Then the "checkbox" checkbox should be checked
 
   
 
 
 
Checks, that checkbox with specified in|name|label|value is checked.
 
  @Then the checkbox "checkbox" is|should be checked
 
   
 
 
 
Checks, that checkbox with specified in|name|label|value is unchecked.
 
  @Then the "checkbox" checkbox should not be checked
 
   
 
Checks, that checkbox with specified in|name|label|value is unchecked.
 
  @Then the checkbox "checkbox" should be unchecked|not be checked
 
   
 
   
 
Checks, that checkbox with specified in|name|label|value is unchecked.
 
  @Then the checkbox "checkbox" is unchecked|not checked)
 
 
 
Attaches file to field with specified id|name|label|value.
 
  @When I attach the file "File name" to "field"
 
 
 
 
 
Fills a form with field/value data.
 
  @Given I set the following fields to these values:
 
   
 
 
 
Sets the specified value to the field.
 
  @Given I set the field "field_string" to "field_value_string"
 
   
 
   
 
   
 
Sets the specified value to the field with xpath.   
 
  @Given I set the field with xpath "fieldxpath_string" to "field_value_string"
 
   
 
 
 
Checks, the field matches the value.
 
  @Then the field "field_string" matches value "field_value_string"
 
 
 
Checks, the field does not match the value.
 
  @Then the field "field_string" does not match value "field_value_string"
 
   
 
   
 
Checks, the provided field/value matches.
 
  @Then the following fields match these values:
 
 
 
 
 
Checks that the provided field/value pairs don't match. 
 
  @Then the following fields do not match these values:
 
   
 
 
 
Checks, that given select box contains the specified option.
 
  @Then the "select_string" select box should contain "option_string"
 
   
 
 
 
Checks, that given select box contains the specified option.
 
  @Then the "select_string" select box should not contain "option_string"
 
 
 
====Checking For Specfic Visible Elements====
 
 
 
Checks, that (?P<num>\d+) CSS elements exist on the page
 
  @Then I should see (?P<num>\d+) "element" elements
 
 
 
Checks, that the specified element is visible. Only available in tests using Javascript.
 
  @Then I should see the element "element" "selectortype"
 
 
 
Checks, that field with specified identifier exists on page.
 
  @Then I should see the field "field"
 
     
 
 
 
Checks, that current page PATH matches regular expression.
 
  @Then the url should match "pattern"
 
   
 
Prints current URL to console.
 
  @Then print current URL
 
   
 
 
 
Checks, that the specified element is visible. Only available in tests using Javascript.
 
  @Then "$elements" "$selector" should be visible
 
   
 
  @Then "$element" "$selector" should not be visible
 
 
 
 
 
Checks, that the specified element is visible inside the specified container. Only available in tests using Javascript.
 
  @Then "$element" "$selector" in the "$elementcontainer" "$selector" should be visible   
 
 
 
 
 
 
 
Checks, that the specified element is not visible inside the specified container. Only available in tests using Javascript.
 
  @Then "$element" "$selector" in the "$elementcontainer" "$textselector" should not be visible
 
         
 
 
 
Checks, that page contains specified text. It also checks if the text is visible when running Javascript tests.
 
  @Then I should see "$text"
 
   
 
 
 
Checks, that page doesn't contain specified text. When running Javascript tests it also considers that texts may be hidden.
 
  @Then I should not see "$text"
 
 
 
 
 
Checks, that the specified element contains the specified text. When running Javascript tests it also considers that texts may be hidden.
 
  @Then I should see "$text" in the "$element" “$sector”
 
 
 
 
Checks, that the specified element does not contain the specified text. When running Javascript tests it also considers that texts may be hidden.
 
  @Then I should not see "$text" in the “$element" "$selector”
 
 
 
 
 
 
 
Checks, that the first specified element appears before the second one.
 
  @Given "$element" "$selector" should appear before "$element" "$selector"
 
   
 
 
 
 
 
Checks, that the first specified element appears after the second one.
 
  @Given "$element" "selector" should appear after $"element"  $"selector"
 
 
 
 
 
Checks, that the first specified element appears after the second one.
 
  @Given "following_element_string" "selector1_string" should appear after "preceding_element_string" "selector2_string"
 
   
 
Checks, that element of specified type is disabled.
 
  @Then the "$element" "$selector" should be disabled
 
   
 
 
 
Checks, that element of specified type is enabled.
 
  @Then the "$element" "$selector" should be enabled
 
   
 
 
 
Checks the provided element and selector type are readonly on the current page.
 
  @Then the "$element" "$selector" should be readonly
 
 
 
 
Checks the provided element and selector type exists in the current page.
 
-This step is for advanced users, use it if you don't find anything else suitable for what you need.
 
  @Then "$element" "$selector" should exists
 
 
 
 
Checks that the provided element and selector type not exists in the current page.
 
-This step is for advanced users, use it if you don't find anything else suitable for what you need.
 
  @Then "$element" "$selector" should not exists
 
 
 
 
 
Checks that an element and selector type exists in another element and selector type on the current page.
 
-This step is for advanced users, use it if you don't find anything else suitable for what you need.
 
  @Then "$element" "$selector" should exist in the "$element" "$selector"
 
 
 
 
Checks, that page contains text matching specified pattern.
 
  @Then I should see text matching "pattern"
 
 
 
Checks, that page doesn't contain text matching specified pattern.
 
  @Then I should not see text matching "pattern"
 
   
 
   
 
Checks, that HTML response contains specified string.
 
  @Then the response should contain "text"
 
 
 
 
 
 
 
Checks, that HTML response doesn't contain specified string.
 
  @Then the response should not contain "text"
 
   
 
Checks, that element with specified CSS contains specified text.
 
  @Then I should see "text>" in the "element" element
 
 
 
 
 
 
 
Checks, that element with specified CSS doesn't contain specified text.
 
  @Then I should not see "text" in the "element" element
 
 
 
 
 
Checks, that element with specified CSS contains specified HTML.
 
  @Then the "element" element should contain "value"
 
   
 
 
 
 
 
Checks, that element with specified CSS doesn't contain specified HTML.
 
  @Then the "element" element should not contain "value"
 
 
 
 
 
Checks, that element with specified CSS exists on page.
 
  @Then I should see an? "element" element
 
   
 
Checks, that element with specified CSS doesn't exist on page.
 
  @Then I should not see an? "element" element
 
 
 
   
 
 
 
 
 
====Printing/Display====
 
 
 
 
 
Prints last response to console.
 
  @Then print last response
 
 
 
Prints current URL to console.
 
  @Then print current URL
 
 
 
 
 
 
 
 
 
====Ungrouped Steps====
 
 
 
Opens last response content in browser.   
 
  @Then show last response
 
   
 
Internal step definition to find exceptions, debugging() messages and PHP debug messages.
 
  @Given I look for exceptions
 
 
 
 
 
Follows the page redirection. Use this step after any action that shows a message and waits for a redirection
 
  @Given I wait to be redirected
 
   
 
 
 
Accepts the currently displayed alert dialog. This step does not work in all the browsers, consider it experimental.
 
  @Given I accept the currently displayed dialog
 
 
 
   
 
 
 
This step triggers cron like a user would do going to admin/cron.php.
 
  @Given I trigger cron
 

Latest revision as of 14:43, 7 August 2020

Welcome to Behat testing with Mahara

https://docs.behat.org/en/latest/guides.html the behat home site! Behat testing is writing an automated test that tests the developers code by automatically, clicking around Mahara to check the devs submitted code doesn't break anything. This wiki section will teach you step by step how to write and submit your first Behat test.

Follow the steps below:

  1. Setting up your Mahara with Behat
  2. Running tests
  3. Writing tests
    1. Basics
    2. Characteristics of a good test
    3. Data set up
    4. Steps
    5. Selectors and elements
  4. Errors and solutions
  5. Submitting tests
  6. Jenkins maintenance

Behat code snippets

If you want to get a human-readable(ish) list of all the existing Behat test scenarios:

cd /var/www/mahara
find test/behat -name "*.feature" -exec grep -i -P "(Scenario)|(Feature)" {} \;