Actions

Testing Area/Visual regression testing: Difference between revisions

From Mahara Wiki

< Testing Area
No edit summary
Line 1: Line 1:
=== Visual Regression Testing with BackstopJS ===
===Visual Regression Testing with BackstopJS===
Set up:


#Create a new directory e.g. backstopjsmahara and cd into it
#Create a new directory e.g. backstopjsmahara and cd into it
#Run <code>npm i backstopjs</code>
#Run <code>npm i backstopjs</code>
#Run <code>backstop init</code>
#Run <code>backstop init</code>
#Get the list of urls from your local site with this formula in the console in devtools: <code>n=$$('a');for(u in n)console.log(n[u].href)</code>
#Get the list of urls from your local site with this formula in the console in devtools: <code>n=$$('a');for(u in n)console.log(n[u].href),</code> or use a web scraper
#Copy and paste and turn these into a JavaScript array of strings as the value of urls in urls.js
#Copy and paste and turn these into a JavaScript array of strings as the value of urls in urls.js
#The urls.js file creates a scenarios.json file - run on cli with <code>node urls.js</code>
#The urls.js file creates a scenarios.json file - run on cli with <code>node urls.js</code>
#Copy and paste the json array from the generated file into the backstop.json file as the value for <code>"scenarios": (json array)</code>
#Copy and paste the json array from the generated file into the backstop.json file as the value for <code>"scenarios": (json array)</code>
#INSERT HERE: Final state of backstop.json file and onReady.js script used in this context


We're going to start by running <code>backstop test</code> - Make sure your site is the state you want to compare it against i.e. the 'correct' state
We're going to start by running <code>backstop reference</code> - This will create reference screenshots for you to compare with when you run your test.  


When you run <code>backstop test</code> - It will say 'reference url not found' - this is ok because we will use this test to create the references
Then you can make your changes or checkout your test patch


For some reason the report tries to open in Matrix... But you can view the report results by opening the generated backstop_data/html_report/index.html in a browser. Check that the test shots are in the state you want them to be in.
Then you will run <code>backstop test</code> - this will take screenshots and compare them with your reference screenshots


Then you'll run <code>backstop approve</code> - This will make all the tests become the reference files, so next time you run a test, you will have reference files to compare with
For some reason the report tries to open in Matrix... But you can view the report results by opening the generated backstop_data/html_report/index.html in a browser.


Then you can make your changes or checkout your test patch and run <code>backstop test</code> again and the report will show any visual regressions
Then you can run <code>backstop approve</code> - This will make '''all''' the tests become the reference files. Pass a <code>--filter=<image_filename_regex></code> argument to promote only the test captures matching your scenario filename. The filenames (which by default include scenario and viewport names) are displayed in the visual and cli reports.

Revision as of 10:57, 30 Haziran 2022

Visual Regression Testing with BackstopJS

Set up:

  1. Create a new directory e.g. backstopjsmahara and cd into it
  2. Run npm i backstopjs
  3. Run backstop init
  4. Get the list of urls from your local site with this formula in the console in devtools: n=$$('a');for(u in n)console.log(n[u].href), or use a web scraper
  5. Copy and paste and turn these into a JavaScript array of strings as the value of urls in urls.js
  6. The urls.js file creates a scenarios.json file - run on cli with node urls.js
  7. Copy and paste the json array from the generated file into the backstop.json file as the value for "scenarios": (json array)
  8. INSERT HERE: Final state of backstop.json file and onReady.js script used in this context

We're going to start by running backstop reference - This will create reference screenshots for you to compare with when you run your test.

Then you can make your changes or checkout your test patch

Then you will run backstop test - this will take screenshots and compare them with your reference screenshots

For some reason the report tries to open in Matrix... But you can view the report results by opening the generated backstop_data/html_report/index.html in a browser.

Then you can run backstop approve - This will make all the tests become the reference files. Pass a --filter=<image_filename_regex> argument to promote only the test captures matching your scenario filename. The filenames (which by default include scenario and viewport names) are displayed in the visual and cli reports.

Retrieved from ‘https://wiki.mahara.org/index.php?title=Testing_Area/Visual_regression_testing&oldid=13317’