Manual acceptance in a spreadsheet
A person follows a checklist and records the results in a spreadsheet.
Comparison
Avalyz, manual testing or testing tools: which to choose?
Three common ways to run a website's acceptance testing, and what Avalyz does or does not add alongside them. Every statement about a third-party technology points to its official source, listed at the bottom of the page.
The approaches compared
Code-it-yourself test framework
A team writes automated tests with an open source project, for example:
- Playwright : End-to-end test framework: Chromium, WebKit and Firefox, with mobile emulation of Chrome for Android and Mobile Safari. [1]
- Cypress : Free, open source, locally installed app for writing and running tests of anything that runs in a browser. [5]
- Selenium : A project of tools and libraries for browser automation that implements the W3C WebDriver specification. [7]
Generic SaaS monitoring or testing platform
An online service monitors websites or runs scenarios; features and prices vary by vendor.
Avalyz
Online website acceptance testing software for agencies: in-browser tests, client report and PDF sign-off report.
Criterion by criterion
| Criterion | Manual acceptance in a spreadsheet | Code-it-yourself test framework | Generic SaaS monitoring or testing platform | Avalyz |
|---|---|---|---|---|
| Setup time | Immediate: a spreadsheet template and a checklist are enough. | Install the tool and its environment (for example Node.js for Playwright), then write the tests. [1] | Create an account, then configure the checks or scenarios the platform offers. | Paste a website address: nothing to install. An instant test without sign-up is available, 3 times a day. |
| Skills required | Rigour and knowledge of the site; no programming. | Programming: TypeScript or JavaScript, Python, .NET or Java for Playwright; Java, Python, C#, Ruby, JavaScript or Kotlin for Selenium. [1] [2] [8] | They vary from one platform to another, from entering addresses to writing scripts: check its documentation. | No programming for a test. User journeys are written as plain sentences (French or English) or imported from Gherkin or CSV. |
| Sign-off report for the client | To be written by hand from the spreadsheet. | The tools produce test reports for the team (Playwright opens an HTML report); a sign-off report for the client still has to be written. [1] | Depends on the platform: check its documentation. | PDF sign-off report and a report in plain client language, under the agency's brand, shared through a protected, expiring link. |
| Accessibility | Possible if the person knows the criteria. Human judgement remains essential for part of the checks. [10] [11] | Possible by adding a checking engine. Automated tests detect some common problems; many can only be found through manual testing. [4] | Depends on the platform. No tool can automatically check every aspect of accessibility. [10] | Partial automated checks: each defect found is linked to the WCAG 2.2 and RGAA 4.1 criteria it affects. It is not an RGAA audit. |
| Cost | No licence: the cost is the time spent, on every acceptance round. | Free, open source projects (Apache-2.0 licence for Playwright and Selenium, MIT for Cypress); the cost is development time and running the tests. [3] [5] [6] [9] | Subscription, according to each vendor's pricing model. | €29 excl. VAT per month (Solo) or €79 excl. VAT per month (Team); 14-day trial with no credit card. |
| Maintenance | Keep the list up to date when the site changes, and redo every check by hand. | The test code follows the site's changes and is maintained like any code. | The platform is maintained by its vendor; your scenarios still need to be kept up to date. | The software is maintained by Avalyz. When a selector in a user journey no longer finds its element, a repair is proposed and applied only if you confirm it. |
Which to choose?
- Manual acceptance remains necessary for anything that needs judgement: content, usability, accessibility audit.
- A code-it-yourself framework suits a development team that wants to control every test in its own code and pipeline.
- A generic platform suits needs for monitoring or testing without an acceptance deliverable for a client.
- Avalyz targets repeated acceptance testing of websites delivered to clients, with a report and a sign-off report to hand over.
- These approaches can be combined: Avalyz replaces neither human acceptance testing nor tests coded by your team.
What Avalyz does not do
- No full RGAA audit: automated checks are partial; conformity is verified by a human audit.
- No native mobile app testing (iOS, Android): Avalyz tests web pages, on desktop and at phone size.
- The standard test is read-only: it submits no form and stops at 6 pages.
- No done-for-you services: no acceptance testing or advice carried out by a person.
Sources
- Playwright — official documentation, "Installation" (browsers, mobile emulation, Node.js, HTML report) — https://playwright.dev/docs/intro, .
- Playwright — official website (languages: TypeScript, JavaScript, Python, .NET, Java) — https://playwright.dev/, .
- Playwright — official repository licence (Apache-2.0) — https://github.com/microsoft/playwright/blob/main/LICENSE, .
- Playwright — official documentation, "Accessibility testing" — https://playwright.dev/docs/accessibility-testing, .
- Cypress — official documentation, "Why Cypress?" — https://docs.cypress.io/app/get-started/why-cypress, .
- Cypress — official repository licence (MIT) — https://github.com/cypress-io/cypress/blob/develop/LICENSE, .
- Selenium — official documentation, "The Selenium Browser Automation Project" — https://www.selenium.dev/documentation/, .
- Selenium — official documentation, installing the libraries by language — https://www.selenium.dev/documentation/webdriver/getting_started/install_library/, .
- Selenium — official repository licence (Apache-2.0) — https://github.com/SeleniumHQ/selenium/blob/trunk/LICENSE, .
- W3C WAI — "Selecting Web Accessibility Evaluation Tools" — https://www.w3.org/WAI/test-evaluate/tools/selecting/, .
- DINUM — RGAA, "Critères et tests" (test methodology) — https://accessibilite.numerique.gouv.fr/methode/criteres-et-tests/, .