- Creating an AQuA Affiliate account
- Uploading app details and selecting attributes
- The Quality Levels and Quality Badge
- Auditing and preventing mis-use of the Directory and the Quality Badge
- The QAD-approved Test Houses
- The AQuA Testing Criteria and the Online Testing Criteria Tool
- The Online Java Pre-Tester Tool
Why have a Quality App Directory?
To give visibility throughout the industry to apps that have reached a specified level of quality relating to usability and application behavior within the device. This will help those apps stand out from the crowd.
Who benefits from the Directory, and how?
Huge benefits are gained by developers uploading details of apps. They are able to demonstrate for the first time the quality of their apps (and, by association, their organisation).
AQuA members have access to all of the information within the Directory, and are making decisions based on that information. It also helps AQuA members and others identify how much additional testing they need to do themselves if they were to include an app within an embedded or over-the-air distribution channel.
"AQuA contacted Shape to tell us that 5 of our apps had been tested by an AQuA Core Member and could go into the Quality App Directory with an ‘AQuA Member Verified’ quality level. This opened us a door to AT&T (another AQuA Core Member) and their Certified Solutions Catalog.
AQuA connected us with AT&T team in a quick and friendly manner. Shortly all Shape's applications such as IM+, iDisplay, RDM+, Business Card Reader and others were published in the AT&T Certified Solutions Catalog.
We highly appreciate our cooperation with AQuA and would recommend this opportunity to our partners."
Natalia Kasyanova, Shape AG, June 7 2013
Finally, the Directory is open to the public, enabling them to check the quality of an app before they download it from a distribution channel: if the app displays the AQuA App Quality Badge (with the badge linking back to the developer’s app page within the Directory), they'll know it's reached an approved standard.
What mobile platforms does the Directory cover?
It started it with Android apps, and now iOS is included too.
Who is determining the level of quality?
The App Quality Alliance (AQuA) (with industry input) has compiled a set of Testing Criteria for Android, Windows and iOS Apps.
When uploading details of their apps, developers who have registered to use the Directory will have confirmed that each app has passed the Criteria.
How is the Directory funded?
The AQuA Members are funding both the Directory and the auditing of the 'self-verified' apps. There is no cost to the developer for submitting details of apps into the Directory. However, if you want your app to feature the new Quality Badge, you'll need to have it tested using one of the AQuA-approved test houses.
Why should I create an AQuA Affiliate account?
Anyone wishing to upload details of their apps into the Directory and / or wishing to use the Online Testing Criteria Tool needs to create an account.
When you create an account, you will become an AQuA Affiliate. This gives you access not only to the Quality App Directory but also a number of online tools that aren't directly related to the Directory.
How much is it?
Is the account for me or my organisation?
For your organisation, although as many people as you like can ‘join’ that organisation, and therefore your organisation's AQuA Affiliate account.
The person to set the account up is the main contact, and can invite others to join them. Plus, others can request to join them.
Anyone listed in the Directory within that organisation can then upload details of apps and use the Online Testing Criteria Tool. Click on register now and take a look.
What if I don't want anyone to know I've created an account?
When you create an account, you will have the option of stating whether you want your organisation to be publicly listed on the AQuA site as an AQuA Affiliate. If you don't, as long as you haven't uploaded details of apps into the Directory, your AQuA Affiliate status will not be made public. If you have uploaded details of apps into the Directory, your apps will need to be associated with your organisation, and therefore your organisation will be automatically displayed on the site.
Have you seen the other new AQuA Membership levels?
Can any developer submit app details into the Directory?
Yes. Any developer with an AQuA Affiliate account wishing to submit details of apps that meet the quality standard can create an account. They can then upload information for each app.
The first person within an organisation to create an account becomes the 'main developer'. They then have to approve the authenticity of anyone else from within that organisation wanting to submit app details.
Do the apps themselves need to be uploaded?
No. This isn't an app store. It's a reference point for AQuA members, the public, developers and other interested parties to see details of apps that have met certain quality standards.
What app details needs to be input?
Name of app
Languages its available in
Distribution channels its available through (and link to the app if appropriate)
The category it fits into
The platform version used to develop and test the app
The device attributes it was designed to utilise (screen size, input mode, orientation)
Quality Level (self-verified or test house-verified)
Selection of the one device against which it has been tested
Selection of any devices against which it has been smoke-tested
How will you know the apps have met the standard?
Because developers will have signed the Ts and Cs to tell us that they'll only upload details of apps that have met the standard: by the 'standard', we mean apps that have passed the tests outlined in the Testing Criteria for Android apps or the Testing Criteria for iOS apps.
What are the different quality levels?
When uploading details of an app, the developer is asked to select a quality level for that app:
- Self-Verified: the developer states that the app has been tested against the latest version of the AQuA Testing Criteria for Android Apps or the Testing Criteria for iOS Apps and has passed, but has no proof of this.
- Test House Verified: the developer states that the app has been tested and passed the Android or iOS Testing Criteria using a QAD-approved test house. In this instance, the developer selects from a drop-down menu the name of the test house the app has been tested with, and email is automatically sent to that test house asking them to upload the test report into the system. Once this has been done, the status of the developer's app changes to 'Test House Verified'.
- AQuA Trusted Status Verified: the developer is an AQuA Member with Trusted Status, and states that the app has been tested and passed the Android or iOS Testing Criteria. Members that have gained Trusted Status can upload apps directly to the Directory which they have tested themselves, without using a test house.
- AQuA Core Member Verified: this is where an app has been previously successfully tested by at least one of the AQuA Core members. They're sharing this information for the benefit of other distribution channels, and to save the developer having to test it again.
Is there a Quality Badge developers can use in association with apps featured in the Directory?
Yes. The AQuA App Quality Badge is available for apps that have met the quality level of 'Test House- Verified' or 'AQuA Trusted Status Verified' or 'AQuA Core Member-Verified'. (It's not for the developer but for the actual app). Once an app has been approved to use the Quality Badge, the developer will be asked to sign additional Ts and Cs, agree to the Usage Guidelines and then download the Badge.
- If the AQuA App Quality Badge is inappropriately used, or used in conjunction with a non-approved app, the appropriate legal action may be taken.
- The Ts and Cs state how and where the Badge can be used, and emphasize the importance of:
a) linking back to the app's entry in the Quality App Directory at all times when used online and
b) having a line of text stating the meaning of the Badge if used offline.
How do you know an app has met the appropriate quality level?
AQuA conducts audits on self-verified apps periodically (at our own cost), and if they fail the audit, we make that clear within the Directory, having given the developer time to rectify the situation.
We reserve the right to remove apps at any time, and blacklist developers abusing the directory. This is made clear within the Terms & Conditions each developer signs when creating their account.
How will you prevent mis-use of the Quality Badge?
If the AQuA App Quality Badge is inappropriately used, or used in conjunction with a non-approved app, the appropriate legal action will be taken, which will include posting a notification on www.qualityappdirectory.org of the offending app and associated developer.
Both of the above are in place to retain the integrity of developers using the Directory, their apps, and AQuA as an organisation.
Who are the QAD-approved test houses, and will it cost the developer money to use them?
We have three QAD-approved test houses through which developers can test their apps against the AQuA Testing Criteria to gain the 'Test House Verified' quality level:
- CTTL SmarterApps (also an AQuA-Endorsed Testing Service)
- VMC (also an AQuA-Endorsed Testing Service)
The test houses have their own cost structure, and will charge developers for app testing, but we have worked with them to ensure the costs are competitive. It is in AQuA's interest for the testing costs to be as low as possible to encourage developers to get their apps 'Test House Verified'.
AQuA takes no revenue from the costs of testing. The relationship is between the developer and the test house.
Please note - the testing we're talking about here relates purely to testing the app for usability against the AQuA Testing Criteria for Android, Windows and iOS Applications. The test houses mentioned here (and others) can of course also provide additional QA support to you, and if you'd like to be quoted for any additional service, we're sure they'd be delighted to hear from you!
The cost of testing depends on which AQuA-approved test house the developer chooses to use and the type of test needed - eg framework app tests, complex app tests, simple app tests.
- The approximate cost of having just content tested within a framework (eg a tour guide for Italy as part of a series of tour guides built in the same way) is $40.
- The approximate cost of testing a typical simple app (eg a sports news and scores reporting app) is $150.
- The approximate cost of testing a complex app (eg an integrated business mileage tracker) is $250.
Click through to each of the test house sites above and contact any or all of them for more information. Please remember, if you’d like to reach the Quality Level of Test House Verified, you need to have your app tested with a test house before selecting that Quality Level within the Directory.
Do developers need to contract with a test house before uploading details of their apps?
Yes. When a developer uploads details of an app that is 'test house verified' this assumes the app has already been successfully tested, and that the test house has the report and is waiting to upload it. The trigger for the test house to upload the report is when the developer selects the 'test house verified' quality level for that app within the Directory.
Can developers change the quality level of an app once details have been input?
Yes. If, for example, an app has the quality level of 'Self Verified', and the developer then decides to use a test house to test that app, once successfully tested, they can simply change the quality level to 'Test House Verified' which triggers the request to the test house to upload the report. Once uploaded, the quality level is automatically changed to 'Test House Verified', and the app can be used in conjunction with the AQuA App Quality Badge (subject to the Ts and Cs).
Likewise, if the quality level of an app is 'Self Verified', and an AQuA member has since tested that app successfully, AQuA can go into that app record and update the quality level to 'AQuA Core Member Verified'. At this point the developer is informed automatically by email that the quality level has been changed and the app can be used in conjunction with the AQuA App Quality Badge (subject to the Ts and Cs).
What device does an app need to be tested on?
When a developer uploads details of an app, they are prompted to select just one device from a list of Android devices. The device they select will be the one they say the testing has been carried out on. If the device they have tested their app against doesn't appear within the list, they can let AQuA know and request that it is added before they continue to upload the details of their app.
What is the AQuA Testing Criteria?
The Testing Criteria is an evolving set of platform-specific tests, built up from input from developers, test houses and AQuA members. It provides tests that compliment the functional testing carried out during development, and includes a range of non-functional, usability, installation, performance, compatibility and functional sanity testing.
However, it intentionally does not look at the usefulness or function of the application, and isn't intended to capture everything. It is simply a guide to the level of acceptable usability AQuA members feel an app should have before it is commercialised.
All apps detailed within the Quality App Directory need to have been tested against the AQuA Testing Criteria for Android, iOS or Windows apps.
Those displaying the quality level of 'Test House Verified' need to have been successfully tested by one of the above test houses, with the test report uploaded into the Directory by the test house.
What is the Online Testing Criteria Tool?
It's a tool available from an AQuA Affiliate member's dashboard to help you run through the tests within the AQuA Testing Criteria and assess the result. The tool is available for the Baseline Testing Criteria for Android, iOS and Windows Apps and the Accessibility Testing Criteria for Android and Windows Apps.
- Input a few details of your app, and the tool selects the relevant tests. Then run through the tests, scoring the app as you go. (You can pause and then go back to the testing whenever you like.)
- Once you've finished the test suite for an app, you'll see the result - pass or fail - and can create a report.
All your existing and past tests are clearly shown in the system, for you to go back to at any time.