NSI Guide Supplement for QA Best Practices Certification/Verification
Issue 1.2 - March 2017
Table of Contents
1.1 Referenced Specifications
1.2 The NSI Quality Assurance Best Practice Conformance Requirements Documents
1.3 Preparing for Quality Assurance Best Practice Certification or Verification
2.1 Registering your Business Practice within the Web-Based Certification/Verification System2.1.1 Payment2.2 Producing a Conformance Statement
2.1.2 Payment for Two Concurrent Quality Assurance Best Practice Certifications or Verifications
2.3 Project Definition and Document Submission2.3.1 Project Definition2.4 Completing the Checklist
2.3.2 Document Submission2.4.1 Documentation Checklist2.5 Assessment Process
2.4.2 Assessment Checklist
2.6 Completion of Certification/Verification
This document is a supplement to the Guide to the NSI Certification and Verification Program and is intended to be used in conjunction with that document. This supplement is for organizations who want to certify or verify a business practice in the North American Association of State and Provincial Lotteries (NASPL) Standards Initiative Certification and Verification Program for Quality Assurance Best Practices.
This Guide Supplement will guide you through all the Quality Assurance specific steps required for certification or verification. Steps that are common to all NASPL Standards Initiative (NSI) certification/verification programs are provided in the Guide to the NSI Certification and Verification Program.
Note: In the event of any conflicts, this guide defers to the NSI Certification/Verification Policy and Policy Supplements, the applicable NASPL Trademark License Agreement, and the NSI Certification/Verification Agreement.
The specifications currently referenced for QA Best Practices certification/verification are:
- Best Practice for Quality Assurance of Product Development in the Lottery Industry: Requirements Definition, Document Number BP0401, April 2004, and any Corrigenda issued against such Best Practice
- Best Practice for Quality Assurance of Product Development in the Lottery Industry: Development Process, Document Number BP0402, April 2004, and any Corrigenda issued against such Best Practice
- Best Practice for Quality Assurance of Product Development in the Lottery Industry: Acceptance Testing, Document Number BP0403, April 2004, and any Corrigenda issued against such Best Practice
These are referenced by the NSI Conformance Requirements documents and are located at http://www.opengroup.org/naspl/published.
The current NSI Conformance Requirements documents against which Quality Assurance business practices can be certified are:
The current NSI Conformance Requirements documents against which Quality Assurance business practices can be verified are:
Certification/verification will involve validating that your organization has correctly and completely implemented the Best Practice(s) within your organization. The first step towards certification or verification against any of the Quality Assurance Best Practices is to deploy the Best Practice(s) within your organization. This involves making the changes necessary to your business processes to ensure that your organization is operating in accordance with the requirements defined in the Best Practice(s). Your organization then needs to use the overall business practice (i.e., for Requirements Definition, Development Process, etc.) on several projects related to the creation or updating of hardware and/or software systems for a NASPL lottery environment.
To achieve certification or verification, your organization must complete both the organization-level steps described in the Guide to the NSI Certification and Verification Program as well as the Quality Assurance specific steps described below.
The organization-level steps are submitting your organization's registration to the Certification and Verification Authority (CVA) and completing the applicable Trademark License and Certification/Verification Agreements. These steps need to be performed only once per organization.
Prior to registering your business practice, please ensure that:
- Your organization registration is submitted to the CVA, as per Section 3.1, "Registering your Organization", of the Guide to the NSI Certification and Verification Program
- You are either a Key Contact or Technical Contact for your organization, and thus have permission to register your business practice.
The first step for QA Best Practices certification or verification is to register your business practice. Then, the following three steps may be performed in parallel for producing a conformance statement, selecting projects and providing documentation on those projects, and completing the checklist.
To register a business practice to be certified or verified, you must submit the completed Registration Form to the Certification and Verification Authority.
When you register your business practice, you are required to identify the NSI Conformance Requirements document against which you wish to certify or verify, provide information on your business practice, and have the ability to send notes to the Certification and Verification Authority. You must specify the following:
- Conformance Requirements document - the Conformance Requirements document against which you want to certify or verify your business practice. You must select the applicable Conformance Requirements document.
- Business Practice Location - the specific, identified, physical location in which your organization's business practice is carried out under the leadership of the Business Practice Manager(s) and in which the documented processes and procedures of the business practice are accessible.
Concurrent Certification/Verification - Specify if your organization intends to concurrently certify or verify your product development business practice against two Quality Assurance Conformance Requirements Documents. Your organization must meet all of the following conditions in order to qualify for concurrent certification/verification:
- Your organization must identify for certification or verification against each Quality Assurance Conformance Requirements document separately within the system (i.e., for "Requirements Definition for Vendors" and "Development Process" or "Requirements Definition for Lotteries" and "Acceptance Testing").
- The registrations must occur within 4 calendar days of each other. Specifically, the second registration must be submitted by the end of the 4th calendar day following the initial registration.
- Within the Registration Form, your organization must express its intent for these two registrations to be considered as registered together. This is done on the first registration by filling in the Concurrent Certification/Verification section. On the second registration, you must also identify the prior registration with which you wish to be concurrently certified or verified.
- Both registrations must specify the same Business Practice Location. The assessment process will then be carried out at this location, at the same time, for both certifications/verifications.
Business Practice Manager(s) - the specific individual(s) within your organization identified as having the overall responsibility for managing the business practice on a day-to-day basis and ensuring that it is carried out in accordance with the documented processes and procedures.
The managers to be specified will vary by Conformance Requirements document:
- For "Requirements Definition for Lotteries" and "Requirements Definition for Vendors", specify the lead person within your organization responsible for managing the requirements definition process.
- For "Development Process", specify the lead technical and quality assurance persons within your organization who have overall responsibility for managing the technical and quality aspects of the product development process. These managers may be the same person.
- For "Acceptance Testing", specify the lead person within your organization responsible for managing the acceptance testing process.
- Confidentiality - allows you to indicate whether you would like this certification or verification to remain confidential for a period of time, as defined in both the NSI Certification/Verification Agreement and the Certification/Verification Policy
- General Notes - this is an area for you to provide any notes to the Certification and Verification Authority about this registration. Anything included in this area will be confidential between you and the Certification and Verification Authority.
In addition, you will need to specify the certification/verification contacts specific to certification/verification of the business practice you are registering:
- Primary Technical Contact - the primary contact for all notices, status updates, and issues related to certification/verification for this business practice. The Primary Technical Contact and the Alternate Technical Contact, if specified, along with the Key Contacts, are the only users who will have the ability to manage the progress of this certification/verification process. The Primary Technical Contact is the person primarily responsible for submission of all certification/verification information to the Certification and Verification Authority and is the primary point of contact in your organization for the assessment process.
- Alternate Technical Contact (optional) - the back-up for the Primary Technical Contact. Will receive the same information on the certification/verification progress as the Primary Technical Contact and also has the same abilities within the system to progress the certification/verification registration through the certification/verification process.
- Finance Contact (optional) - the contact for any payment and/or invoicing issues related to this certification/verification; if not specified, will default to the organization Finance Contact
- Marketing Contact (optional) - the contact for any marketing-related certification/verification issues for this business practice; if not specified, will default to the organization Marketing Contact, if one was specified
Once you have completed all fields, select the "Submit for registration" button. You will then be presented with the NSI Certification/Verification Agreement, which you must accept in order to proceed with this business practice registration. The final step in the registration phase is providing payment information, as detailed in Section 2.1.1, Payment, below.
You are then returned to your organization's certification or verification home page, from which you can select the "Next" link for the business practice being submitted. Through use of the "Next" link, the system will guide you through the steps that you need to follow to certify or verify a business practice.
Details concerning payment procedures common to all NSI certifications/verifications can be found in Section 4.1.1, "Payment", in the Guide to the NSI Certification and Verification Program.
See Section 2.1.2, "Payment for Two Concurrent Quality Assurance Best Practice Certifications/Verifications", below, for payment procedures that are specific to NSI QA Best Practices certifications/verifications.
For lottery organizations that have made payment in advance directly to NASPL for a Quality Assurance Best Practice Verification Fee, select the "Pre-Payment to NASPL" option.
If you have indicated that you would like to concurrently certify or verify your product development business practice against two QA Conformance Requirements Documents, please note that although you will be asked to pay the standard certification/verification fee for the first registration during the registration process and will be asked to provide information concerning your method of payment, processing of your payment will be deferred for up to four days. If a second registration for concurrent certification/verification takes place within this time frame, the registration form will reflect the remainder of the concurrent certification/verification fee for this second registration and The Open Group will collect a single payment for the two concurrent certifications/verifications.
Otherwise, if a second registration is not made within four days, the standard fee will be collected for the initial registration. Any subsequent certification/verification registrations will be charged the standard certification/verification fee.
This step, along with the following two (Project Selection and Document Submission, and Completing the Checklist) can all be completed in parallel. All must be completed before the Certification and Verification Authority can begin the assessment process for your organization.
A Conformance Statement describes your business practice and how it meets the conformance requirements. Your Conformance Statement will be linked into the Register entry for the business practice once it is certified or verified. A Conformance Statement Questionnaire is available for each Conformance Requirement.
The currently available Conformance Statement Questionnaires are listed below. There is one corresponding to each Conformance Requirements document:
- QA Requirements Definition for Lotteries
- QA Requirements Definition for Vendors
- QA Development Process
- QA Acceptance Testing
You must complete the relevant questionnaire to create a Conformance Statement for your business practice. See Sections 2.3 and 3.3 of the NSI Certification/Verification Policy document for further information on the purpose of the Conformance Statement.
The Conformance Statement Questionnaires have common front matter, which asks for the following information about your organization, the business practice to be certified or verified, and rationales for any requirements of the Best Practice(s) that your organization did not implement:
- Submitter Information - your organization's name and your name as author of the Conformance Statement. The organization name that you registered will automatically be entered in the Conformance Statement Questionnaire by the system.
- Business Practice Information - your business practice location and the name(s) and title(s) of your business practice manager(s).
- Best Practice Implementation - your organization is required to implement all of the applicable "Should" requirements in the Best Practice(s) or provide a rationale for why any are not implemented in your business practice. This section enables your organization to provide any required rationales for "Should" requirements that your business practice does not support at all or does not support as a normal course of business.
The Conformance Statement Questionnaires also contain questions specific to the set of conformance requirements against which your business practice is to be certified or verified. These questions are intended to characterize how your business practice meets the conformance requirements.
Prior to submitting documents to the Certification and Verification Authority to provide evidence that your organization has correctly implemented the Best Practice, you are required to provide information on several recent projects on which the Best Practice has been deployed. The Certification and Verification Authority will then select two of those projects for which your organization will submit the relevant documentation.
Please note that the documentation provided to the Certification and Verification Authority at this stage will be used by the assessors for the documentation and telephone assessment phases of the assessment process and that, during the on-site assessment, the assessors reserve the right to review and inspect additional documentation on the specified projects or other projects, as deemed appropriate by the assessors.
There is a section in the Registration Form for defining a project. You should complete it for each project for which you are providing information. You should provide information on at least four projects that have been performed over the past two years. You should specify at least one project that includes a new lottery system component and at least one that includes an update to an existing lottery system component.
You may modify or delete project definitions up to the point at which you submit them to the Certification and Verification Authority.
To define a project, you will need to provide the following information:
- Project ID/Name - this is a unique textual identifier for the project that will enable you and the Certification and Verification Authority to clearly communicate with respect to this project.
- Phase completion date - this is the month and year in which the applicable phase of the overall development activity was completed.
- Phase Duration - this is the duration of the applicable phase of the overall project. The phase duration may be specified in either calendar months or person-months. If the duration of this phase is indicated in the High-level Project Plan for the overall project (as specified in the Requirements Definition Best Practice), please use that duration.
- Type of changes to lottery environment - in this section, you should indicate all of the various changes that were included in the project. If specific changes are not listed, select Other and specify the changes in the text box.
In the event that you are unable to fully meet the requirements of providing information on at least two projects and specifying at least one project that includes a new lottery system component and at least one that includes an update to an existing lottery system component, you must provide a rationale for not meeting these requirements.
Your rationale should explain:
- Why you are unable to provide definitions for two projects, including both a new lottery system component and an update to an existing lottery system component
- Why you believe the projects whose definitions you have provided supply sufficient evidence of your organizations' implementation of the QA Best Practice, such that the Certification and Verification Authority will be able to adequately evaluate your implementation
In your rationale, please briefly cover what your organization has done thus far and what your future plans are with respect to deploying the applicable NSI Quality Assurance Best Practice within your organization. This could include information on other recent projects, plans for future projects, or a description of your QA practices on prior projects.
Submission of relevant documentation is done via Dropbox for Business. The CVA will provide instructions on how to do this.
Documentation must be submitted for each project. In addition, for certification against the Development Process Conformance Requirements, additional documentation on the organization's processes must be submitted.
You may upload more than one file to satisfy the requirement for a particular document, if that document is contained in multiple files. Likewise, a single file may satisfy the requirements for more than one type of documentation. The Document Title and the Documentation Checklist in the Registration Form will be used to map specific files to the documentation requirements.
For each document to be uploaded, you will need to supply the following:
- Project Identifier/Process - select the identifier associated with the project or, for Development Process certifications/verifications, the type of process associated with the document.
- Document Title - the unique file name--of the document.
- Comments - optional comments concerning the document.
Documentation requirements are as follows:
Requirements Definition for Lotteries
- Requirements specification
- Documented acceptance criteria
Requirements Definition for Vendors
- System design documentation
- User interface documentation
- High-level project plan
Process documentation on:
- Detailed design process
- Implementation approach and supporting practices
- Internal testing process
- Change control management process
- Problem reporting process
From recent projects:
- Vendor's internal test plan
- Release notes
- Integration plan
- Test summary report
- Process documentation on:
- Acceptance test plan
- Test report from acceptance test execution
- Definition of acceptance test environment
Completing the Checklist must be completed and submitted to the CVA in order for the Certification and Verification Authority to begin the assessment process on your organization.
The checklist identifies the various documents your organization is uploading with its certification or verification registration along with information on where in each of these documents there is evidence that the your organization has correctly deployed the Best Practice requirements.
This section of the checklist contains a table for each of the types of documents that you need to submit to the Certification and Verification Authority. You must submit documents for both of the two projects and in the case of the Development Process Conformance Requirements, you will need to submit process documentation as well.
In each table, for each submitted document, enter the Document Title. The Comments field may be used to provide any comments to the Certification and Verification Authority. (For example, if the same document file includes information that satisfies multiple requirements, the document should be listed in each table, with a comment indicating such.)
This Checklist contains tables listing each requirement (from the Best Practices) that your business practice must meet in order to be considered conformant. You should fill out these tables as completely as possible.
For both of the two projects, you will be submitting the relevant documents to the Certification and Verification Authority. In these tables, you must specify the Document Title for the document in which a given requirement is met, and provide a reference, where applicable, to the specific section of the document that provides the evidence that your organization has met the requirement.
Your organization is required to implement all of the 'Must' requirements. Your organization is also required to implement all of the 'Should' requirements, unless you specified otherwise on your Conformance Statement. You should be able to provide evidence for most of the requirements with a Level of 'Must' or 'Should'. Any requirements with a Level of 'May' are optional, and evidence only needs to be provided in cases where that requirement is implemented within your business practice.
Requirements for which evidence of implementation cannot be provided via submitted documentation must have their implementation validated during the On-site Assessment.
During the assessment process, the Certification and Verification Authority will take the lead in progressing your registration through the process. At any point in time, you will be able to see the status and progress of the assessment process in the web-based certification/verification system by selecting the "Next" step button for your certification or verification registration from your organization's home page.
In cases where your organization has responsibility for progressing the assessment process, notification will be sent to your Primary Technical Contact and Alternate Technical Contact regarding what is required of your organization.
Throughout the process you will be notified:
- When Assessor(s) assigned and date the assignment was made
- If the Documentation Assessment has been completed, the completion date; otherwise, an indication that the Documentation Assessment is still in progress
- If the Telephone Assessment has been completed, the completion date; otherwise, the scheduled date for the Telephone Assessment
- If the On-site Assessment has been completed, the completion date; otherwise, the scheduled date for the On-site Assessment
Once the On-site Assessment has been completed, you will also see an indication of whether corrective action is required.
If corrective action is required, you will receive notification of the following:
- Date by which your corrective action plan must be delivered to the Certification and Verification Authority
- Form for uploading your corrective action plan
- Date the corrective action plan was submitted to the Certification and Verification Authority
- Date by which your organization must complete the actions in the corrective action plan
- Date on which the Certification and Verification Authority reviews and approves your corrective action plan
- Type and date of follow-up assessment
- Date on which the Certification and Verification Authority reviews and approves the Assessment Report
Once the assessment process has been completed, your web view will display either the date the Assessment Report was approved or else an indication that review of the Assessment Report is still in progress.
Once your submission, including the assessment process, is complete and the Certification/Verification and the NASPL Trademark License Agreements are in place, you will be notified of successful certification or verification within 10 business days. If for any reason the submission was not complete, you will be notified so that any corrections can be made and resubmitted. This step, which is common to all NSI Certification and Verification Programs, is described in detail in the Guide to the NSI Certification and Verification Program.
You should note that certification/verification is valid for a defined period as stated in Section 9.1, "Duration of Certification/Verification", of the NSI Certification/Verification Policy document. At the end of that period, if you wish your business practice to remain certified or verified, you will need to renew your certification or verification as defined in Section 3, Renewals Process, below.
When your business practice is certified or verified, it remains so for a defined period, after which it must be renewed or the business practice will no longer continue to be certifiedl/verified. At any time, you may view the renewal date for each of your certified or verified business practices or technologies on the appropriate register.
Renewal implies that your business practice continues to conform for the duration of the renewal period.
Around three months prior to the expiration of your business practice's certification or verification, your Primary and Alternate Technical Contacts will be notified that renewal is due. They must respond to the Certification and Verification Authority within 30 calendar days indicating whether or not your organization would like to renew certification/verification; otherwise, your certification or verification will expire on the renewal date. Renewal is subject to a re-assessment of your business practice and must be completed prior to your renewal date.
The Renewal Form is available on the appropriate Documents section on the program's nav bar.
Please see Section 9, "Renewal", of the NSI Certification/Verification Policy document for further information on the requirements and process for renewal.
Any modifications to your Certified/Verified Entity must be made in accordance with the requirements defined in the Certification/Verification Policy. You should read the requirements in the policy documents thoroughly since they describe multiple scenarios related to business practice changes. In some cases, the modification or update necessitates a new certification or verification, whereas in other cases, you can update your existing certification or verification.
Updates to your business practice's certification or verification can be done by submitting the Update Form to the Certification and Verification Authority.
The types of changes that may be made to a business practice certified or verified against the Quality Assurance Best Practices are defined in Section 5, "Certification/Verification Requirements for Modifications of a Certified/Verified Entity", of the NSI Certification/Verification Policy Supplement for QA Best Practices. Please note that you are required to contact the Certification and Verification Authority within 30 calendar days of any changes to your business practice.
If the "Certification/Verification Requirement" identified in the policy document is either "Certification/Verification Information update" or "Conformance Statement update", you must complete the Update form to provide information on the requested change and provide the Certification and Verification Authority with any required information or documentation. Upon review and acceptance of such information or documentation and receipt of payment for amending your registration, the Certification and Verification Authority will update your certification/verification information.
If the "Certification/Verification Requirement" is "New Certification/Verification", then this change is considered a new business practice and is subject to a new certification or verification. In this case, your organization should register using the Registration Form.
Any other change must be communicated to the Certification and Verification Authority within 30 calendar days of such change. The Certification and Verification Authority will then determine what the associated assessment and certification/verification requirements are for such change.