Service Level Agreement
The following table details the service level agreements (SLAs) for XBOX Requirement testing against games releasing on XBOX consoles and XBOX on PC.Product types that require certification
All XBOX product types are subject to XBOX Requirement testing and must pass XBOX Certification before they can be released. Depending on your product and submission type, different requirements and processes might apply. The following types of Products must go through XBOX Certification: Product types:- Full games
- Game Previews
- Betas
- Demos
- Digital download only
- Disc only
- Disc and digital
- Content update
Certification test pass types
Consider the intent of the submission. If you are looking for directional feedback on release readiness, you may want to consider an optional submission. On the other hand, if you intend to release the submission on the XBOX network as a final product, then choose a final submission.Optional submissions
Optional submissions are a paid testing service provided by XBOX Certification. They provide an opportunity to validate how a title is doing from a Certification perspective before submitting your release candidate build. Optional submissions are recommended but aren’t required. They provide an opportunity to discover show-stopping XR issues ahead of final submission and improve your chances of passing first time. However, optional testing isn’t a replacement for good QA which is still essential throughout the development phase. Optional submissions are per package and fees are applied accordingly. For example, if you have both XBOX One and XBOX Series X|S versions and wanted an optional submission for both, these would need to be paid for individually. For complete details on the Optional Submissions paid service, please refer to the XBOX Certification Paid Services Program documentation. Optional submissions are only tested in the CERT.DEBUG sandbox, which is a Microsoft-controlled sandbox that only individuals in certification have access to. It is a developer sandbox where only red signed content is permitted to run (similar to yours). Discs are not created for optional submissions. Absence of CFRs (Conditions for Resubmission) in the optional complete report does not mean the title has passed it’s Final submission. The optional complete report is an assessment of where the title is in terms of compliance with XBOX Requirements at that time. A Pass/Fail decision is not made for Optional submissions like it is for Final submissions. For additional details and a list of the different tiers of optional testing, refer to the Optional Submissions documentation.Final submissions
Final submission is the last certification test pass before a product is released to the public or a disc is Released to Manufacturing (RTM). Communicate with your Microsoft contacts to plan and coordinate your final submission. Final submissions on console are entirely tested in the CERT sandbox, which is a Microsoft-controlled sandbox that only individuals in certification have access to. It is a Retail sandbox where only green signed content is permitted to run. A limited set of tests may be completed in the CERT.DEBUG sandbox with red signed content, but the majority of testing Final submissions is in the CERT sandbox. Certification consoles are retail consoles running the current OS Recovery that is published in RETAIL. PC final submissions are also tested in the CERT sandbox. In contrast to optional submissions, final submissions are entirely tested on retail devices with green-signed binaries and discs. When a product passes final submission, a pass report will be published, and the product can be released. If a product fails final submission, a fail report detailing the Conditions for Resubmission (CFRs) will be published. All CFRs identified in a fail report must be fixed prior to a resubmission. Resubmissions must be made the same way as the previous final submission.Add-on content
Games that support add-on content (durables and consumables) must have at least one product for each type of supported downloadable content (DLC) that is certified alongside the main game. For example, a racing game that plans to release tracks, cars, and packs of virtual currency would need to certify at least one track, one car, and one virtual currency pack add-on. Work with your Microsoft contact to ensure add-on content is published to the Certification sandboxes prior to optional and final submissions.Test disc verification
Products that are releasing on physical discs must have test discs ordered and approved by the XBOX Certification team. It is recommended that you place a disc order alongside the product being submitted for final certification. However, this isn’t mandatory. Test discs are ordered by the XBOX Certification team when a disc product has been submitted in Partner Center and BVTs have passed. Test discs will be shipped to up to three destinations and to internal Microsoft addresses. These are ‘silver’ discs, without artwork, that only contain basic information such as the BinaryID and manufacturing date. The product name and other information are not included for security reasons. Discs come in two formats: BD25 and BD50. Multi-disc format is supported but uncommon, and we recommend working with your Microsoft contacts if that is the case. All test discs are validated by the test disc manufacturer using offline and online console configurations. If a validation check fails, XBOX Certification performs targeted follow-up on the failed check before issuing the certification result. Partners must still perform all necessary testing before approving Release to Manufacturing (RTM). It is critical that partners perform all necessary testing prior to providing approval to RTM. The disc tests performed by XBOX Certification are available in the Media Validation Test Steps document.Certification testing stages
When a submission is received by XBOX Certification, it is tested in three main stages:- Submission checks cover submission validator review and malware scanning.
- Build verification testing (BVTs) are performed to verify that the title is in a testable state and that any previously issued CFRs have been resolved.
- XBOX Requirement (XR) testing is the bulk of the testing and is where all applicable XR test cases will be validated against your title.
- Build a clean release build.
- Test the deployment of loose files.
- Build a submission package with MakePkg.exe.
- Publish a build that can be downloaded from the Store in your development sandbox. Only users who are given access to your sandbox will be able to download the build. This will enable you to acquire and test the title the same way Certification does.
Submission checks
There are two submission checks—Submission Validator and Malware scan.Submission Validator
Submission Validator gives feedback to developers and allows them to address common problems that would cause titles to fail ingestion into Partner Center and/or Certification. When you run MakePkg Pack to produce a package for submission or testing, Submission Validator performs several error checks. Always use the most recent Submission Validator component; this will ensure that you benefit from the latest validation and that your title’s submission proceeds as smoothly as possible. Submission Validator is not a stand-alone tool. Rather, it is called automatically whenever the MakePkg Pack command is used with the /L or /LT switch. Submission Validator will run before the title package is created. For detailed information about Submission Validator, refer to the Submission Validator GDK documentation.Malware scan
To prevent viruses and other malware from contaminating your product, prior to submission, run a malware scanner of your choice against the package. Before performing the scan, confirm that the malware scanner has up-to-date definitions. There’s no need to provide a log or other attestation from your malware scanner. Microsoft will be running its own malware utilities against your submission when it is received. However, running a malware scanner prior to submission will ensure that you identify accidental infections early and don’t wait for Microsoft to find them.Build verification testing
Build verification testing (BVTs) is a reduced test pass designed to make sure a product is fully testable and configured properly before entering full certification testing. This is done by running a suite set of test cases in the main areas of a product. All products must pass BVTs before they are scheduled for full certification Testing. For an in-depth look at BVTs and additional guidance on how to avoid failing BVTs, refer to:XBOX Requirement testing
Once a product has passed BVTs, it is scheduled for XBOX Requirement (XR) testing. The intent of the XR test pass is to validate product compliance with all applicable XRs for that submission and product type. XRs consist of the policies, technical requirements, and product component–related requirements to which all developers and publishers of XBOX products must conform. XRs ensure that products created for XBOX are not only stable and reliable but also provide a user experience that is consistent, safe, secure, and enjoyable.XBOX Requirements
A comprehensive list of XRs for each platform and product type can be found here:- Console and PC: XBOX Requirements for XBOX Games (this page includes requirements for XBOX PC and mobile titles)
- Game Preview and Beta: XBOX Requirements for Open Betas and Game Previews
XBOX Requirement test cases
Each XR that is tested in Certification has a detailed test case that covers how the requirement is tested. The test cases include Test Steps, Expected Behavior and Pass/Fail examples.- Console and PC test cases: Certification Tested XBOX Requirements for XBOX Games (this page includes test cases for XBOX PC titles)
Certification issue types
Issues found during a Certification test pass are categorized into different types based on severity, probability, and repeatability.- Standard Reporting Issue (SRI): SRIs are typically related to an XR but are not deemed severe enough to fail a title. They should be fixed in a future submission, but it is not required.
- Condition for Resubmission (CFR): CFRs are title failing issues. These issues violate the intent of an XR and are deemed severe enough to prevent a title from being released. CFRs must be fixed prior to resubmission.
- Issue of Note (ION): IONs are issues that are not related to an XR or XR test case. They may be issues that a developer wants to fix, but it is not required.
- Non-Tested: If there are XRs and/or test cases that could not be tested during a certification test pass, a list of these XRs and test cases will be included in the report.
Placing submissions on hold
If an issue blocks testing, a submission may be placed on Hold while Certification works with your Microsoft contact to obtain the information or access needed to continue. In some cases, the submission may instead be failed and require resubmission. Holds can introduce delays in the test pass. Common reasons that a title may be placed on Hold include:- Missing or non-functioning partner accounts that are required for testing.
- Partner services do not support the CERT.DEBUG and CERT sandboxes.
- Multiplayer does not work as expected.
Requesting exceptions
If your title is not fully compliant with any XBOX Requirement (XR), you can request an exception for that XR. This might occur if your title is introducing innovative features or if strong technical limitations would prevent the title from meeting the XR. Work with your Microsoft contacts well in advance of submitting to Certification if an exception is needed. Approval of an exception is not guaranteed, however, obtaining advanced feedback would give you plenty of time to resolve any issues. You must close all outstanding exception requests before submitting your title for certification. Allowing time to discuss exceptions and resolutions can prevent delays. Microsoft does not want to block new or innovative features. If you are creating something new, please tell us in your exception request. To request an exception prior to submitting to Certification, reach out to your Microsoft contacts.Certification reports
All titles receive a report at the conclusion of test. There are different report types based on the Certification Test Pass Type. The different report types are:- Optional complete report
- Final pass/fail report
- Interim report