BETA

Description

NCF Beta Testing Group consists of members willing to assist Office Staff in testing proposed changes in a controlled environment before rolling it out to all members.

Guidelines

1. Participation in the Beta Testing Group is voluntary. Members can opt in and opt out of Beta Testing Group at anytime. Email beta@ncf.ca
2. Participation in each Test Case is by invitation and requires Beta testers to opt-in.
   • Members may not be invited to participate in a Test Case if they have a pending trouble ticket or service issue that will taint the interpretation of testing results.
3. Members can opt in or opt of each Test Case they're invited to.
4. Members will generally participate in one Test Case at a time
5. NCF will not provide direct monetary compensation for testing (e.g. Compensation for time or expertise)
   • NCF may however pass on incentives for testing hardware and services when available.
6. Members can abandon testing if service disruptions occur and contact the Help Desk for help troubleshooting their service issue.

Process

1. Beta Testing Group will be managed by the Service Operations Manager and the Service Development Manager
2. Beta Testing Group will be managed from RT to provide visibility for all staff.
3. Beta Testing will be screened for Test Case suitability on each test case.
4. Most communication with Beta testers will be by email via RT for visibility to all staff. Calls will be documented as per best practices.
5. Each Test Case will be ticketed on the member level. Format: aa000 Beta Tester 613 <BETA - Test Case 1 - Test Goal> Service Plan
   • area code will denote the local area code from which the member has DSL service. Example: 613 denotes DSL member in Ottawa, area code 819 denotes DSL member in Gatineau
   • area code 000 denotes non-DSL member
6. Beta Test Group will be provided with clearly defined plan for each Test Case before they opt into it. Test Case Plans will include:
   • Adequate notice: provide a minimum of 1 week to opt into a Test Case
   • Test Case Goal: define what is being tested and why
   • Test Case Requirements Clear directions on what's need from them during each stage of the Test Case.
     • Schedule of Testing Activities Soft and hard copies (if necessary)
     • Test Constraints Specify if testers are allow to Tinker with test elements and if so when. E.g. for the first 24 hours please make no changes to the modem so we can measure line performance.
     • Escape Plan Describe clearly what testers should do if things go wrong (including steps to take before contacting Help Desk).
   • Feedback Solicit tester response on the Test Case and how to improve Beta Testing in General.
   • Report Provide a summary to testers on what was discover or determined during testing.

Service Disruptions during Testing

1. Answer simple Beta testing questions and update member specific tickets
2. Troubleshoot any service issues as normal and track normally. E.g. DSL No Sync issues that come up will be ticketed and tracked as a DSL No Sync issue.
3. Refer any specific Beta testing issues and concerns that you cannot address to the Service Operations Manager or Service Development Manager (depending on who has been running a particular Test Case)

Test Areas

1. Hardware - modems, routers and accessories
2. Network
3. Products and Services