Party Bus Mission DevOps & Ops Support Levels
Overview
Use this document to learn the MDO and Operations support levels.
Customer Support Models
Below are the support models that MDO uses to quickly address product team needs. Party Bus MDO is not a 24/7 support organization. We do not support mission application 24/7 requirements. The MDO team provides support from 0800-1800 Central Standard Time, Monday-Friday only. MDO will address any support requests outside of business hours the next business day.
DISCLAIMER
Party Bus will only provide support if it is demonstrated that the issue is with the Party Bus platform, and not with application team-managed resources or Iron Bank images.
Tiered Support
Tiered support escalates issues to personnel with the technical expertise to resolve them. The PB Ops team ensures tickets are immediately escalated when needed and worked promptly when placed in a new queue, preventing customers from experiencing longer queue times and ultimately creating a better support experience.
Swarming Support
Swarming IT support functions by allowing engineers to collaborate on an issue until it's resolved- issues that are best suited for swarming support are complex and involve multiple steps. Swarming support teams proactively monitor incoming tickets that may be best resolved this way, then work to resolve them as quickly as possible. The goal is to acquire the correct resources and to resolve the issue as quickly as possible in conjunction with fellow team members.
Hybrid Tiered/Swarm Support Model
A Hybrid Tiered/Swarm support model allows for tickets to be assigned at a Tiered level and swarmed when needed. As issues leave the Tier 1 or Tier 2 support level, they are assigned to the Tier 3 or Tier 4 support level and worked on solo or in collaboration, as needed, until resolved.
Tiered MDO Support Levels
| Support Level | Function | Swarm? | Team | Escalation | Outage Response Time | SLA Time to Initial Response (Working Hours) | SLA Time to Resolved (After Initial Response) (Working Hours) |
|---|---|---|---|---|---|---|---|
| Tier 0 Automation and Self-Service | Users have identified an issue and conduct independent research and information retrieval (i.e., self-help). Users enter search criteria in the P1 Help Desk search bar to identify potential resolutions to their issue. | No | Self Service (user) | If the user is unable to resolve their issue through independent research, they should open a support ticket. | 0800-1800 CST, M-F | N/A | N/A |
| Tier 1 | After a user has opened a ticket, this first level of support addresses the issue with generalist knowledge. Support at this level has no in-depth technical knowledge. | No | Razor Crest | Ticket is opened with the P1 Help Desk. | 0800-1800 CST, M-F | N/A | N/A |
| Tier 2 | If Tier 1 is unable to resolve or the ticket is triaged as Tier 2, Tier 2 serves as the second level of support and assists in resolving the issue. This may require teaming up with developers to research and resolve the issue. NOTE: Swarming may be required at this level, but it is not typical. | No | Razor Crest/MDO | - Elevated from Tier 1 or Ticket is triaged immediately as Tier 2. - May need to swarm at this level. | Immediate 0800-1800 CST, M-F | Staging < 16 hours Production < 8 hours | Staging < 24 hours Production < 24 hours |
| Tier 3 CI/CD Changes, Manifest Changes, Overlay Changes | If Tiers 1 and 2 are unable to resolve or the ticket is triaged as Tier 3, this third level of support comprises expert product and service-level personnel, including developers and other product specialists. This tier may require teaming up with developers to research and resolve the issue. Subject to internal prioritization. | Yes | MDO/PB Ops | - Elevated from Tier 2 or Ticket is triaged immediately as Tier 3. - This is where the MDO and PB Ops teams swarm on an issue. | < 1 hour PB Ops Support 0800-1800 CST M-F | < 16 hours | Depends on the type of issue. May be a new feature request subject to prioritization by the development team. No guarantee on timeframe for resolution, or resolution at all. |
| Tier 4 Architectural Changes Beyond CI/CD, Vendor Intervention | Tier 4 comprises internal expert support or external vendor support that goes far beyond the norm. If an issue has reached this tier, it is because it requires access to the platform's underlying architecture. | Yes | PB Ops | - Tier 4 is the last line of defense. - External Vendor Support or external to Party Bus support | < 5 days | Depends on the type of issue. May be a new feature request subject to prioritization by the development team. | No guarantee on timeframe for resolution, or resolution at all. |
| Tier 5 Beyond Help Desk - Request for Feature | Brand new integrations to the service catalog/infrastructure Strategies to enable and actual applications fall under new Customer Feature Request . Tickets that enter this queue may be denied due to internal deliberation and/or cybersecurity posture. | N/A | Party Bus Leadership | Upvoting places the feature request at a higher priority because it shows multiple customers would like to see this feature implemented. | TBD 7 a.m. to 5 p.m. EST | < ? hours | < ? hours |
| SCA MDO | Trufflehog, Twistlock, and Anchore findings. | N/A | MDO | - Triaged immediately. - All submitted issues are resolved same day. - SCA issues that can be resolved in 5-15 minutes. | Immediate 0800-1800 CST M-F | < 8 hours | < 3 hours |
| SCA CAT | GitLab SAST, Sonarqube, Dependency Check, and Pen Test findings | N/A | CAT | 0800-1800 CST M-F | < 24 hours | < 8+ hours |
DISCLAIMER
Time to resolution may vary based on issues/blockers found during the research process.
NOTE
Frequent software package upgrades for Big Bang tools may cause outages or service interruptions. In accordance with our ATO, we are required to be an N-1 software version organization, meaning we must maintain the latest software versions and packages for our Cyber scanning tools at all times. P1 provides announcements via multiple communication channels to notify users of these upgrades and potential outages, and includes a detailed README of all changes. While Big Bang upgrades are rigorously tested, not all issues are captured. This may cause an outage that requires Party Bus to respond within the Tier 3 SLA timeframe.
Triage
Triage managers perform the initial review of tickets and place them into the appropriate queues. The primary intent is for the triage manager to get a feel for the types and quantities of tickets received and to add automation to better serve the customer's needs.
Pipeline/Deployment Support
The MDO team supports IL2/IL4/IL5 deploys only. SIPR/AFSCI pipeline support may differ from the table below. Please contact the Classified Ops team through their Mattermost channel .
Party Bus product teams are expected to keep their code base up to date by actively running production pipelines at least 1x per week.
| Request Type | Escalation | SLA Time to Initial Response | SLA Time to Resolve (After Initial Response) | Notes |
|---|---|---|---|---|
| Pipeline Request | MDO PM PB Ops PM | 1-2 weeks | 1-2 weeks 0800-1800 CST, M-F | Includes staging deploy configurations |
| Auxiliary Deployment Request | MDO PM PB Ops PM | 1-2 weeks | 1-2 weeks 0800-1800 CST, M-F | For large applications, the time to resolve will vary greatly |
| Production Deploy Request | MDO PM PB Ops PM | 1-3 days | 1-2 days 0800-1800 CST, M-F | Requires signed CtF letter |
| CTF Renewal/Extension Request | MDO PM PB Ops PM | 1-3 days | 1-2 days 0800-1800 CST, M-F | Requires signed CtF letter |
DISCLAIMER
Time to resolution may vary based on issues/blockers found during the pipeline creation process.
Support is provided 0800-1800 Central Standard Time.
Compute and Storage Thresholds
This table defines the compute and storage limitations and thresholds for Party Bus Application Teams. If these are insufficient for an application, Party Bus product teams will be subject to an audit and potential fees.
| Type | Threshold | Notes |
|---|---|---|
| Staging/Production Deployment Pod Resource defaults | limit cpu: .5 limit memory: 256mb request cpu: .1 request memory: 64 mb | |
| Redis/ELK/MongoDB PVC | limit cpu: .5 limit memory: 256mb request cpu: .1 request memory: 64MB Storage limit: 50GB ELK (MDO PROD IL2) | No High Availability |
| S3 | Total storage: No limit Single file size limit: | Total storage: typically a 1-2 TB |
| Pipeline Artifacts | 2000MB max | Expires in 7 days. Used to store build artifacts and small files |
| Pipeline Jobs | cpu: 4 cores limit memory: 10GB request cpu: 0.25 cores request memory: 1GB | All pipeline jobs |
| Postgres/MySQL RDS | 200GB max |
Product Team Support Levels
This table defines the Support Levels Party Bus product teams can choose when onboarding or renewing. If the support hours are insufficient for an application team, Party Bus product teams will be subject to an audit and potential fees.
| Level | Hour Limit | Additional Benefits | Notes |
|---|---|---|---|
| Platinum | 451 | P1 partner Priority 1 support response (automatic priority escalation over other tickets of the same type) 1:1 support and conference call assistance provided as needed | Required if application is identified as Complexity 4 |
| Gold | 290 | With Party Bus leadership approval, 1:1 support and conference call assistance (case by case) | Default recommendation if application is identified as Complexity 3 |
| Silver | 173 | With Party Bus leadership approval, an initial 1:1 conference call assistance | Default recommendation if application is identified as Complexity 2 Minimum support level if application requires dedicated database |
| Bronze | 32 | Help desk ticket support only No 1:1 conference call assistance | Default recommendation if application is identified as Complexity 1 |
Related Content/References
Submit Requests to the Help Desk
- Request Jira/Confluence Support
- Request Party Bus Pipeline Support
- Request a Customer Feature
- Request Access