MDO Announcements - 2022
16 Nov 2022 : Automatic Pipeline Runs
Starting November 28th Cyber and Mission DevOps teams will perform daily scans of applications that are deployed. They will be checking all deployed images for the application to see if they have had a pipeline run of their master branch within the last week. If there has not been one, we will automatically kick off a pipeline run. The reason for this is because new CVEs/Vulnerabilities are discovered weekly and if your application is not going through our security scans, your application could have an exposed attack vector. Please address any new findings that may come to light from these pipeline runs.
08 Aug 2022 : Anchore
As a part of an effort to add more layers of vulnerability detection, Party Bus will be rolling out Anchore image vulnerability scanning today for deployment pipelines on 8/8 in IL2/IL4/IL5. This new feature will be added to the current image scan stage and will perform similar duties as the Twistlock scan. Thus, teams may see both scans fail in the scan-image stage for the same vulnerabilities. If this happens and the team has proper justification for whitelisting, then please open a single ticket to address vulnerabilities in both scans. That being said, this will currently be a beta feature and will be allowed to fail for the time being.
Additionally, There will also be an anchore job that generates a Software Bill of Materials (SBOM) as an artifact in order to give a more complete picture of what software is installed on each image. Note that This stage is not a gate check and should always pass unless technical issues arise
19 Jul 2022 : Dependency Check Upgrade to Version 7.1.1
Dependency Check Update to 7.1.1 on Tuesday 7/19/2022 Fix bash globbing bug that failed to handle wildcards properly such as
**/*.jar
Impact:
some pipelines may exhibit additional dependency check findings. These findings most likely already existed in the project but were obscured by the globbing bug. some pipelines may observe some false positives eliminated. See the release notes for version 7.1.1
https://github.com/jeremylong/DependencyCheck/releases
28 Jun 2022 : CtF Check
PB-MDO will be releasing a new version ctf_check that will validate CTF letter info during production deployment jobs. This new version is backwards compatible and we expect zero impact if you are already able to deploy with the current system. Please submit a help disk ticket if you encounter issues.
20 Jun 2022 : Python pylint Job
Party Bus Mission DevOps will be pushing a change to the python pylint job. This change will go into affect on Monday 6/20/2022.
This job will run the pylint command with no additional switches. Application teams will manage their own .pylintrc files in their repository to manage pylint settings.
Documentation for configuring pylint.
04 May 2022 : ClamAV
Hello folks 👋 If your app allows users to upload files, Cyber requires that they be scanned. To satisfy this requirement, we have implemented a multitenant instance of clamav for product teams to use. The service is available in IL2/4/5 in staging and production. Please see the following guide for implementation instructions, ClamAV Customer Guide.
04 May 2022 : Deploy to Staging
Party Bus Mission DevOps is planning to push a change at 1700 EST/1500 MST that involves a change in configuration for deploying product team containers. This change will be rolled out to IL2, IL4, and IL5 and will involve utilizing different tokens for manipulating the kustomization deployment manifests. These tokens are already in place and therefore projects should not see changes in their deploy to staging or release stages. This also should not affect the argo portion of the deploy stage. That being said if you experience issues with deploy-to-staging and/or releases please create a help desk ticket and we will treat it as priority.
17 Mar 2022 : Commit Signing
@here REMINDER: code.ilx.dso.mil will start enforcing code signing March 18th. Please open your Code Sign verification ticket soon. Here are the Commit Signing with CAC and Commit Signing with GPG setup articles. Please NOTE: Setup in your IDE is not supported by the MDO Team. Please test at the command line.
09 Mar 2022 Use of harden-nodejs in image builds
It has come to our attention that teams are still using the registry.ilx.dso.mil harden-nodejs image and possibly other base images within registry.ilx.dso.mil that were NOT created specifically for their team. These images were created as a stopgap and are no longer necessary for use. If you are using any of the harden images, please switch to using the most recent ironbank image instead. see this doc for guidance:
02 Mar 2022: Code Signing
On Friday March 18th, we will be turning on a setting in GitLab that will require teams to sign their commits. Commits can be signed with a GPG Key or your CAC. However, if your projects are in IL4 and IL5 you should use your CAC as we will restrict signatures to CACs only, in the future. Commits that aren't signed will be rejected through a pre-receive hook. Please follow our guides on setting up code signing: Commit Signing with GPG and Commit Signing with CAC.
28 Jan 2022: Fortify Gate-check
Tomorrow (Jan 28th), MDO will be pushing out an update to Fortify's gate-check script. This update may cause some Fortify jobs that are currently passing to fail. We will be enabling allow_failure for the Fortify job in order to give product teams time to remediate the findings.
Please take time to review your pipelines and the Fortify Job in order to determine if your team will need to address any findings. Fortify job failures will be allowed until Feb 4th.
14 Jan 2022: Turning Off allow_failure Setting
IMPORTANT ANNOUNCEMENT: Allowing failures in staging pipelines will no longer be permitted effective 4 Feb 2022. We are turning off the allow_failures = true setting. If your application currently allows failures in staging, your code will no longer be allowed to deploy.
This means that in order to deploy to staging, unit tests and lint need to pass and all compliance checks and SCA findings will have to be resolved before deploying to staging.
The only exception to this are jobs that have been recently introduced to the pipeline and are set to allow failure at a global level. These jobs are: find unauthorized, project configs, lint for gradle and maven only, e2e tests, and pen test.
This is a P1 security decision, endorsed by our ISSO, supporting the cATO.
05 Jan 2022: Default Runner Build Directory
Hello everyone. MDO will be updating the default build directory for runner jobs to be /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME. This will standardize the path across all teams and across all impact levels. The GitLab default was causing issues between jobs for some teams that relied on absolute paths.
What this means for you:
- if your project is using relative paths, you likely won't notice anything
- if your project uses absolute paths in any of your jobs (usually sonarqube), we've probably had to implement a workaround to allow continuous operation of your pipeline. We will be removing these workarounds. If any of your jobs are suddenly failing because it is unable to locate files, please let us know
- you can see what the the build directory is set to for any runner job at the beginning of the log, it looks like:
Initialized empty Git repository in <BUILD DIR>
Relevant links:
https://docs.gitlab.com/ee/ci/runners/configure_runners.html#custom-build-directorieshttps://jira.il2.dso.mil/browse/PBMD-2632