Testing Phase I Integrations

[qcorporation]

Security Integrations targeted for the initial release for the 8.18 release, are:

Office 365
Okta
AWS Security Hub
SentinelOne
AbuseCH
Microsoft Defender Cloud
Microsoft 365 Defender
Microsoft Defender for Endpoint
Google Security Command Center
Google Workspace
Tenable IO
Wiz
Qualys VMDR

Stretch Goal:
Microsoft Sentinel

Description / Task

Based upon the availability of credentials from #11811, test each integration for all supported inputs if they can:

• ingest all data streams supported for that integration
• handle pod restarts and cursor resumes (validate that no duplicate data is in ES)

Test Plan

1. Using devenv to setup an agentless setup locally using Aleks comment as a reference https://github.com/elastic/security-team/issues/8883#issuecomment-2192911250

• This potentially will require you to add permissions for to k8s-gitops-control-plane, an example PR would be here: https://github.com/elastic/catalog-info/pull/232

2. Override the agent image to one that was before the persistent storage @olegsu can help with finding a hash
3. Push the Okta integration with these modification to the manifest.yml to your local setup

policy_templates:
  - name: okta
    title: Okta logs
    description: Collect logs from Okta
    deployment_modes:
      default:
        enabled: true
      agentless:
        enabled: true
        organization: elastic
        division: engineering
        team: security

3a. You can use these instructions https://www.elastic.co/guide/en/integrations-developer/current/elastic-package.html#_customization so you can elastic-package build and elastic-package install --zip <okta.build.zip>
Use the ELASTIC_PACKAGE_KIBANA_HOST, ELASTIC_PACKAGE_ELASTICSEARCH_HOST, and ELASTIC_PACKAGE_ELASTICSEARCH_API_KEY settings

4. I will send you credentials for the different systems
5. You need to create a script that will scrape data pre-pod restart and then after pod-restart and validate that there is duplicate data within elasticsearch
6. Now override agent image, docker.elastic.co/cloud-release/elastic-agent-service:sha256-ff43ceebf9971d80937beb91175ab39f8cc3e99c680421673a8db3e49a25f23c
7. Make sure you clear the data from elasticsearch
8. Test again steps 5 but now validate that the data is NOT duplicate. This means that the cursor has been saved across pod restarts

[elasticmachine]

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)