Frame requestByID panics when initializing networking

[inancgumus]

Brief summary

There is a flaky bug in (*Frame).requestByID that happens while initializing (*NetworkManager).initEvents. The test run ID is 155045.

xk6-browser version

0fcdd36

OS

Linux

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Run the following script on a resource restricted platform several times as the issue is flaky.

The script

import { browser } from 'k6/experimental/browser'
import { check } from 'k6'

export const options = {
  scenarios: {
    browser: {
      executor: 'per-vu-iterations',
      maxDuration: '15m',
      vus: 5,
      iterations: 10,
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
  }
}
export default async function () {
  const page = browser.newPage();

  try {
    // User journy in SPA application
    await page.goto('https://.../'); // TODO: change it to the otel app

    const productCard = page.locator('(//div[@data-cy="product-card"])[1]')
    productCard.click()

    const quantityOption = page.locator('[data-cy="product-quantity"]')
    quantityOption.selectOption('3')
    const addToCardBtn = page.locator('[data-cy="product-add-to-cart"]')
    addToCardBtn.click()

    check(page, {
      'cart item name': page => page.locator('//p[text()="National Park Foundation Explorascope"]').isVisible() === true,
      'cart item quantity': page => page.locator('//p[text()="3"]').isVisible() === true
    })

    // Check GH link to force another navigation
    await Promise.all([
      page.waitForNavigation(),
      page.locator('footer a').click(),
    ]);
    check(page, {
      'gh-author': page.locator(
        'div#repository-container-header span.author a'
      ).textContent().includes('open-telemetry'),
    });
  } finally {
    page.close();
  }
}

Expected behaviour

No panics.

Actual behaviour

goroutine 246996 [running]:
github.com/grafana/xk6-browser/common.(*Frame).requestByID(0xc0089428c0, {0xc009718454, 0x7})
	github.com/grafana/xk6-browser@v1.0.0-rc0.0.20230726091449-0fcdd3618f29/common/frame.go:365 +0x5a
github.com/grafana/xk6-browser/common.(*FrameManager).requestFailed(0xc001d700c0, 0xc0055d4640, 0x0)
	github.com/grafana/xk6-browser@v1.0.0-rc0.0.20230726091449-0fcdd3618f29/common/frame_manager.go:450 +0x4d7
github.com/grafana/xk6-browser/common.(*NetworkManager).onLoadingFailed(0xc008dd3ea0, 0xc007489200)
	github.com/grafana/xk6-browser@v1.0.0-rc0.0.20230726091449-0fcdd3618f29/common/network_manager.go:345 +0x116
github.com/grafana/xk6-browser/common.(*NetworkManager).handleEvents(0xc008dd3ea0, 0xc004206960)
	github.com/grafana/xk6-browser@v1.0.0-rc0.0.20230726091449-0fcdd3618f29/common/network_manager.go:318 +0x288
github.com/grafana/xk6-browser/common.(*NetworkManager).initEvents.func1()
	github.com/grafana/xk6-browser@v1.0.0-rc0.0.20230726091449-0fcdd3618f29/common/network_manager.go:301 +0x37
created by github.com/grafana/xk6-browser/common.(*NetworkManager).initEvents
	github.com/grafana/xk6-browser@v1.0.0-rc0.0.20230726091449-0fcdd3618f29/common/network_manager.go:300 +0x14d

Tasks

Preview Give feedback

1. Add nil checks in requestByID #1220
   +0
   
2. Remove logging in-flight requests #1246
   maintenance +1
   
3. Add browser 986 k6#3652
   +0
   

[DefCon-007]

Reproducible when multiple browser VUs are assigned i.e. multiple browser containers are initialised

[ankur22]

While taking a look at this discussion here, I looked through the code again and it does indeed seem that the issue lies on line 412 which would indicate that the original frameSession from line 408 was nil, or its networkManager were nil.

I think the best course of action is:

1. Protect against NPD by ensuring that both the mainFrameSession and its networkManager are not nil, and log a warning when either of them are.
2. Replicate the issue to diagnose exactly why we get into this state. I currently cannot see any reason for the mainFrameSession to be nil on a page. When a page is created, the mainFrame is also created, along with that the networkManager is also created. So how does a page end up with a nil mainFrame or how does the mainFrame end up with a nil networkManager?

[inancgumus]

I agree that we need to first determine which one (or both) is nil. We can add a simple logger to determine 👍

[ankur22]

As a step towards understanding the root cause we've created #1220 to help us identify exactly which component is nil. We might be in a better position to find the root cause of the issue once we see the logs.

[inancgumus]

Closed by #1219.

[ankur22]

We've seen more cases of this even after #1219. Reopening.

[inancgumus]

Happened in test run ID: 2480444 (prod).

[joanlopez]

Happened in test run ID: 2480444 (prod).

Another occurrence in test run ID: 2539482 (prod).