[vs tests] Disable NAT tests to unblock Jenkins

[daall]

Signed-off-by: Danny Allen daall@microsoft.com

What I did
Disabled NAT tests.

Why I did it
All Jenkins builds in swss and utilities have been failing for the past 4 days with the following error:

            if started > timeout:
>               raise ValueError(out)
E               ValueError: arp_update                       STOPPED   Not started
E               bgpd                             STOPPED   Not started
E               buffermgrd                       RUNNING   pid 594, uptime 0:01:10
E               fpmsyncd                         RUNNING   pid 390, uptime 0:01:22
E               intfmgrd                         RUNNING   pid 547, uptime 0:01:16
E               nbrmgrd                          RUNNING   pid 602, uptime 0:01:09
E               neighsyncd                       RUNNING   pid 344, uptime 0:01:25
E               orchagent                        RUNNING   pid 151, uptime 0:01:28
E               portmgrd                         RUNNING   pid 414, uptime 0:01:18
E               portsyncd                        RUNNING   pid 189, uptime 0:01:27
E               redis-server                     RUNNING   pid 87, uptime 0:01:30
E               restore_neighbors                STOPPED   Not started
E               rsyslogd                         RUNNING   pid 53, uptime 0:01:32
E               sflowmgrd                        RUNNING   pid 615, uptime 0:01:06
E               start.sh                         EXITED    Jan 22 10:29 PM
E               staticd                          RUNNING   pid 581, uptime 0:01:12
E               syncd                            RUNNING   pid 119, uptime 0:01:29
E               teammgrd                         RUNNING   pid 398, uptime 0:01:20
E               teamsyncd                        RUNNING   pid 367, uptime 0:01:24
E               vlanmgrd                         RUNNING   pid 559, uptime 0:01:15
E               vrfmgrd                          RUNNING   pid 406, uptime 0:01:19
E               vxlanmgrd                        RUNNING   pid 605, uptime 0:01:08
E               zebra                            RUNNING   pid 572, uptime 0:01:13

conftest.py:288: ValueError

This is because natmgrd and natsyncd were added to the process list that check_ready() is searching for to verify that the vs docker is up and running. However, the PR that activates these two processes in the SONiC image (sonic-net/sonic-buildimage#3494) is not merged yet. This PR needs to be merged before conftest can be updated and the NAT tests can be enabled.

How I verified it
I ran the tests locally and they now proceed as normal.

Details if related
Opened #1178 for tracking.

[yxieca]

Why is it necessary to comment out the actual test file as well? Would comment out test invoke enough?

Is there a better way of commenting out the whole file, like ''' / '''?

If the intention is to revert this change as whole when the buildimage change is in, then I would say delete the affected test files for now.

[daall]

Why is it necessary to comment out the actual test file as well? Would comment out test invoke enough?

Is there a better way of commenting out the whole file, like ''' / '''?

If the intention is to revert this change as whole when the buildimage change is in, then I would say delete the affected test files for now.

Actually, pytest has a built-in skip fixture that we could use for now. That might be the best option as we can provide a reason for the skip, so if we look at the test output it will say something like "SKIPPED - Requires changes in buildimage." I feel like this is a good compromise as we'll still have a very obvious record of these tests existing, but the test suite won't be negatively impacted.