add lag member failed on mellanox platform

[lguohan]

Apr 14 19:40:46 str-msn2700-04 NOTICE orchagent: :- addLagMember: Add member Ethernet120 to LAG PortChannel03 lid:20000000005a1 pid:100000000000f
Apr 14 19:40:46 str-msn2700-04 ERR syncd: :- handle_generic: failed to create -1
Apr 14 19:40:46 str-msn2700-04 ERR syncd: :- processEvent: failed to execute api: create, key: SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005b8, status: SAI_STATUS_FAILURE
Apr 14 19:40:46 str-msn2700-04 ERR syncd: :- processEvent: field: SAI_LAG_MEMBER_ATTR_LAG_ID, value: oid:0x20000000005a1
Apr 14 19:40:46 str-msn2700-04 ERR syncd: :- processEvent: field: SAI_LAG_MEMBER_ATTR_PORT_ID, value: oid:0x100000000000f
Apr 14 19:40:46 str-msn2700-04 NOTICE syncd: :- exit_and_notify: sending switch_shutdown_request notification to OA
Apr 14 19:40:46 str-msn2700-04 NOTICE orchagent: :- handle_switch_shutdown_request: switch shutdown request
Apr 14 19:40:46 str-msn2700-04 ERR orchagent: :- on_switch_shutdown_request: Syncd stopped
Apr 14 19:40:46 str-msn2700-04 INFO swss.sh[2648]: terminate called without an active exception

[lguohan]

Please find the attached sai record.

sairedis.2017-04-14.10_30_11.766709.txt

[lguohan]

it looks like we are recreating the LAG_MEMBER for the same lag_id and port_id pair.

2017-04-14.10:30:34.146625|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005a6|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x20000000005a1|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x100000000000f


2017-04-14.19:40:46.847206|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005b8|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x20000000005a1|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x100000000000f

[stcheng]

2017-04-14.10:30:15.085076|c|SAI_OBJECT_TYPE_LAG:oid:0x200000000059d|NULL=NULL
2017-04-14.10:30:15.176111|c|SAI_OBJECT_TYPE_LAG:oid:0x200000000059e|NULL=NULL
2017-04-14.10:30:15.226323|c|SAI_OBJECT_TYPE_LAG:oid:0x20000000005a1|NULL=NULL
2017-04-14.10:30:15.237719|c|SAI_OBJECT_TYPE_LAG:oid:0x20000000005a2|NULL=NULL
2017-04-14.10:30:34.041837|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005a5|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x20000000005a2|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x1000000000010
2017-04-14.10:30:34.146625|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005a6|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x20000000005a1|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x100000000000f
2017-04-14.10:30:34.316485|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005a7|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x200000000059e|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x100000000000e
2017-04-14.10:30:34.589054|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005a8|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x200000000059d|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x100000000000d
2017-04-14.19:40:44.314727|r|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005a6
2017-04-14.19:40:46.847206|c|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1a0000000005b8|SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x20000000005a1|SAI_LAG_MEMBER_ATTR_PORT_ID=oid:0x100000000000f

The SAI function call looks fine. After the creation of all LAGs, it first removes one LAG member and then re-create it. I will double check with the latest build.

[lguohan]

@marian-pritsak

[stcheng]

@liatgrozovik hey Liat, could you also help to take a look at this? I notice that attaching/detaching members of a LAG will cause creation failure.

[marian-pritsak]

Tried on HEAD.220-68fcc36

Commands:

teamdctl PortChannel0 port remove Ethernet0
teamdctl PortChannel0 port add Ethernet0

syncd does not crash, LAG is still working after member is added back to LAG

[stcheng]

@marian-pritsak i tried also and it seems that everything is working well. I will double check nightly test to see if it is still an issue or I will close it.

[lguohan]

what is the root cause for this?