【PIR API adaptor No.161、162】Migrate `paddle.vision.ops.nms` `paddle.nn.functional.one_hot` into pir #58735

ooooo-create · 2023-11-06T13:30:41Z

PR types

Others

PR changes

APIs

Description

No.161-162

新IR Python API适配升级 #58067
paddle.vision.ops.nms 单测覆盖率 2/2
paddle.nn.functional.one_hot 单测覆盖率 3/13
BadInputTestOnehotV2” 当前不支持
目前pir的OpTest逻辑还不是支持lod的处理。所以one_hot继承自OpTest的单测先不用适配了

MarioLulab · 2023-11-08T09:06:29Z

test_nn_functional_hot_op.py 里出现的报错:

AssertionError: Detect there is KernelSignature for `one_hot_v2` op, please set the `self.python_api` if you set check_dygraph = True

是因为没有在继承自 OpTest 的单测中添加 python_api 属性，需要在 setUp 中配置一下 self.python_api

ooooo-create · 2023-11-09T04:51:43Z

test_nn_functional_hot_op.py 里出现的报错:
AssertionError: Detect there is KernelSignature for `one_hot_v2` op, please set the `self.python_api` if you set check_dygraph = True
是因为没有在继承自 OpTest 的单测中添加 python_api 属性，需要在 setUp 中配置一下 self.python_api

添加之后出现和 test_one_hot_v2_op 一样的报错

1265:  .E...EEE
1265: ======================================================================
1265: ERROR: test_check_output (test_nn_functional_hot_op.TestOneHotOp)
1265: ----------------------------------------------------------------------
1265: Traceback (most recent call last):
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 49, in test_check_output
1265:     self.check_output(check_dygraph=False, check_pir=True)
1265:   File "/paddle/build/test/legacy_test/op_test.py", line 2686, in check_output
1265:     check_pir=check_pir,
1265:   File "/paddle/build/test/legacy_test/op_test.py", line 2548, in check_output_with_place
1265:     pir_checker.check()
1265:   File "/paddle/build/test/legacy_test/op_test.py", line 2149, in check
1265:     self.calculate_output()
1265:   File "/paddle/build/test/legacy_test/op_test.py", line 2335, in calculate_output
1265:     pir_outs = self.op_test._calc_pir_output(place)
1265:   File "/paddle/build/test/legacy_test/op_test.py", line 1330, in _calc_pir_output
1265:     ) = self.get_ir_input_attr_dict_and_feed(stop_gradient=True)
1265:   File "/paddle/build/test/legacy_test/op_test.py", line 1267, in get_ir_input_attr_dict_and_feed
1265:     if OpTestUtils.is_bfloat16_type(tup[1].dtype)
1265: IndexError: list index out of range

0x45f · 2023-11-09T09:32:08Z

对于one_hot的单测：单测文件内继承自OpTest的单测因为配置了x_lod参数，目前pir的OpTest逻辑还不是支持lod的处理。所以one_hot继承自OpTest的单测先不用适配了，先把继承自unittest.TestCase的单测适配一下~

ooooo-create · 2023-11-09T12:31:39Z

对于one_hot的单测：单测文件内继承自OpTest的单测因为配置了x_lod参数，目前pir的OpTest逻辑还不是支持lod的处理。所以one_hot继承自OpTest的单测先不用适配了，先把继承自unittest.TestCase的单测适配一下~

thanks~适配继承unittest.TestCase的单测有报错

1265: ======================================================================
1265: ERROR: test_api_with_depthTensor (test_nn_functional_hot_op.TestOneHotOpApi)
1265: ----------------------------------------------------------------------
1265: Traceback (most recent call last):
1265:   File "/paddle/build/python/paddle/pir_utils.py", line 115, in impl
1265:     func(*args, **kwargs)
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 136, in test_api_with_depthTensor
1265:     self._run(num_classes)
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 165, in _run
1265:     return_numpy=False,
1265:   File "/paddle/build/python/paddle/base/executor.py", line 1644, in run
1265:     return_numpy=return_numpy,
1265:   File "/paddle/build/python/paddle/base/executor.py", line 1947, in _run_pir_impl
1265:     scope,
1265:   File "/paddle/build/python/paddle/base/executor.py", line 1037, in get_pir_program_and_executor
1265:     program, fetch_list=fetch_list, fetch_var_name=fetch_var_name
1265:   File "/paddle/build/python/paddle/base/executor.py", line 511, in _add_pir_fetch_ops
1265:     global_block, fetch_list, fetch_var_name, fetch_op
1265:   File "/paddle/build/python/paddle/base/executor.py", line 430, in has_fetch_operations
1265:     "There is a fetch op in Program which will fetch variable that is not belong to fetch_targets."
1265: Exception: There is a fetch op in Program which will fetch variable that is not belong to fetch_targets.
1265: 
1265: ======================================================================
1265: FAIL: test_error (test_nn_functional_hot_op.BadInputTestOnehotV2)
1265: ----------------------------------------------------------------------
1265: Traceback (most recent call last):
1265:   File "/paddle/build/python/paddle/pir_utils.py", line 115, in impl
1265:     func(*args, **kwargs)
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 183, in test_error
1265:     self.assertRaises(TypeError, test_bad_x)
1265: AssertionError: TypeError not raised by test_bad_x
1265:

MarioLulab · 2023-11-10T02:04:42Z

对于one_hot的单测：单测文件内继承自OpTest的单测因为配置了x_lod参数，目前pir的OpTest逻辑还不是支持lod的处理。所以one_hot继承自OpTest的单测先不用适配了，先把继承自unittest.TestCase的单测适配一下~

thanks~适配继承unittest.TestCase的单测有报错

1265: ======================================================================
1265: ERROR: test_api_with_depthTensor (test_nn_functional_hot_op.TestOneHotOpApi)
1265: ----------------------------------------------------------------------
1265: Traceback (most recent call last):
1265:   File "/paddle/build/python/paddle/pir_utils.py", line 115, in impl
1265:     func(*args, **kwargs)
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 136, in test_api_with_depthTensor
1265:     self._run(num_classes)
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 165, in _run
1265:     return_numpy=False,
1265:   File "/paddle/build/python/paddle/base/executor.py", line 1644, in run
1265:     return_numpy=return_numpy,
1265:   File "/paddle/build/python/paddle/base/executor.py", line 1947, in _run_pir_impl
1265:     scope,
1265:   File "/paddle/build/python/paddle/base/executor.py", line 1037, in get_pir_program_and_executor
1265:     program, fetch_list=fetch_list, fetch_var_name=fetch_var_name
1265:   File "/paddle/build/python/paddle/base/executor.py", line 511, in _add_pir_fetch_ops
1265:     global_block, fetch_list, fetch_var_name, fetch_op
1265:   File "/paddle/build/python/paddle/base/executor.py", line 430, in has_fetch_operations
1265:     "There is a fetch op in Program which will fetch variable that is not belong to fetch_targets."
1265: Exception: There is a fetch op in Program which will fetch variable that is not belong to fetch_targets.
1265: 
1265: ======================================================================
1265: FAIL: test_error (test_nn_functional_hot_op.BadInputTestOnehotV2)
1265: ----------------------------------------------------------------------
1265: Traceback (most recent call last):
1265:   File "/paddle/build/python/paddle/pir_utils.py", line 115, in impl
1265:     func(*args, **kwargs)
1265:   File "/paddle/build/test/legacy_test/test_nn_functional_hot_op.py", line 183, in test_error
1265:     self.assertRaises(TypeError, test_bad_x)
1265: AssertionError: TypeError not raised by test_bad_x
1265:

test_api_with_depthTensor：问题的原因应该和 PIR Python API 适配升级任务的 bug 修复手册 #58259 (comment) 是一样的，按照这个修改吧
FAIL: test_error (test_nn_functional_hot_op.BadInputTestOnehotV2)：先跳过 test_errors 的 pir 测试吧，并且在 PR 描述里说明一下

paddle-ci-bot · 2023-11-19T03:20:30Z

Sorry to inform you that 7dbb662's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

MarioLulab · 2023-11-23T06:32:55Z

当前问题可以按上述方式修复 😄

ooooo-create · 2023-11-23T10:25:10Z

当前问题可以按上述方式修复 😄

好哒好哒~

… ooooo-pir161162

MarioLulab

test/legacy_test/test_zero_dim_tensor.py 文件里还有 TestNoBackwardAPIStatic.test_one_hot_label 单测没有迁移，麻烦迁移一下吧~

test/legacy_test/test_ops_nms.py

test/legacy_test/test_one_hot_v2_op.py

python/paddle/vision/ops.py

test/legacy_test/test_ops_nms.py

test/legacy_test/test_nn_functional_hot_op.py

test/legacy_test/test_one_hot_v2_op.py

into ooooo-pir161162

ooooo-create · 2023-12-01T03:37:12Z

感谢这么详细的指导~有个问题，test_one_hot_v2_op 里的 label.desc.set_need_check_feed(False)我注释掉，然后aistudio上验证没有报错，但是 ci 里好像说

是不是因为我注释掉那句话了，还是为什么，然后不注释掉，本地aistudio 上会有下面这个问题

0x45f · 2023-12-01T10:58:37Z

感谢这么详细的指导~有个问题，test_one_hot_v2_op 里的 label.desc.set_need_check_feed(False)我注释掉，然后aistudio上验证没有报错，但是 ci 里好像说是不是因为我注释掉那句话了，还是为什么，然后不注释掉，本地aistudio 上会有下面这个问题

这个问题改一下单测的代码吧，在老的静态图下才进行set_need_check_feed，在pir下把label_data 转换成int64类型~

ooooo-create · 2023-12-01T14:08:03Z

感谢这么详细的指导~有个问题，test_one_hot_v2_op 里的 label.desc.set_need_check_feed(False)我注释掉，然后aistudio上验证没有报错，但是 ci 里好像说是不是因为我注释掉那句话了，还是为什么，然后不注释掉，本地aistudio 上会有下面这个问题

这个问题改一下单测的代码吧，在老的静态图下才进行set_need_check_feed，在pir下把label_data 转换成int64类型~

好哒~，done

MarioLulab

麻烦更新一下 paddle.nn.functional.one_hot 的单测覆盖率。并且在 pr 描述中将已经解决问题的报错栈删除，方便我们后续排查~

test/legacy_test/test_ops_nms.py

test/legacy_test/test_nn_functional_hot_op.py

MarioLulab

LGTM

python/paddle/vision/ops.py

test/legacy_test/test_ops_nms.py

fix

f0c00d0

paddle-bot bot added the contributor External developers label Nov 6, 2023

luotao1 mentioned this pull request Nov 7, 2023

新IR Python API适配升级 #58067

Closed

luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Nov 7, 2023

luotao1 assigned luotao1 and MarioLulab Nov 7, 2023

update setup

c8c8c25

fix

7dbb662

ooooo-create added 2 commits November 25, 2023 09:08

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

621bf21

… ooooo-pir161162

fix

a6f8e70

MarioLulab reviewed Nov 29, 2023

View reviewed changes

ooooo-create added 2 commits November 30, 2023 12:57

fix

6d2452d

Merge branch 'ooooo-pir161162' of https://github.com/ooooo-create/Paddle

ce9f96f

into ooooo-pir161162

ooooo-create requested a review from MarioLulab November 30, 2023 13:19

fix

e15dea2

update

c64cde9

MarioLulab reviewed Dec 2, 2023

View reviewed changes

MarioLulab approved these changes Dec 4, 2023

View reviewed changes

0x45f reviewed Dec 4, 2023

View reviewed changes

python/paddle/vision/ops.py Outdated Show resolved Hide resolved

test/legacy_test/test_ops_nms.py Outdated Show resolved Hide resolved

fix

55a4b4f

0x45f approved these changes Dec 6, 2023

View reviewed changes

0x45f merged commit e8eb917 into PaddlePaddle:develop Dec 6, 2023

ooooo-create mentioned this pull request Jan 22, 2024

[WeeklyReports] 2023.12.25~2024.1.19 周报收集 PFCCLab/Starter#97

Closed

24 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【PIR API adaptor No.161、162】Migrate `paddle.vision.ops.nms` `paddle.nn.functional.one_hot` into pir #58735

【PIR API adaptor No.161、162】Migrate `paddle.vision.ops.nms` `paddle.nn.functional.one_hot` into pir #58735

ooooo-create commented Nov 6, 2023 •

edited

Loading

MarioLulab commented Nov 8, 2023

ooooo-create commented Nov 9, 2023

0x45f commented Nov 9, 2023

ooooo-create commented Nov 9, 2023

MarioLulab commented Nov 10, 2023

paddle-ci-bot bot commented Nov 19, 2023

MarioLulab commented Nov 23, 2023 •

edited

Loading

ooooo-create commented Nov 23, 2023

MarioLulab left a comment

ooooo-create commented Dec 1, 2023

0x45f commented Dec 1, 2023

ooooo-create commented Dec 1, 2023

MarioLulab left a comment

MarioLulab left a comment

【PIR API adaptor No.161、162】Migrate paddle.vision.ops.nms paddle.nn.functional.one_hot into pir #58735

【PIR API adaptor No.161、162】Migrate paddle.vision.ops.nms paddle.nn.functional.one_hot into pir #58735

Conversation

ooooo-create commented Nov 6, 2023 • edited Loading

PR types

PR changes

Description

MarioLulab commented Nov 8, 2023

ooooo-create commented Nov 9, 2023

0x45f commented Nov 9, 2023

ooooo-create commented Nov 9, 2023

MarioLulab commented Nov 10, 2023

paddle-ci-bot bot commented Nov 19, 2023

MarioLulab commented Nov 23, 2023 • edited Loading

ooooo-create commented Nov 23, 2023

MarioLulab left a comment

Choose a reason for hiding this comment

ooooo-create commented Dec 1, 2023

0x45f commented Dec 1, 2023

ooooo-create commented Dec 1, 2023

MarioLulab left a comment

Choose a reason for hiding this comment

MarioLulab left a comment

Choose a reason for hiding this comment

【PIR API adaptor No.161、162】Migrate `paddle.vision.ops.nms` `paddle.nn.functional.one_hot` into pir #58735

【PIR API adaptor No.161、162】Migrate `paddle.vision.ops.nms` `paddle.nn.functional.one_hot` into pir #58735

ooooo-create commented Nov 6, 2023 •

edited

Loading

MarioLulab commented Nov 23, 2023 •

edited

Loading