Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Feature][core] add parameters in json format #1392

Closed
3 tasks done
aubdiy opened this issue Nov 11, 2022 · 3 comments · Fixed by #1393
Closed
3 tasks done

[Feature][core] add parameters in json format #1392

aubdiy opened this issue Nov 11, 2022 · 3 comments · Fixed by #1393
Labels
feature-request this is a feature requests on the product

Comments

@aubdiy
Copy link
Contributor

aubdiy commented Nov 11, 2022

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

新增个 -pj 参数, 用于接收 json 格式的自定义参数

Use case

  • commond
bin/start-chunjun                              \
    -mode local                                \
    -jobType sync                              \
    -chunjunDistDir chunjun-dist               \
    -job stream.json                           \
    -p a=a1,b=b1                               \
    -pj '{"c":123,"d":"ddd"}'
  • stream.json
{
  "job": {
    "content": [
      {
        "reader": {
          "parameter": {
            "column": [
              {
                "name": "id",
                "type": "id"
              },
              {
                "name": "name",
                "type": "string"
              },
              {
                "name": "content",
                "type": "string"
              },
              {
                "name": "${a}_${b}_${c}_${d}",
                "type": "string"
                
              }
            ],
            "sliceRecordCount": [
              "2"
            ],
            "permitsPerSecond": 1
          },
          "table": {
            "tableName": "sourceTable"
          },
          "name": "streamreader"
        },
        "writer": {
          "parameter": {
            "column": [
              {
                "name": "id",
                "type": "id"
              },
              {
                "name": "name",
                "type": "string"
              }
            ],
            "print": true
          },
          "table": {
            "tableName": "sinkTable"
          },
          "name": "streamwriter"
        },
        "transformer": {
          "transformSql": "select id,name from sourceTable where CHAR_LENGTH(name) < 50 and CHAR_LENGTH(content) < 50"
        }
      }
    ],
    "setting": {
      "errorLimit": {
        "record": 100
      },
      "speed": {
        "bytes": 0,
        "channel": 1,
        "readerChannel": 1,
        "writerChannel": 1
      }
    }
  }
}

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@aubdiy aubdiy added the feature-request this is a feature requests on the product label Nov 11, 2022
@FlechazoW
Copy link
Member

-p 和 -pj 是做同一个事情的,用一个就好了。-p '{"c":123,"d":"ddd"}' 这样会不会更好一些呢?

@aubdiy
Copy link
Contributor Author

aubdiy commented Nov 11, 2022

'{"c":123,"d":"ddd"}' 这样会不会更好一些呢?

为了兼容历史任务, 不然升级改动太大了.
也可以在处理 '-p' 参数内容时, 根据解析 json 异常与否, 来兼容之前的写法,
但是如果这样的话, 这个兼容就永远无法下线了,
最终,
还是觉得新增一个 -pj 参数接收 json 格式数据,
-p 标记为即将失效,
然后再未来某个合适的时机, 去掉原来的 -p 功能

@FlechazoW
Copy link
Member

了解,那先做成-pj 的方式来表示是否是json内容,在下个大版本中对这部分做调整优化。

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
feature-request this is a feature requests on the product
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants