Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add note about using dag_run.conf in BashOperator #9143

Merged
merged 3 commits into from
Jun 8, 2020

Conversation

ashb
Copy link
Member

@ashb ashb commented Jun 4, 2020


Make sure to mark the boxes below before creating PR: [x]

  • Description above provides context of the change
  • Unit tests coverage for changes (not needed for documentation changes)
  • Target Github ISSUE in description if exists
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions.
  • I will engage committers as explained in Contribution Workflow Example.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

@ashb ashb requested a review from kaxil June 4, 2020 14:00
@ashb ashb force-pushed the update-example-trigger-dag branch from 08d2452 to d10f545 Compare June 4, 2020 14:43
Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc building error

==================== Error   2 ====================
 WARNING: Could not lex literal_block as "python". Highlighting skipped.

File path: /opt/airflow/docs/howto/operator/bash.rst (67)

  63 | 
  64 | Instead, you should pass this via the ``env`` kwarg and use double-quotes
  65 | inside the bash_command, as below:
  66 | 
  67 | .. code-block:: python
  68 | 
  69 |     bash_task = BashOperator(
  70 |         task_id="bash_task",
  71 |         bash_command='echo "here is the message: \\'$message\\'"',
  72 |         env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},
==================================================

@ashb
Copy link
Member Author

ashb commented Jun 5, 2020

Hmmmm, I wonder what is going on between sphinx and pygmentize:

airflow ❯ pygmentize -l python -F raiseonerror < test-lex.py
    bash_task = BashOperator(
        task_id="bash_task",
        bash_command='echo "here is the message: \'$message\'"',
        env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},

works fine and doesn't error.

@ashb
Copy link
Member Author

ashb commented Jun 5, 2020

(I tried with \ first, that failed, and with \\ too wondering if sphinx was treating \\ as something. That failed too)

@ashb ashb merged commit 4d8599e into apache:master Jun 8, 2020
@ashb ashb deleted the update-example-trigger-dag branch June 8, 2020 08:33
@ashb ashb added this to the Airflow 1.10.11 milestone Jun 8, 2020
ashb added a commit that referenced this pull request Jun 8, 2020
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 15, 2020
(cherry picked from commit 4d8599e)
(cherry picked from commit 74d6d1d)
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 15, 2020
(cherry picked from commit 4d8599e)
(cherry picked from commit 74d6d1d)
(cherry picked from commit 2591294)
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 15, 2020
(cherry picked from commit 4d8599e)
(cherry picked from commit 74d6d1d)
(cherry picked from commit 2591294)
@kaxil kaxil added the type:doc label Jul 1, 2020
kaxil pushed a commit that referenced this pull request Jul 1, 2020
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants