Skip to content
This repository has been archived by the owner on May 29, 2024. It is now read-only.

Rename PyVAST Threat Bus to VAST Threat Bus #159

Merged
merged 1 commit into from
Aug 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/python-egg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,13 +74,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USER }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
tags: latest,${{ github.sha }}
- name: Publish pyvast-threatbus Docker Image
- name: Publish vast-threatbus Docker Image
uses: elgohr/Publish-Docker-Github-Action@3.04
with:
name: tenzir/pyvast-threatbus
name: tenzir/vast-threatbus
username: ${{ secrets.DOCKERHUB_USER }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
dockerfile: docker/pyvast-threatbus/Dockerfile
dockerfile: docker/vast-threatbus/Dockerfile
tags: latest,${{ github.sha }}
- name: Publish stix-shifter-threatbus Docker Image
uses: elgohr/Publish-Docker-Github-Action@3.04
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ A threat intelligence dissemination layer for open-source security tools.
- **Official Plugins**: We maintain many plugins right in the official Threat
Bus repository. Check out our integrations for [MISP][misp], [Zeek][zeek],
[CIFv3][cif], and generally apps that connect via [ZeroMQ][zmq], like
[pyvast-threatbus][pyvast-threatbus] and our
[vast-threatbus][vast-threatbus] and our
[OpenCTI connector][opencti-connector].

- **Snapshotting**: The snapshot feature allows subscribers to directly request
Expand Down Expand Up @@ -256,7 +256,7 @@ Threat Bus comes with a [3-clause BSD license][license-url].
[misp-zmq-config]: https://github.com/MISP/misp-book/tree/master/misp-zmq#misp-zeromq-configuration
[plugin-types]: https://docs.tenzir.com/threatbus/plugins/overview
[plugin-development]: https://docs.tenzir.com/threatbus/plugins/plugin-development
[pyvast-threatbus]: https://github.com/tenzir/threatbus/tree/master/apps/vast
[vast-threatbus]: https://github.com/tenzir/threatbus/tree/master/apps/vast

[pypi-badge]: https://img.shields.io/pypi/v/threatbus.svg
[pypi-url]: https://pypi.org/project/threatbus
Expand Down
5 changes: 5 additions & 0 deletions apps/vast/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ Every entry has a category for which we use the following visual abbreviations:

## Unreleased

- ⚡️ We renamed PyVAST Threat Bus to VAST Threat Bus for clarity. The PyPI
package name and the binary name change from `pyvast-threatbus` to
`vast-threatbus` accordingly.
[#159](https://github.com/tenzir/threatbus/pull/159)

- 🎁 Live matching with VAST works again!
[#156](https://github.com/tenzir/threatbus/pull/156)

Expand Down
33 changes: 17 additions & 16 deletions apps/vast/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
PyVAST Threat Bus App
=====================
VAST Threat Bus App
===================

Threat Bus is a publish-subscribe broker for threat intelligence. It is expected
that applications register themselves at the bus. Since VAST cannot do so on its
own (yet), `pyvast-threatbus.py` implements that functionality in the meantime.
Threat Bus is a publish-subscribe broker for threat intelligence. It is
expected that applications register themselves at the bus. Since VAST cannot do
so on its own (yet), `vast-threatbus.py` implements that functionality in the
meantime.

The application provides a thin layer around
[PyVAST](https://docs.tenzir.com/vast/python-bindings/overview), VAST's Python
Expand All @@ -12,12 +13,12 @@ instance.

## Installation

Install `pyvast-threatbus` via `pip`. Optionally, use a virtual environment.
Install `vast-threatbus` via `pip`. Optionally, use a virtual environment.

```
virtualenv venv # optional
source venv/bin/activate # optional
python -m pip install pyvast-threatbus
python -m pip install vast-threatbus
```

## Development
Expand Down Expand Up @@ -47,7 +48,7 @@ Bus, or pass a path to configuration file via `-c /path/to/config.yaml`.
Start the application:

```sh
pyvast-threatbus
vast-threatbus
```

### Docker
Expand All @@ -56,20 +57,20 @@ You can also run this app via Docker.

- Build it:
```
docker build . -t tenzir/pyvast-threatbus:latest
docker build . -t tenzir/vast-threatbus:latest
```
- Run it to print the helptext.
```
docker run tenzir/pyvast-threatbus:latest
docker run tenzir/vast-threatbus:latest
```
- Run and mount a custom config file into the container:
```
docker run --net=host -v /path/to/your/conf.yaml:/opt/tenzir/threatbus/pyvast-threatbus/config.yaml tenzir/pyvast-threatbus:latest -c config.yaml
docker run --net=host -v /path/to/your/conf.yaml:/opt/tenzir/threatbus/vast-threatbus/config.yaml tenzir/vast-threatbus:latest -c config.yaml
```

## Features

This section explains the most important features of `pyvast-threatbus`.
This section explains the most important features of `vast-threatbus`.

### IoC Matching

Expand All @@ -82,13 +83,13 @@ VAST's
[live matching](https://docs.tenzir.com/vast/features/threat-intel-matching)
works as
[continuous query](https://docs.tenzir.com/vast/cli/vast/export/#documentation).
`pyvast-threatbus` subscribes to those continuous query results and reports all
`vast-threatbus` subscribes to those continuous query results and reports all
new IoC matches from VAST to Threat Bus as `Sightings`. You can enable live
matching in the config file by setting `live_match: true`.

#### Retro Matching

`pyvast-threatbus` supports retro matching. You can enable it in the config file
`vast-threatbus` supports retro matching. You can enable it in the config file
by setting `retro_match: true`. This instructs the application to translate IoCs
from Threat Bus to normal VAST queries instead of feeding the IoCs to a live
matcher.
Expand All @@ -99,7 +100,7 @@ VAST by setting the config option `retro_match_max_events` to a positive integer

### Sighting Context Transformation

You can configure `pyvast-threatbus` to invoke another program for parsing
You can configure `vast-threatbus` to invoke another program for parsing
Sighting `context` data via the config option `transform_context`.

If set, the app translates the `x_threatbus_sighting_context` field of a STIX-2
Expand All @@ -122,7 +123,7 @@ actually matched IoC.

### Custom Sinks for Sightings

`pyvast-threatbus` offers to send Sighting context to a configurable `sink`
`vast-threatbus` offers to send Sighting context to a configurable `sink`
_instead_ of reporting them back to Threat Bus. This can be configured via the
`sink` configuration parameter. The special placeholder `STDOUT` can be used to
print the Sighting context to `STDOUT`.
Expand Down
2 changes: 1 addition & 1 deletion apps/vast/config.yaml.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ logging:
console_verbosity: INFO
file: false
file_verbosity: INFO
filename: pyvast-threatbus.log
filename: vast-threatbus.log

metrics:
interval: 10 # set to 0 to disable metrics
Expand Down
6 changes: 3 additions & 3 deletions apps/vast/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
],
description="Connect the open source telemetry engine VAST with Threat Bus, the open source threat intelligence dissemination layer",
entry_points={
"console_scripts": ["pyvast-threatbus=pyvast_threatbus.pyvast_threatbus:main"]
"console_scripts": ["vast-threatbus=vast_threatbus.vast_threatbus:main"]
},
include_package_data=True,
install_requires=[
Expand All @@ -47,8 +47,8 @@
license="BSD 3-clause",
long_description=long_description,
long_description_content_type="text/markdown",
name="pyvast-threatbus",
packages=["pyvast_threatbus"],
name="vast-threatbus",
packages=["vast_threatbus"],
python_requires=">=3.7",
setup_requires=["setuptools", "wheel"],
url="https://github.com/tenzir/threatbus",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from typing import Tuple, Union
import logging

logger_name = "pyvast-threatbus"
logger_name = "vast-threatbus"
logger = logging.getLogger(logger_name)

vast_ioc_type_map = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
import time
import zmq

logger_name = "pyvast-threatbus"
logger_name = "vast-threatbus"
logger = logging.getLogger(logger_name)
matcher_name = None
# List of all running async tasks of the bridge.
Expand Down Expand Up @@ -82,7 +82,7 @@ def validate_config(config: Settings):
when=Validator("logging.file", eq=True),
default="INFO",
),
Validator("logging.filename", default="pyvast-threatbus.log"),
Validator("logging.filename", default="vast-threatbus.log"),
Validator("vast", default="localhost:42000"),
Validator("vast_binary", default="vast"),
Validator("threatbus", default="localhost:13370"),
Expand Down Expand Up @@ -261,7 +261,7 @@ async def write_metrics(every: int, to: str):
@param to the filepath to write to
"""
while True:
line = f"pyvast-threatbus,host={socket.getfqdn()} "
line = f"vast-threatbus,host={socket.getfqdn()} "
start_length = len(line)
for m in metrics:
if not m.is_set:
Expand Down Expand Up @@ -314,7 +314,7 @@ async def receive(pub_endpoint: str, topic: str, indicator_queue: asyncio.Queue)
continue
# the topic is suffixed with the message type
if not topic.endswith("indicator"):
# pyvast-threatbus is not (yet) interested in Sightings or SnapshotRequests
# vast-threatbus is not (yet) interested in Sightings or SnapshotRequests
logger.debug(f"Skipping unsupported message: {msg}")
continue
await indicator_queue.put(msg)
Expand Down Expand Up @@ -789,7 +789,7 @@ def main():
config = Dynaconf(
settings_files=settings_files,
load_dotenv=True,
envvar_prefix="PYVAST_THREATBUS",
envvar_prefix="VAST_THREATBUS",
)

try:
Expand Down Expand Up @@ -828,7 +828,7 @@ def main():
if user_exit:
# Tasks were cancelled because the user stopped the app.
return
logger.info("Restarting pyvast-threatbus ...")
logger.info("Restarting vast-threatbus ...")


if __name__ == "__main__":
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ COPY threatbus threatbus
RUN python3 -m pip install .

# Install the app.
WORKDIR /opt/tenzir/threatbus/pyvast-threatbus
WORKDIR /opt/tenzir/threatbus/vast-threatbus
COPY apps/vast/setup.py .
COPY apps/vast/README.md .
COPY apps/vast/pyvast_threatbus pyvast_threatbus
COPY apps/vast/vast_threatbus vast_threatbus
RUN python3 -m pip install .

RUN echo "Adding threatbus user" && useradd -m -d /home/threatbus --user-group threatbus
RUN chown -R threatbus .
USER threatbus:threatbus

ENTRYPOINT ["pyvast-threatbus"]
ENTRYPOINT ["vast-threatbus"]
2 changes: 1 addition & 1 deletion plugins/apps/threatbus_zmq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ pip install threatbus-zmq
## Configuration

The plugin uses ZeroMQ to communicate with applications, like
[pyvast-threatbus](https://github.com/tenzir/threatbus/tree/master/apps/vast).
[vast-threatbus](https://github.com/tenzir/threatbus/tree/master/apps/vast).
The plugin serves three ZeroMQ endpoints to connect with. One endpoint for
managing subscriptions (and thus snapshot requests). The other two endpoints
exist for pub-sub operations.
Expand Down
24 changes: 12 additions & 12 deletions systemd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@ Threat Bus Systemd Unit
=======================

This folder provides `systemd` service units to run Threat Bus and
`pyvast-threatbus` as system services. The services are sandboxed and run with
`vast-threatbus` as system services. The services are sandboxed and run with
limited privileges.

## Prepare the Host System

The systemd unit declares a private user. Hence, all logs go to
`/var/log/private` by default. The following section explains how to configure
file-logging for Threat Bus and `pyvast-threatbus`. Skip the following
file-logging for Threat Bus and `vast-threatbus`. Skip the following
instructions if you configure your applications to use console-logging.

Find the `logging` config section at the top of your Threat Bus or
`pyvast-threatbus` configuration file and change it to use the private log
`vast-threatbus` configuration file and change it to use the private log
directory:

- `/var/log/private/threatbus/threatbus.log` (Threat Bus)
- `/var/log/private/pyvast-threatbus/pyvast-threatbus.log` (`pyvast-threatbus`)
- `/var/log/private/vast-threatbus/vast-threatbus.log` (`vast-threatbus`)

See the following YAML snippet for a configuration example.

Expand All @@ -34,9 +34,9 @@ logging:

Before you begin, find the line beginning with `ExecStart=` at the very bottom
of the `[Service]` section in the unit file. Depending on your installation path
you might need to change the location of the `threatbus` and `pyvast-threatbus`
you might need to change the location of the `threatbus` and `vast-threatbus`
executable packages and configuration files. Similarly, you need to change the
environmentvariables `THREATBUSDIR` and `PYVAST_THREATBUSDIR` according to your
environmentvariables `THREATBUSDIR` and `vast_THREATBUSDIR` according to your
installation paths.

- Threat Bus
Expand All @@ -45,17 +45,17 @@ installation paths.
ExecStart=/installation/path/threatbus --config=/installation/path/threatbus/config.yaml
```

- `pyvast-threabus`
- VAST Threat Bus
```bash
Environment="PYVAST_THREATBUSDIR=/installation/path"
ExecStart=/installation/path/pyvast-threatbus --config=/installation/path/pyvast-threatbus/config.yaml
Environment="vast_THREATBUSDIR=/installation/path"
ExecStart=/installation/path/vast-threatbus --config=/installation/path/vast-threatbus/config.yaml
```

Then copy (or symlink) the unit file to `/etc/systemd/system`.

```bash
systemctl link "$PWD/threatbus.service"
systemctl link "$PWD/pyvast-threatbus.service"
systemctl link "$PWD/vast-threatbus.service"
```

To have the services start up automatically with system boot, you can `enable`
Expand All @@ -64,6 +64,6 @@ them via `systemd`. Otherwise, just `start` it to run it immediately.
```bash
systemctl enable threatbus
systemctl start threatbus
systemctl enable pyvast-threatbus
systemctl start pyvast-threatbus
systemctl enable vast-threatbus
systemctl start vast-threatbus
```
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ After=network-online.target
Type=simple

DynamicUser=yes
LogsDirectory=pyvast-threatbus
LogsDirectory=vast-threatbus

# capabilities
RestrictNamespaces=yes
Expand All @@ -28,7 +28,7 @@ ProtectKernelLogs=yes

# service specifics
TimeoutStopSec=600
ExecStart=/opt/tenzir/threatbus/pyvast-threatbus -c /opt/tenzir/threatbus/pyvast-threatbus.yaml
ExecStart=/opt/tenzir/threatbus/vast-threatbus -c /opt/tenzir/threatbus/vast-threatbus.yaml

[Install]
WantedBy=multi-user.target