Skip to content

Latest commit

 

History

History
72 lines (54 loc) · 4.27 KB

MongoDB-batchsink.md

File metadata and controls

72 lines (54 loc) · 4.27 KB

MongoDB Batch Sink

Description

This sink writes to a MongoDB collection.

Configuration

Reference Name: Name used to uniquely identify this source for lineage, annotating metadata, etc.

Host: Host that MongoDB is running on. Enter the hostname only. The connection string is constructed by the plugin.

Port: Port that MongoDB is listening to.

Database: MongoDB database name.

Collection: Name of the database collection to write to.

ID Field: Which input field should be used as the document identifier. Identifier is expected to be a 12-byte value or it's 24-byte hexadecimal string representation. If none is given, an identifier will be automatically generated.

Username: User identity for connecting to the specified database.

Password: Password to use to connect to the specified database.

Connect Using SRV String: Toggle to determine whether to use an SRV connection string for MongoDB. It can be enabled if the MongoDB deployment supports SRV DNS records for connection resolution.

Connection Arguments: A list of arbitrary string key/value pairs as connection arguments. See Connection String Options for a full description of these arguments.

Data Types Mapping

| CDAP Schema Data Type | MongoDB Data Type     | Comment                                            |
| --------------------- | --------------------- | -------------------------------------------------- |
| boolean               | Boolean               |                                                    |
| bytes                 | Binary data, ObjectId | 12-byte value of the identifier field is mapped to |
|                       |                       | ObjectId.                                          |
| string                | String, ObjectId      | 24-byte hexadecimal string value of the identifier |
|                       |                       | field is mapped to ObjectId.                       |
| date                  | Date                  |                                                    |
| double                | Double                |                                                    |
| decimal               | Decimal128            |                                                    |
| float                 | Double                |                                                    |
| int                   | 32-bit integer        |                                                    |
| long                  | 64-bit integer        |                                                    |
| time                  | String                | Time string in the following format: HH:mm:ss.SSS  |
| timestamp             | Date                  |                                                    |
| array                 | Array                 |                                                    |
| record                | Object                |                                                    |
| enum                  | String                |                                                    |
| map                   | Object                |                                                    |
| union                 |                       | Depends on the actual value. For example, if it's  |
|                       |                       | a union ["string","int","long"] and the value is   |
|                       |                       | actually a long, the MongoDB document will have    |
|                       |                       | the field as a 64-bit integer. If a different      |
|                       |                       | record comes in with the value as a string, the    |
|                       |                       | MongoDB document will end up with a String for     |
|                       |                       | that field.                                        |