Deployment of DealsHistory

Environment variables

For run of DealsHistory you need to set ENV variables:

CONFIG_PATH='settings.json'                       # Path to config file. Ex: https://trading-vault.rock-west.net/ui/vault/secrets/keys/show/accounttype_settings

FETCH_CHANNEL=NULL                                # Channel for fetch data (HTTP or NATS or NULL). Ex: NATS - for NATS, HTTP - for HTTP, NULL - for disabled mode.
FETCH_CHANNEL_NATS_STREAM_NAME=DEALS              # Only for: FETCH_CHANNEL=NATS. NATS stream name (if channel is NATS)
FETCH_CHANNEL_NATS_SUBJECT=DEALS.*                # Only for: FETCH_CHANNEL=NATS. NATS subject (if channel is NATS)
FETCH_DURATION_TIME=60s                           # Only for: FETCH_CHANNEL=HTTP. Period of requests to the server RwManager, example: 60s
FETCH_SYNC_LOCK_DURATION=3m                       # Only for: FETCH_CHANNEL=HTTP. Duration of lock sync group (default = 3m)
FETCH_SYNC_STEP_DURATION=3m                       # Only for: FETCH_CHANNEL=HTTP. Duration of step sync group (default = 3m)

SYNCER_DEALS_ENABLED=false                        # Is enable syncer deals? (default = false)
SYNCER_DEALS_SOURCE_TABLE=deals_sync_data         # Source table name for syncer deals (default = deals_sync_data)
SYNCER_DEALS_TARGET_TABLE=deals                   # Target table name for syncer deals (default = deals)
SYNCER_DEALS_SYNC_DURATION=1s                     # Duration of syncer deals (default = 1s)

CH_DATABASE_HOST=                                 # Clickhouse Host
CH_DATABASE_PORT=                                 # Clickhouse Port
CH_DATABASE_NAME=                                 # Clickhouse Name
CH_DATABASE_USER=                                 # Clickhouse Username
CH_DATABASE_PASS=                                 # Clickhouse Password
CH_DATABASE_READ_TIMEOUT=5                        # Clickhouse Read timeout (default = 5)
CH_DATABASE_WRITE_TIMEOUT=5                       # Clickhouse Write timeout (default = 5)
CH_DATABASE_DEBUG_FLAG=false                      # Clickhouse Is need use Debug mode?
CH_DATABASE_TLS_ROOT_CA_CERT_FILE=""              # Clickhouse path to CA Root Certificate
CH_DATABASE_SCHEME=deals_history                  # Clickhouse Scheme
CH_DATABASE_MAX_PARTITION_PER_INSERT_BLOCK=1000   # Clickhouse Max partition per insert block (default = 1000). We recommend using a value the same as BATCH_WRITE_BUFFER_SIZE
CH_DATABASE_ASYNC_INSERT=                         # Clickhouse Is need use async insert? (default = false)
CH_DATABASE_WAIT_FOR_ASYNC_INSERT=                # Clickhouse Is need wait for async insert? (default = false). It will be used if CH_DATABASE_ASYNC_INSERT=true

BATCH_WRITER_DEALS_ENABLED=false                  # Is enable batch writer deals? (default = false). Need to set true, if you want to use fetcher (via NATS or HTTP)
BATCH_WRITER_DEALS_TABLE_NAME=deals_sync_data     # Batch writer deals table name (default = deals_sync_data). This table will be used for store deals from fetcher (via NATS or HTTP)
BATCH_WRITER_DEALS_TYPE_QUERY="by-one"            # Batch writer deals type query (default = "by-one"). Available values: "by-one", "ch-batch". We recommend using "by-one" for small volumes of data, and highly recommended "ch-batch" for large volumes of data.

BATCH_WRITE_INTERVAL=1s                           # Batch writer interval (default = 1s). When batch will be sent to Clickhouse?
BATCH_WRITE_TIMEOUT=5s                            # Batch writer timeout (default = 5s)
BATCH_WRITE_BUFFER_SIZE=1000                      # Batch writer buffer size (default = 1000). Will be sent to clickhouse, if the length will be the same. Or if the time will be more than BATCH_WRITE_INTERVAL.
BATCH_WRITER_POOL_WORKERS_COUNT=1                 # Batch writer pool workers count (default = 1)

BATCH_WRITER_BONUS_ENABLED=false                  # Is enable batch writer bonus history? (default = false)

INSTRUMENTS_ENABLED=false                         # Is enable instruments? (default = false)
INSTRUMENTS_PERIOD=24h                            # Period of checking/updating instruments in DB (default = 24h)

# Will be used if FETCH_CHANNEL=NATS or SENDER_NATS_ENABLED=true
NATS_JS_ADDRESS=                                  # NATS JS address, ex: nats://localhost:4222
NATS_JS_PASSWORD=                                 # NATS JS password
NATS_JS_USER=                                     # NATS JS user

# It's using in deals databus client
NATS_ADDRESS=                                     # NATS address, ex: nats://localhost:4222
NATS_USER=                                        # NATS auth: user
NATS_PASSWORD=                                    # NATS auth: password


HEALTH_ENABLED=false                              # Is enable health for k8s? (default = false)
HEALTH_PORT=7000                                  # Health port (default = 7000)
HEALTH_ENDPOINT=/health                           # Health endpoint (default = /health)
METRICS_ENABLED=false                             # Is enable metrics for prometheus? (default = false)
METRICS_PORT=7777                                 # Metrics port (default = 7777)

SENDER_NATS_ENABLED=false                         # Is enable sending to NATS? (default = false)
EMITTER_ENABLED=false                             # Is enable emitter to NATS? (default = false)
DEALS_EMITTER_PERIOD=15s                          # Period of deals emitter (default = 15s)
SESSIONS_EMITTER_PERIOD=15s                       # Period of sessions emitter (default = 15s)

Example with parameters for working with ClickHouse and NATS (+ JetStream):

CONFIG_PATH='/opt/config/accounttype_setttings.json'
# Clickhouse
# Local - Docker host
CH_DATABASE_HOST=host.docker.internal
CH_DATABASE_PORT=9000
CH_DATABASE_SCHEME=deals_history
CH_DATABASE_NAME=deals_history
CH_DATABASE_USER=default
CH_DATABASE_PASS=
CH_DATABASE_READ_TIMEOUT=30
CH_DATABASE_WRITE_TIMEOUT=30
CH_DATABASE_DEBUG_FLAG=false
# NATS - Local - Docker host
NATS_ADDRESS=nats://host.docker.internal:4222
NATS_USER=nats
NATS_PASSWORD=nats
# NATS JetStream - Local - Docker host
NATS_JS_ADDRESS=nats://host.docker.internal:4222
NATS_JS_USER=nats
NATS_JS_PASSWORD=nats
NATS_JS_SECURE=false
HEALTH_ENABLED=true
HEALTH_PORT=7000
HEALTH_ENDPOINT=/health
METRICS_ENABLED=true
METRICS_PORT=7777

You can get actual deals by using that SQL request:

SELECT * FROM deals_sync_view ORDER BY ts DESC;

DealsHistory is working on alpine container.

To deploy DealsHistory you have this docker-compose file, where connection to local ClickHouse and NATS is defined. You can edit the file accounttype_settings.json if it is needed. Also you should change {IMAGE}} to docker-compose.yml:

2KB

deals-history.zip

archive

Open

To enable TLS on NATs see the official documentation - https://docs.nats.io/running-a-nats-service/configuration/securing_nats/tls.

To start TLS use this command:

nats-server --tls --tlscert=server-cert.pem --tlskey=server-key.pem

To enable TLS on Deals-History define the environment variables:

Variable	Example of value	Description
NATS_JS_SECURE	true	specifying enabling TLS in deals-history
NATS_JS_TLS_CLIENT_CERT_FILE	certs/client-cert.pem	path to the client certificate file. You need to specify your path to the file.
NATS_JS_TLS_CLIENT_KEY_FILE	certs/client-key.pem	path to the file - client key. You need to specify your path to the file.
NATS_JS_TLS_ROOT_CA_CERT_FILE	certs/rootCA.pem	path to the file - root certificate. You need to specify your path to the file.