ClickHouse - un SGBD open source de Yandex - est traditionnellement utilisé pour l'analyse de divers types de journaux ou de flux d'événements à partir de systèmes en ligne. Cependant, la flexibilité de ClickHouse lui permet d'être utilisé pour une classe plus large de tâches.
Vidéo:
! , ClickHouse . , , ClickHouse – - , , . . ? , , .
– LifeStreet 10 . . ClickHouse . , , Altinity, ClickHouse .
, ClickHouse , . . , . . .
, ClickHouse .
ClickHouse - , . . :
- -. , . .
- AdTech-, . . , . .
- , , . . .
- , DNS, http-. , - .
- . ClickHouse , , .
- .
?
- , ClickHouse . . , Kafka, .
- OLAP-. OLAP? range scan - . , , , , 99,99 % , ClickHouse , . .
, ClickHouse :
- . ClickHouse .
- Key-value.
- .
? , point queries, . . , . , ClickHouse , , .
ClickHouse . , , , . . , ClickHouse , , , , - .
, . . ClickHouse . , .
. . " ". ClickHouse – . , , point queries.
? , . , ClickHouse . ClickHouse .
? . , LifeStreet. HighLoad , - , . . , , 2 . - – 4-5 . , , , 15-20 – . , , , .
? .
. ? , , . - RTB , : , . , , . . .
, ETL. Data Warehouse, ClickHouse. - . , . LifeStreet . .
- , , - . - . , - . : « ?». , , . , , ETL , , . ?
, , . . , . . - – . , -, -. , -, , . ClickHouse.
ClickHouse .
:
- grep - . ? , , . , . . , . , .
- Spark. . , .
- Elastic Search.
ClickHouse. . . - grep Spark. . ClickHouse.
? , , . data-, timestamp, , . , tx. ID . default ParamExtractInt. ? . , .
, , . , . , ETL-, , . , , - . , - , . , , . , log storage.
index_granularity. . DEFAULT. DEFAULT = 8192 ClickHouse. . , – .
DEFAULT visitParamExtractInt? ClickHouse INSERT SELECT . INSERT? INSERT «tx». ClickHouse . row, JSON, ID, . . ClickHouse JSON . , . . ID . , SELECT, , .
, . , , .
? ALTER TABLE DEFAULT. ClickHouse, , , . ALTER . . , SELECT, ClickHouse . .
. ClickHouse , - , . , , , . . . , .
, ? - , . , . ClickHouse 2/3 . , , ClickHouse C++ . , Java .
, . , . , . , . , , . 10 500 – 1,3 , 64 . , brotli, 61 .
ClickHouse, . , ClickHouse. 84 . brotli 61 , ClickHouse – 84 . , . 5 , RAID- . -, 60 RAID-10. . . 30 . , 17 . , 200 .
. ? .
, , , - ClickHouse. page cache, ClickHouse , . , , , , . , , .
, , , , , - . .
, , . . - - . , . . . 
, , . . , . 17 . . , .
. - .
-, . 15-20 . ClickHouse . , , N-. , , ClickHouse . , . .
. ClickHouse - . , . , , .
index_granularity, 1024 , , ? ClickHouse . , , N-.
N = 8192. . .
? , , . . - , ClickHouse . . . , + - . . . , . . - .
, , index-granularity , , , , . - , 8 , 8 . 8 granularity, 8 . . . .
. , granularity – - .
, . . , , 3 . , 8 . , ClickHouse , 8 . 2,5 . , 8 .
. . . 17 . , 12 . - ClickHouse point queries , . , , , , . 12 – .
. , - . ?
- .
- . - .
, . .
, in-memory storage, . request, , , . . , .
, , , , , 30 , - . - , , . , , , 30 . , . , ClickHouse.
ClickHouse . -, , user ID . AggregatingMergeTree.
AggregatingMergeTree – ClickHouse, , . , . . , .
, . argMax. , . – . argMax . - .
AggregatingMergeTree ClickHouse, state , . . - , .
, , . . . – .
- , . . ? user id . - .
, user ID, . , , . . . , in-memory-, ClickHouse.
, , :
ClickHouse JSON. , - JSON.
Point-queries – , . . . ClickHouse, , , , - , .
, , . , - , , JSON. JSON - JSON , , , . , . - 80 ClickHouse. , . , , . , . point-queries, , .
, ClickHouse , , , , Point-queries .
AggregatingMergeTree , , .
. - ClickHouse.
, . , - . . - . – , .
. . - . , - . , . , , . ., , . , . . . data points, .
ClickHouse? ?
. . , , . . . . . .
«Engine = Memory». , . . .
? , – K, K+1. . – . .
ClickHouse? ?
. ClickHouse . .
– , . , , HighLoad. , - subids. , name-value pairs. . .
.
? . . , . .
? , A Programming Language, . . APL. 1960- . . , , .
. ? . , . . APL, J. , , . J, , .
. , -, 70- . . , KDB+. , in-memory . J, K. .
: « ?». ClickHouse. , . , .
ClickHouse, . - - - . , , . , .
ClickHouse, , -. , -, , . . , , ID. .
1. C , . , . . . arrayMap , . : C K. .
: S - . . ClickHouse.
, , . , C(i) . S – , . V – , . , , . . , .
. SQL- , : , . , ClickHouse.
- .
- , .
- . , , C++. ClickHouse , ClickHouse.
ClickHouse . , . , arrayDifference .
. . . arrayDifference. , .
, , , ClickHouse. arrayDifference, pool request. , . ClickHouse «arrayDifference». . .
- ClickHouse – OLAP-. , ClickHouse – , . .
- ClickHouse – . DBMS , . , , , , , JOIN.
- , , , . .
- , , , . ClickHouse – , , , , , . . , , , .
- . — https://t.me/clickhouse_ru.
ClickHouse . !
.
! , , ? , ?
. . . , . – , . ClickHouse, , . Oracle Postgres, . , , , .
max state. , , . ? , , , , . run hash, , ?
, . , . , - . , 5 . . , , .
, ! ? ? ?
, . storage, . , . , 80 , , . . , — , , . , . , , , . , 100 Elastic, , . 5 . 1 000 . ?
!
, . . . , 12-17 , ID, , , JSON?
, ClickHouse. . , , . ClickHouse . , . JSON INSERT. . . ClickHouse JSON INSERT DEFAULT.
, INSERT , , , , .
ALTER TABLE, . , INSERT , , , .
. !
«argMaxState». , . , , , ? . ?
TS – DateTime. , , , TS , .
. . ?
DateTime – , . , - . , , .