Introduction

A query engine over semi-structured (JSON) logs.

Similar to trino, but doesn't require a table's schema (column & types) before executing a query.

While trino receives SQL and starts returning results once the entire query finishes (batch ETL), miso's query API receives a KQL query (think SQL but with bash like pipes, see example below), and streams back the results using SSE (stream ETL).

It supports the same optimization based predicate pushdown mechanism in trino, so a query transpiles as many query steps as its connector supports into the connector's query language, returning fewer documents over the network (which is usually the bottleneck), and utilizing the connector's indexes, making queries return much faster.

Here's an example of a query supported today by miso (qw is a Quickwit connector to localhost:7280/, es is an Elasticsearch connector to localhost:9200/):

# curl supports SSE by adding the -N flag.
curl -N -H 'Content-Type: application/json' localhost:8080/query -d '{
  "query": "
    qw.hdfs
    | union (es.hdfs)
    | where @time > now() - 1d
    | summarize
        minTenant = min(tenant_id),
        maxTenant = max(tenant_id),
        count = countif(severity between (50 .. 100))
      by bin(@time, 1h)
    | join (
        qw.stackoverflow
        | where questionId > 80
      ) on $left.minTenant == $right.questionId
    | top 10 by count desc
  "
}'